Out of memory exception with large zip files

Jun 9, 2011 at 5:06 PM

I have used this great zip dll for a while in my projects, however an issue has started appearing over the last few days I am unable to fix. An application i wrote zips up files and downloads them via ASP.NET using the code at the bottom of the post. There seems to be an error when I add other .zip files into my Ionic.Zip object, and causes a System.OutOfMemoryException. I've tried adding changing the compression level and setting the parallel defalte option as suggested in other posts but this doesn't work either. The zip files I am downloading are between 150 - 400MB (total 1.3GB), I've tried selecting only 2 of the smallest which had a total file size of 330MB and although this didn't give any exceptions it didn't appear to do anything, the browser tab keep processing for 2 hours until I closed the tab without actually starting a download (I'm using firefox and the little processing icon kept spinning).

 

protected void StreamFilesIntoZipFile(string zipFileName, List<string> filePaths) {
            string displayFilename = HttpUtility.UrlPathEncode(zipFileName);
            Response.AddHeader("Content-disposition", "attachment; filename=" + displayFilename);
            Response.ContentType = "application/zip";

            using (Ionic.Zip.ZipFile zipFile = new Ionic.Zip.ZipFile()) {
                zipFile.CompressionLevel = Ionic.Zlib.CompressionLevel.BestSpeed;
                zipFile.ParallelDeflateThreshold = -1;
                //Ionic.Zip.ZipFile zipFile = new Ionic.Zip.ZipFile();
                foreach (string filePath in filePaths) {
                    zipFile.AddFile(filePath, "");
                }
                zipFile.Save(Response.OutputStream);
            }
            //Response.Close();

        }

Jun 9, 2011 at 9:42 PM

I am a new user to this library and have also noticed something similar. I am processing a large number of zip files.. and even though I discard that object, it appears as though the application doesn't release the memory. Could there be a memory leak in this release? Over time, I will too receive an OutOfMemoryException because I never regain the RAM from discarded ZipFile objects.

Coordinator
Jun 18, 2011 at 7:10 AM

I suppose the OutOfMemory problem is occurring because by default ASPNET buffers all the output before sending the first byte.  This means if you zip a 1.3gb file, ASPNET tries to hold the compressed version of that thing into a memory buffer. 

I know of two good ways to avoid this:

- turn off response buffering.  See here: http://stackoverflow.com/questions/2277704/enable-chunked-transfer-encoding-in-asp-net/2277764#2277764

- save to a filesystem file, then send that file down to the client.

 

Good luck.

 

Apr 10, 2012 at 6:25 PM

I have exactly the same problem as nullvalue. When I create a large number of zip files, the application memory just grow until I get an OutOfMemoryException.

Any solution or workaround to this issue?