Problems with a large number of files

Apr 6, 2011 at 5:57 PM

I'm working on some code that I want to run on our web server to allow users to delete PDF documents from our online library of articles.  So I thought before deleting the selected files, it would be a good up to zip them up and send them to the user.  The following code works great when I'm dealing with small number of files (~100).  But when I tried with a few thousand for testing reasons, I just get 56 byte file that is obviously not right.  I'm not sure what's wrong here, but I'm sure someone can show me what I'm doing wrong.


Response.BufferOutput = false;

System.Web.HttpContext c = System.Web.HttpContext.Current;

var downloadFileName = string.Format("LitRackBackup-{0}.zip", DateTime.Now.ToString("yyyy-MM-dd-HH_mm_ss"));
Response.ContentType = "application/zip";
Response.AddHeader("Content-Disposition", "filename=" + downloadFileName);

using (var zip = new ZipFile())
    foreach (MissingDoc deleteDoc in deleteDocs)



Apr 6, 2011 at 8:10 PM

Hmm, your code looks fine. Not sure why you'd see the behavior you are seeing. You might want to explore where the boundary is - where does it stop working and start failing?  At what number of files, or at what size download?

There were some problems with large downloads from IE. I can't find the article now that describes the problem.  This was an IE problem not a server issue. If this were what you were seeing, then you should be able to succeed downloading using Firefox, or some automated HTTP client (like a .NET app using System.Net.HttpClient or whatever the namespace is).

If you have your app hosted by a service provider, they may have a max download size limit set for your app.  I don't know how to set this in IIS but I'm pretty sure it can be done.  Maybe check that.

Also - depending on the client, sometimes the streamed downloads don't work properly. Mac browsers can download, but then the zip utility builtin to MacOS cannot open the streamed zip.  This has been discussed before here on the forums, maybe search the archives ifyou wanna learn more. The way to avoid this problem is to write the file to a temporary location on the server, then download the file after you've saved it, then later clean up (delete) the temporary file from the server.


Apr 6, 2011 at 8:20 PM

Thanks for the feedback.  I'll check out the boundaries and see if I can narrow it down.

As for large downloads, I'm not sure what the size is, but it shouldn't be larger than a couple hundred megabytes.  The size of the entire repository is 471MB and I'm wanting to zip up about a third of those (PDF files).  I just checked and it fails on Firefox also.  Just the 56 bytes I'm getting on IE.

It's not hosted.  Currently it's running locally on my machine for testing.  Then it will be moved to our development server.  There are no limits set unless they're a default value.  Still, I'll move it to the development server to see if that makes a difference.

Don't have to worry about Macs.  This is for our local intranet and we're a Microsoft shop.

Thanks for the tip about save the file on the server, then sending it.  I'll get it a shot.

Apr 14, 2011 at 6:22 PM

By the way, I found the problem.  It was completely unrelated to my zip code.  Due to another bug, I was trying to put the same file in the zip twice.

Apr 15, 2011 at 10:46 PM

Hmm, glad you found the problem, but ... I think that adding a file twice should fail in a clearer way.

Apr 15, 2011 at 10:51 PM

That would have been nice.  It was throwing an ArgumentException.  But for some reason I don't recall off the top of my head, that wasn't stopping my code.  It appeared to run fine.  It was only stepping through with the debugger that finally opened my eyes. And like I said, it was a bug in the other portion of my code that had nothing to do with zipping.  I was building my list of files incorrectly.


On a completely unrelated note, I was working on this yesterday when I heard someone a couple cubes down raving to someone else about Ionic.Zip!  He's using it to unzip files for something unrelated and was just loving it.  And it was someone else we work with who showed him.  :)

Apr 16, 2011 at 9:29 AM

Ah, terrific!  Glad to hear it!

On the unrelated note - now you know the guy who wrote it.  I'm a Steelers fan, and I wrote it because I hated the lack of a workable, rational free zip lib for .NET. Then it just took off and got a life of its own.

Anyway, good luck.

Apr 19, 2011 at 8:17 PM

Thanks!  Nice to meet you too.  Though I gotta say I sure hope I don't need to return here anytime soon! ;-)