Slow Read on Zip files with a large number of entries

Feb 5, 2015 at 8:48 PM
I'm working on application that is randomly required to retrieve a small file from a set (20ish) large (500MB) ZiP files each of which may contain 250,000 or more entries.

I have my Ionic zip working and it is a very simple program. The problem is the program grinds to halt for over a minute at the line

Using zip1 As ZipFile = ZipFile.Read(ZipToUnpack)

As a user is waiting for the returned file this is just too long and I need to get it down to <10seconds

I had considered opening all of the archives and keeping them open but I have a feeling that this is going to be a very bad thing to do especially as the number of archive zips I have to access will grow with time.

As the overall number of files in the zips is in millions I thought it a neat solutions not to unpack them especially as the request for files will be few and random. However, I need to be able to get one file out in an acceptable amount of time.

I rather assume the line of code above is loading the entire 250,000 entry directory.

Do you have any thoughts of suggestions on this?

Feb 12, 2015 at 12:37 PM
For the benefit of others I have overcome this problem. Perhaps not in the way I hoped.
I have the sadly found the best answer for me was not to use ionic zip....

System.IO.Compression extracted the file in <10seconds

I suspect the .net native solution came along after ionic but there you go ... that's software (-:

 Using archive As System.IO.Compression.ZipArchive = System.IO.Compression.ZipFile.OpenRead(ZipToUnpack)

                            Dim entry As ZipArchiveEntry = archive.GetEntry("FileName.txt")
                            entry.ExtractToFile(System.IO.Path.Combine(TargetDir, "FileName.txt"))

  End Using