IOException With Large Zip and Many Files

Dec 5, 2008 at 3:57 AM
Edited Dec 5, 2008 at 4:00 AM

I get this exception:

IOException: An attempt was made to move the file pointer before the beginning of the file.

It appears to be this code: (inside ZipEntry._WriteFileData() method)


int n = input1.Read(buffer, 0, READBLOCK_SIZE);
while (n > 0)
{
        output2.Write(buffer, 0, n);
        n = input1.Read(buffer, 0, READBLOCK_SIZE);
}

 

I am trying to zip up about 6 GB of .JPG files among many other MISC files.

I'm setting ForceNoCompression = true (also occurs with ForceNoCompression = false).

After the exception, the zip file I was trying to create is 2,153,996,868 bytes in size. (Approx 1/3rd through source files).

I believe this is completely unrelated to the OutOfMemory Exception that was addressed in v1.6.

I'll see if I can track this issue down further than this. I am using v1.6.3.14 of the library.
Coordinator
Dec 5, 2008 at 6:06 AM
whoops! 
file too large. 
You need zip64 support. which the DotNetZip library currently lacks.

Or at the very least, a sensible exception saying  "too much data to zip!"
Dec 6, 2008 at 3:33 PM
What needs done to address this? How can I assist in making this feature become part of the library?

Thanks,
~Nate
Not sent from an iPhone

-----Original Message-----
From: "Cheeso" <[email removed]>
To: [email removed]
Sent: 12/5/08 12:06 AM
Subject: Re: IOException With Large Zip and Many Files [DotNetZip:41546]

From: Cheeso

whoops! 
file too large. 
You need zip64 support. which the DotNetZip library currently lacks.

Or at the very least, a sensible exception saying  "too much data to zip!"

Coordinator
Dec 6, 2008 at 5:35 PM
Edited Dec 12, 2008 at 4:21 PM
For archives that have entries of more than 4g in size, or
for archives themselves that are over 4g in size,
the zip archive needs to conform to the zip64 extensions to the zip spec.
The extensions to the zip spec are pretty straightforward.
It won't be a difficult task for me to do this. But it requires thought and testing.
For me, it's a matter of time and priorities. I'll have time later in December to do it.
What I need are sample zip64 archives that I can use to test the library against.
There was another person who even submitted some code that he said fixed the problem. And he was right, for the limited thing he was trying to do. But those changes did not deliver general support for zip64, which is what people want. There was no error checking, there were no tests submitted with the changes, there was no support for writing large archives, there was no way in the programming interface to query if zip64 was in use, and so on. The changes worked for his scenario (I think it was reading a specific archive), and I'm glad the library worked for his purpose. But to address what people really want - zip64 in all scenarios (writing, updating, reading broken or corrupted archives, etc) requires more changes.

Dec 6, 2008 at 7:06 PM
Hi Cheeso,

I don't know if you see my post (in another thread).
I have post an url to download a ZIP64 archive sample :
http://www.archive.org/details/test_zip64
Dec 6, 2008 at 8:58 PM
If you need help testing the zip library's creation of zips with large datasets let me know, i'll be happy to help test/debug.

Thanks,
~Nate
Not sent from an iPhone

-----Original Message-----
From: "DomZ" <[email removed]>
To: [email removed]
Sent: 12/6/08 1:06 PM
Subject: Re: IOException With Large Zip and Many Files [DotNetZip:41546]

From: DomZ

Hi Cheeso,

I don't know if you see my post (in another thread).
I have post an url to download a ZIP64 archive sample :
http://www.archive.org/details/test_zip64

Coordinator
Dec 12, 2008 at 4:22 PM
OK, I will let you know.