Why files over 500 MB making problems?

Nov 10, 2008 at 12:58 PM

I wish to write a app for me, that get´s my database backups (Zip File) and extract them.
The Files are over 500 MB big.
Every time I try to extract the file, I get the Error:
"xxx is not a valid zip file".
With lower files, e.g. 20 MB I have no problems.

Can anyone help me?
Nov 10, 2008 at 6:44 PM
If you give me a sample of a zip file that causes the problem, I can look at it.
I realize there may be data privacy issues.

Also, how did you create the zip file?  Did you use DotNetZip?  Or was it created with some other tool or library?
if so , which one? 

Nov 10, 2008 at 7:23 PM

I´m sorry but I can´t give you an exmaple, we must do it without.
It was created by a automaticly WinRAR Script, but it is a ZIP File.
Its 550 MB big and extracted over 4-5GB
Nov 10, 2008 at 8:55 PM
Unfortunately I will have a tough time understanding what is wrong with the zipfile without even a stack trace of the error.
To track it down, I really need the actual zip file that causes the problem.
Nov 12, 2008 at 10:19 AM
For what it's worth, I tried extracting a 954MB text file compressed using WinRar (Best-zip compression) and it extracted just fine. The zip file size was 948KB and only contained that one text file. There might be something wrong with your script and the file is actually using rar compression. When I use a rar-compressed file I get that exact error.

Nov 16, 2008 at 2:31 PM
I think it is a common issue with WinRAR

Had the same errors with the application AutoUpdate+ that it could not unzip Zip files created with WinRar.
Try creating the same Zip file manually and zip it with the windows native Zip program. It should be able to extract it.
Nov 16, 2008 at 3:00 PM
It could be, as JohnAndre suggested, that the file is not actually a zip file, compliant with the spec, but is instead a compressed archive that only WinRAR understands. 

If I had the zip file in my hand, I would be able to tell.
Without it, I cannot fix the problem.
Nov 18, 2008 at 10:03 AM

I found a little bit more:

The file over 500 MB can not make to a ZIP File, either with WinRAR nor with Windows native ZIP.
Every time I get the message "It´s too large".
So I think it can be a RAR-File that is renamed or something mysterius else.
Nov 18, 2008 at 10:03 AM
Edited Nov 18, 2008 at 10:24 AM
Here are the Stack Trace:

   at Ionic.Utils.Zip.ZipFile.ReadIntoInstance(ZipFile zf)
   at Ionic.Utils.Zip.ZipFile.Read(String zipFileName, TextWriter statusMessageWriter, Encoding encoding)

And here are the Excaption:

InnerException = {"  ZipEntry::Read(): Bad signature (0x7B99EC00) at position  0x00000050"}
Nov 18, 2008 at 4:41 PM
Kovu, thanks -
That file appears to be either
  • corrupt, or
  • not a zip archive.  It could be a zip64 archive, which this library does not support.

Nov 18, 2008 at 6:48 PM
Edited Nov 18, 2008 at 6:51 PM

To check if it's a ZIP file, and if you find a way to open your file with text editor (not sure that you can open a 500MB file), check the first 2 chars of the file, if you have PK, that's mean that this is a PKZIP file (so a ZIP file).

A rar file begins with : Rar

As Cheeso says, even if it's a Zip file, the file can be corrupt.

Winrar should be able to open a Zip64 archive.
Nov 19, 2008 at 2:39 AM
DomZ - that's right.
But if the error Kovu is seeing, is a "bad signature" at position 50 in the file, then the PK characters have been found in the header of the file.

Nov 19, 2008 at 6:47 AM

This is one of this large files in WordPad. You see: It begin with "PK".
So it seems to be a ZIP.

No other DotNet ZIP Libary can open it, I try some of them.
The Error-Handling in them seems worse then yours and throw a default error.

Nov 19, 2008 at 3:34 PM
If no package can open it, then it seems unanimous that it is not a valid zip file.
it seems to be corrupted.

if you could give me a hex dump of the first 0x200 bytes I would be able to tell you more.

There are "Zip fix" tools out there, I think. I've never used any.  But they say it is possible to repair corrupted zip files.

Nov 20, 2008 at 7:57 AM
I don´t think it´s corrupt.
Because I can unzip it manualy with WinRar and Windows Native Zip. But not zip again, because it´s to large.
Nov 20, 2008 at 12:45 PM
Ok, in the "Info" Button in WinRar I found out that is a ZIP64 File!
Is it possible to get it extract?
Nov 20, 2008 at 4:20 PM
no, DotNetZip does not do Zip64. 
It is an active workitem.

One of the first things I will do is modify the library to detect zip64 and throw a proper exception, noting something like "Zip64 is not yet supported."
Dec 19, 2008 at 2:26 PM
Kovu - can you get me that zip64 archive produced by WinRAR?
I'd like to try to embed support for zip64 in the library, and I need tests.