ZipInputStream -> ZipOutputStream Compression Issue

Jul 27, 2011 at 5:53 AM
On every run output from ZipOutputStream increasing file size (no matter which compression level selected). and if i use same file again it will increase in size again.... don't know how to address this issues!?


byte[] buffer = new byte[67108864];

while (UnzipStream.Read(buffer, 0, buffer.Length) > 0)
{
ZipStream.Write(buffer, 0, buffer.Length);
}
 Not sure how to address this issue since Cheeso mentioned this is not a bug any help with that?
 http://dotnetzip.codeplex.com/workitem/13971
Coordinator
Jul 27, 2011 at 12:34 PM

Cheeso mentioned that it is a question, and is better dealt with on the discussion forums, than in the bug database.  If it does prove to be a bug you or I can create a workitem. but first we need to talk about it.

You'll need to be more clear on what you are asking.  "on every run output from ZipOutputStream increasing file size" - sounds to me like the ZOS is not compressing for you.   What kind of data are you running through the stream?  What does it mean when you say "on every run"?  Does that mean, "with any input"? or "with any file"?  or something else?  Are you trying to re-compress already compressed data?

In the code fragment you provided, what is the reason behind using a buffer of size 67108864? Why so large a buffer?    Can you provide some background on what you are doing, exactly? 

Jul 27, 2011 at 5:45 PM
Edited Jul 27, 2011 at 6:51 PM

Thanks Cheese for looking into it!

           Are you trying to re-compress already compressed data? Yes. Basically using output archive to open and re compress again through ionic zip stream.

           Can you provide some background on what you are doing, exactly?  On current stage read zipped .csv files uncompress, compress and write back on disk with various buffer size from 1024 (*2) up to 64 MB.

           Why so large a buffer?  I have to decompress - compress files which could be ~ 40 GB originally, some logic should be applied to the data but for now I'm not using anything to see bare performance issues with different buffer settings.

 

   Thank you for helping!

 

Coordinator
Jul 27, 2011 at 7:14 PM

Ah, ok I understand now that you are trying to compress data that is already compressed.  This can and will lead to an increase in data size.

That seems to settle the mystery.  This is not a bug in DotNetZip code, this is an artifact of how compression works.

I suggest you compress ONCE. 

 

Jul 27, 2011 at 7:30 PM

Thank you Cheeso! will do as you suggested.