Zipping of logfile been writed - password error ?

Jul 22, 2011 at 12:56 PM
Edited Jul 22, 2011 at 1:19 PM

I have a logfile which i zip with the DotNetZip-library including a password. If i write to the logfile during the zipping progress, the password don't works anymore; but i don't get a zip-error during zipping.

Is it allowed to zip a file, which is currently accessed somewhere from within the application ?

Coordinator
Jul 23, 2011 at 2:19 PM

Well, it's "allowed", depending on how your application handles file sharing (read about FileShare in .NET).  But if the file is actively being modified, you may not get you the results you would like.

What kind of encryption do you use?  The "regular" PKZIP encryption format specifies that the CRC32 checksum is used in creating the encryption keys . This means that the contents of the file or entry must be checksummed BEFORE the encryption begins. This is really bad for performance, but it is not the problem of DotNetZip - it is just the way the specification is written.  In practice, DotNetZip reads the entire contents of the file twice - once to perform the CRC, and then the second time to actually compress and encrypt the data .  If the file is modified after the CRC is calculated, but before the encryption is completed, then you will have the problem you described.  The password will not match during unzipping, because the CRC will be incorrect.

You can try AES encryption - it does not suffer from this "flow twice" problem.  A better approach might be to zip a stable copy of the file - something that is not being changed as you try to read it.   Simply copy the logfile to a temp location, zip it, then delete the temp file.

 

Jul 23, 2011 at 7:07 PM

I use default encryption (i don't set encryption actively)

Maybe i will cancel the password-encryption of the zip archive, so i don't run into problems during unzip :-)

Thanks for helping !

Coordinator
Jul 23, 2011 at 10:09 PM

Yes- not encrypting the entry should also work fine.  Keep in mind that if you are reading the file while it is being changed, the thing you have zipped may not correspond to any particular point-in-time state of the original file.   If the file being changed (and zipped) only gets appends, then it's no problem.  If you're changing the file in the beginning, as well as at the end, then you may run into problems with the zipped content. 

 

Jul 24, 2011 at 8:20 AM

>>> Keep in mind that if you are reading the file while it is being changed, the thing you have zipped may not correspond to any particular point-in-time state of the original file.

Thats correct but doesn't matter for my application.

>>> If you're changing the file in the beginning, as well as at the end, then you may run into problems with the zipped content. 

I use log4net with a RollingFileAppender (zipping the whole log-directory) - maybe i run into problems, when log4net is doing a rollover... (so i could copy the logfile(s) to a temp location...)

Jul 26, 2011 at 12:06 PM

I've got similar problem with zipping log files using encryption. Everything works fine when log files are updated rarely but when there are many IO operations zip file is created without password. Moreover, zip file is built from copied tmp files using background worker that runs once per 30s. What's strange this problem doesn't occur on every OS I've tested. Is there a possibility that OS IO operation's speed may affect this behaviour?

Coordinator
Jul 26, 2011 at 10:16 PM

Without understanding the way you are creating zip files in more detail, I can't say.  I am imagining that you are copying the log files to a temp file, then zipping the copy.  This copied file is not being modified during the zipping process, if i understand correctly.  you're telling me that "sometimes" the encryption doesn't happen.  That sounds very odd, and ... impossible.  But I'm sure there is something I don't understand about what you are doing.

As regards to whether the OS IO speeds can affect the behavior you describe - I don't see how that would cause the zip file to be created without a password. That makes no sense to me.  I cannot envision a scenario where the IO speed causes encryption to "vanish".  So.... no.

There is a feature in DotNetZip where the app can completely SKIP the inclusion of a file if DotNetZip encounters an IO error while reading the input.  Say, for example, your app creates a zip file and add 10 files into it.  Then it calls ZipFile.Save().  The library then reads each file in succession and writes to the output zip file.  If there is an IO error reading one of the files, it's possible for DotNetZip to notify the application  (via the ZipError event) and the application can then advise DotNetZip to skip the file (or retry, or cancel the save entirely.... etc).  

I can imagine a case where a VERY busy IO subsystem could cause a read to fail, and then DotNetZip would detect it, notify the app via the ZipError event, and then the app could advise "Skip".  In this case that entry in the zip file would not be present at all.  It's not possible for the app to change or turn off the encryption on an entry, in this ZipError event. 

Sounds like something else mysterious is going on in your case.