MDF file is corrupt when extracting

Aug 10, 2011 at 6:13 AM
Edited Aug 10, 2011 at 6:28 AM

I have two files that i need to add to one zip file. One is a .mdf file and the other is a .ldf file. This is my code:

 

                string AppPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\PGC\LandscapeBilling\";

using (ZipFile zip = new ZipFile())
{
zip.AddFile(AppPath + "Billing.mdf", @"C:\");
zip.AddFile(AppPath + "Billing_log.ldf", @"C:\");
zip.Save(path);
}

 

Anyway everything works fine until I try and unzip it. The .ldf file is fine but the .mdf file always says that the CRC has failed and the file is corrupt and it wont work. Whats wrong?

Coordinator
Aug 10, 2011 at 2:29 PM

Hmm - how do you unzip the file? What tool are you using?

also I think the path you want for that second argument, might be "" or "\", but not c:\ .

That is the path used internally in the zipfile. It's bad practice to put a drive letter in there. I don't recall if DotNetZip checks for that, but you shouldn't do it. I don't know if it would cause a CRC failure, though...

Also, you may want to look into Path.Combine(), instead of using string concatenation (AppPath + "Billing.mdf"). That's just a style thing. It won't affect the CRc.

can you try again with those changes and let me know.

Aug 10, 2011 at 5:36 PM
Edited Aug 10, 2011 at 7:55 PM

ok so here is the updated zip method

 

                string AppPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\PGC\LandscapeBilling\";

using (ZipFile zip = new ZipFile())
{
zip.AddFile(Path.Combine(AppPath, "Billing.mdf"), "");
zip.AddFile(Path.Combine(AppPath, "Billing_log.ldf"), "");
zip.Save(path);
}

 

I use WinRar and it is telling me that there still is a CRC problem and that the mdf file is corrupt

I also tried to use DotNetZip to unzip it but there is still the error. Here is the code i use.

 

                string AppPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\PGC\LandscapeBilling\";

string zipToUnpack = path;
string unpackDirectory = AppPath;
using (ZipFile zip1 = ZipFile.Read(zipToUnpack))
{
// here, we extract every entry, but we could extract conditionally
// based on entry name, size, date, checkbox status, etc.
foreach (ZipEntry e in zip1)
{
e.Extract(unpackDirectory, ExtractExistingFileAction.OverwriteSilently);
}
}

Also this happens with all .mdf files i have tried to zip. I also am having a problem when trying to zip 3 files together, 1 .ldf file and 2 .mdf files. In this situation the program freezes before finishing making the file while the zip is still named DotNetZip-enldgwfo.tmp and i have to end the process because it is frozen. If i just zip the two mdf files it does finish zipping but both files are still corrupt with a CRC fail message. It doesnt matter what the third file is ive tried txt files instead of the ldf file and it still freezes.

Coordinator
Aug 11, 2011 at 6:16 AM

Are your MDF files OFFLINE when you zip them?  It's possible the db server locks them and prevents a proper read, unless you take the db offline completely.

I'm baffled as to why there'd be CRC errors otherwise. DotNetZip's been tested with many very very large files, so I don't think it's simply a bug in the library relating to large files. I think it must be something specific to the MDF.  So far nothing is seeming really obvious to me, though.

Aug 11, 2011 at 6:27 AM

they are not big files. Less then 3 MB. And no the files are not active / being used by another device. I can use winrar to zip them up and it works fine.

Aug 11, 2011 at 6:45 AM

here is a link to the file if you want to do some testing https://rapidshare.com/files/4286247533/Billing.mdf

Coordinator
Aug 11, 2011 at 11:21 AM

I can't download anything from that URL. 

Not sure what the problem is - I click download, nothing happens.

Aug 11, 2011 at 1:25 PM
Edited Aug 11, 2011 at 2:03 PM

I have the same trouble with MDF and LDF files too. The database has been detached from sql, but every time I get a crc check failed, both in winzip and winrar.

I suspected that the sql server still might have a lock on the file after detaching the database, but I waited 15 seconds before starting zipping the files without any luck.

All other files get in the zipfile fine...

**Update**

When I set CompressionLevel to CompressionLevel.None, I have no issues with the CRC check. I also tried renaming the files, but it had the same result: crc check failed.

Coordinator
Aug 11, 2011 at 3:06 PM

Can you give me a sample .mdf file that causes the problem?

Aug 11, 2011 at 6:18 PM

I dont know why the rapid share didnt work cause it works for me... but here try this link: http://studentweb.cencol.ca/300527709/random/Billing.rar I used winRar to compress this so it should work.

Coordinator
Aug 11, 2011 at 6:42 PM

k, that worked.

I just zipped and unzipped that file with DotNetZip, had no problems. 

Also tried WinRar to unzip the zip produced by DotNetZip - no problems.

Can you try with the zipit.exe tool that comes with DotNetZip?  Tell me if that code works. 

I don't know what problem you're experiencing, but I'm not seeing it. If you have a test case - a short bit of code that reliably reproduces the problem - I'd be glad to look at it.

 

Aug 11, 2011 at 6:59 PM

Yes zipit.exe worked for me too, with the Ionic.Zip.dll. I used the same dll in visual studio and I added the reference. I then added the using statement. Then I used the exact code i sent you before and it always, reliably has the CRC problem.

Aug 11, 2011 at 7:03 PM

here is the exact code i use

 

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using Ionic.Zip;

namespace ZipTest
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                string AppPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\PGC\LandscapeBilling\";

                using (ZipFile zip = new ZipFile())
                {
                    zip.AddFile(Path.Combine(AppPath, "Billing.mdf"), "");
                    zip.AddFile(Path.Combine(AppPath, "Billing_log.ldf"), "");
                    zip.Save(@"C:\a\billing.zip");
                }
            }
            catch (Exception ex)
            {
            }
        }
    }
}

Aug 11, 2011 at 7:16 PM

I can confirm what vwaaijen said. If you set the CompressionLevel to none it works without a CRC fail.

Aug 11, 2011 at 7:26 PM
It happens with any newly created sql 2005 database.
I detach the database and then zip up the mdf and ldf. If you want me
to send an example still, please let me know!
Coordinator
Aug 11, 2011 at 8:36 PM
Edited Aug 11, 2011 at 8:36 PM

ok, Random - your code is silently swallowing exceptions.  In the catch clause - the code does nothing. The program catches the exception and silently moves on.  That's a problem, though I don't know if it is hiding a problem that might occur during zip construction.  But in any case, you need to make a habit of logging every exception. 

When I run your code, with the appropriate logic to print out exceptions in the catch clause, it works.

I don't know what else to tell you - I don't see a problem with the DotNetZip library here.  It sure seems like the problems you are experiencing are due to 

  • a bug in your test code that suppresses errors, OR
  • a situation on your local machine having to do with .mdf files, or that mdf file in particular, OR
  • both?  OR
  • something else specific to your computer

In any case it's not something I can fix.

 (without further diagnostic information from you)

Aug 11, 2011 at 9:23 PM
Edited Aug 11, 2011 at 9:25 PM

this was an example code i was showing you. I didnt want to copy and paste my whole project in here so i just tried to make a small example. I would normally not have a blank exception but it was not throwing exceptions so i didnt think it mattered because it was just a simple example. I removed the try catch and ran in debug mode and it didnt throw any exceptions but the file is still corrupt. This happens with any MDF file i have, not just the one file. I dont know why, if you are doing what I am doing, that you can get it to work and I cant. I am using the exact file i sent to you.

FYI .. I am running a Windows 7 64 bit machine, and Visual Studios 2008 Full version (not express) and the mdf file is a detached database created in SQL Server 2008 Full version (again not express version). I dont know what is wrong.

PS the file is not being used by another process and if I use WinRAR I can fully Zip the file without problems.

And as vwaaijen said if I set the CompressionLevel to none it works without a CRC fail.

Aug 12, 2011 at 7:37 AM
Edited Aug 12, 2011 at 7:57 AM

I think I found the cause of the problem:

I created 2 database: TEST1 and TEST2.

Next, I detached the first database by first putting the database in SINGLE_USER mode and then detaching the database, so all connections were cleared before detaching:

ALTER DATABASE TEST1 SET SINGLE_USER WITH ROLLBACK IMMEDIATE

GO

exec sp_detach_db @dbname = 'TEST1'
GO

The second database I only used

exec sp_detach_db @dbname = 'TEST2'
GO

And then I zipped up the mdf and ldf files from both databases to seperate zipfiles with zipit.exe. Result: TEST1 failed the CRC check and TEST2 went fine. That is probably also the difference in approach you both were dealing with and why it went fine with cheeso, but not with random11 ...

*** UPDATE ***
I also tried the other tools bzip2.exe and gzip.exe and those were both successful, only zipit.exe gave the CRC error when trying to extract the file.

Aug 12, 2011 at 8:27 AM

I did some more troubleshooting:

 

When using the Save() method on the ZipFile, I get the following error when trying to extract with winzip:

 CRC check failed. Expected 7BA23342, actual EFE034C4.

 

But when I use SaveSelfExtractor(@"C:\TEMP\test1.exe", SelfExtractorFlavor.ConsoleApplication) it gives the correct CRC:

C:\TEMP>test1.exe -l
Command-Line Self Extractor generated by DotNetZip.
Extracting Zip file: C:\TEMP\test1.exe

Modified                    Size  Ratio      Packed  pw?      CRC Filename
--------------------------------------------------------------------------------
2011-08-12 08:53:44      2097152    76%      508132    N 7BA23342 test1.mdf

And extraction is also correct. So it looks like there is something going wrong in the Save method on the ZipFile class?

Coordinator
Aug 14, 2011 at 2:07 AM

Very interesting results from the tests.  based on all the observations,  it appears to me that dotnetzip is creating a zip file with a faulty CRC, because the file is being changed, or is range locked, while dotnetzip is reading it.

I cannot explain why a self-extracting archive does not exhibit the problem.  I also cannot explain why WinZip does not have the same problem.

If you can further track this down and isolate just what is happening, I will be happy to look into changing DotNetZip to avoid the problem.  Right now, though, I still don't have a sufficient understanding of the scenario to support such an effort.

 

Aug 16, 2011 at 8:51 PM

You might also try zip.ParallelDeflateThreshold = -1 before calling zip.Save(...); then try unzip it (with the library or with windows zip, winzip, winrar, etc.). Maybe this is just the same problem I discovered with mdb files or pointy with his char file.

Aug 16, 2011 at 9:54 PM
I have turned back to SharpZipLib because that is functioning as I want and I can't delay the release of the tool anymore, but I am still willing and available to get the dotnetzip tool working as it works so much better and easier.

2011/8/16 DexMik <notifications@codeplex.com>

From: DexMik

You might also try zip.ParallelDeflateThreshold = -1 before calling zip.Save(...); then try unzip it (with the library or with windows zip, winzip, winrar, etc.). Maybe this is just the same problem I discovered with mdb files or pointy with his char file.

Read the full discussion online.

To add a post to this discussion, reply to this email (DotNetZip@discussions.codeplex.com)

To start a new discussion for this project, email DotNetZip@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com


Aug 16, 2011 at 11:27 PM

As suggested by DexMik, and looking at the vwaaijen's post above, I think it's the same issue with the file sizes that are multiples of the parallel deflate buffer size (i.e. 128k) as here:

http://dotnetzip.codeplex.com/workitem/14087

 

"C:\TEMP>test1.exe -l
Command-Line Self Extractor generated by DotNetZip.
Extracting Zip file: C:\TEMP\test1.exe

Modified                    Size  Ratio      Packed  pw?      CRC Filename
--------------------------------------------------------------------------------
2011-08-12 08:53:44      2097152    76%      508132    N 7BA23342 test1.mdf"

 

Note that 2097152 = 16 * 131072.

 

Coordinator
Aug 21, 2011 at 6:26 PM

Hmmmmm.... possible.

I will look into it.

Aug 29, 2011 at 12:23 AM
Edited Aug 29, 2011 at 12:24 AM

http://dotnetzip.codeplex.com/workitem/14139

The Code in this workitem plus the attached zipfile produce a similar error while calling zip.Save(...); I just tested it with the file and then added zip.ParallelDeflateThreshold = -1; before calling the Save() and it works fine again...

Just another possible reproducing examble as it seems.

Sep 14, 2011 at 1:54 PM

I get the same issue, on a 400Go archive I get 20+ corrupted files.

with zip.ParallelDeflateThreshold = -1; , no issue.

Coordinator
Sep 14, 2011 at 7:28 PM

Thanks, I'll look into it.

Oct 5, 2011 at 11:31 PM

I am consistently running into this problem, even using the ZipIt.exe tool. Any file that is an exact multiple of 128K fails to extract, even using other extraction tools. At this point it is a blocking issue as ZipIt has no way to turn off the parallel deflate threshold. Any chance of of a fix soon?

Oct 7, 2011 at 8:50 AM

Oh God, it took me forever to find this thread online with a similar issue and i created a CodePlex account just to post.

I am using DotNetZip in a TFS Build Workflow with automated builds, automatically zipping the output of the builds.  Doing this for over 20+ build definitions.  The resulting zip files are used to push to deployment servers...

A portion of the code looks like this:

                    using (ZipFile zip = new ZipFile())
                    {
                        zip.UseZip64WhenSaving = Zip64Option.AsNecessary;
                        zip.AddDirectory(directory.FullName);
                        zip.Comment = "ZIP File Created in TFS 2010 Build Workflow ZipFolder Activity - Folder Compressed : " + directory.FullName + " at " + DateTime.Now.ToString();
                        zip.Save(outputFile.FullName);
                    }

It produces ZIP files every time, no errors..

 

But I just ran into this same exact issue and I know the files I'm compressing arent in use at all, because I just generated them.

When I go to extract the ZIP, CRC error.  Is there a quick fix until a new release comes out?  This seems like a big problem at the core of the DotNetZip library that should be fixed as soon as possible.

Oct 7, 2011 at 9:05 PM

Unfortunately I had to revert back to sharpziplib, which is much slower, but more reliable at the moment...

Op 7 okt. 2011 09:50 schreef "issafram" <notifications@codeplex.com> het volgende:

From: issafram

Oh God, it took me forever to find this thread online with a similar issue and i created a CodePlex account just to post.

I am using DotNetZip in a TFS Build Workflow with automated builds, automatically zipping the output of the builds. Doing this for over 20+ build definitions. The resulting zip files are used to push to deployment servers...

A portion of the code looks like this:

                    using (ZipFile zip = new ZipFile())
                    {
                        zip.UseZip64WhenSaving = Zip64Option.AsNecessary;
                        zip.AddDirectory(directory.FullName);
                        zip.Comment = "ZIP File Created in TFS 2010 Build Workflow ZipFolder Activity - Folder Compressed : " + directory.FullName + " at " + DateTime.Now.ToString();
                        zip.Save(outputFile.FullName);
                    }

It produces ZIP files every time, no errors..

But I just ran into this same exact issue and I know the files I'm compressing arent in use at all, because I just generated them.

When I go to extract the ZIP, CRC error. Is there a quick fix until a new release comes out? This seems like a big problem at the core of the DotNetZip library that should be fixed as soon as possible.

Read the full discussion online.

To add a post to this discussion, reply to this email (DotNetZip@discussions.codeplex.com)

To start a new discussion for this project, email DotNetZip@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe on CodePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at CodePlex.com

Oct 7, 2011 at 9:19 PM
Edited Oct 7, 2011 at 9:19 PM

@issafram - try this:

zip.ParallelDeflateThreshold = -1;

It seems to have fixed the issue for other people in this thread above.

Cheers,

Mike

Oct 11, 2011 at 4:38 AM

@pointy - ParallelDeflateThreshold = -1 does fix the issue, but i looked into this fix and it disables using multiple CPU cores.

While it fixes the issue, it is going around the problem, instead of a good fix.  I am waiting for Cheeso to release a new version that does not need the workaround of disabling multiple core usage.

Nov 3, 2011 at 4:47 PM
Edited Nov 3, 2011 at 4:48 PM

Hello

Just trying out dotnetzip for the first time and I've ran into a similar problem to this except not with MDB files. 

I was zipping RMCOBOL data files and a couple of them were failing the CRC when extracting a zip file (made with dotnetzip) in winrar.

I tried setting the ParallelDeflateThreshold=-1 and that does fix the problem.

I just wanted to point out that I noticed the files before and after using the ParallelDeflateThreshold trick were a different size. The file before setting PDT=-1 was consistently 12,815kb and afterwards the file was 12,788kb. Perhaps that can lead you in the right direction as to what is causing the problem. The Cobol files were definitely not in use when running the zip.

Let me know if you need more information.

Nov 28, 2011 at 4:30 PM

This problem occurs every time I create a zipfile, which includes a dummyfile (fs.create - fs.seek - fs.close).

The CRC which is deposit in the zip is the same as the one of 7zip or windows compression folder.

Could the problem be a NULL in the file? The size of the compressed dummyfile using default compression of DotNetZipLib is bigger than using 7zip or windows compression folder... 

Dec 27, 2011 at 9:52 PM

any update on when this will be fixed?

Coordinator
Dec 29, 2011 at 9:37 PM

Well the disk on my dev machine died a little while ago, and right now I'm working on recovering the data.  Also I've been sort of busy with some unexpected personal things lately. so... unfortunately no, I do not have an estimate on when this will be fixed.  

 

Jan 11, 2012 at 7:48 AM

Hi,

The problem with corrupt zip files (CRC error) occures at what seems random. Sometimes I get a correct zip file, sometimes a corrupted zip file and sometimes the zipFile.Save() never got completeted and froze the application. This with the same source files. Without looking into the source code it sounds like a threading problem. Setting ParallelDeflateThreshold to -1 solved/workaround the problems.

 

Mar 18, 2012 at 6:47 PM

zip.ParallelDeflateThreshold = -1;


Confirmed this workaround for errors occured on compressed sql server backup files.

Apr 17, 2012 at 8:15 PM

I'm having similar, sporadic problems reported to me by my users when zipping sqlite database files.  I tried adding zip.ParalleDeflateThreshold = -1, but that hasn't helped.  Here's the stack trace when I deflate it:

Ionic.Zip.BadReadException: bad read of entry somedb.db3 from compressed archive.
in Ionic.Zip.ZipEntry._CheckRead(Int32 nbytes)
in Ionic.Zip.ZipEntry.ExtractOne(Stream output)
in Ionic.Zip.ZipEntry.InternalExtract(String baseDir, Stream outstream, String password)

Here's how I'm creating it:

using(ZipFile zip = new ZipFile()) {
    zip.ParallelDeflateThreshold = -1;
    zip.AddFile("somedb.db3, "");
    zip.Save("somedb.db3.zip");
}

Any ideas?

Thanks!

Apr 17, 2012 at 11:14 PM
didge wrote:

I'm having similar, sporadic problems reported to me by my users when zipping sqlite database files.  I tried adding zip.ParalleDeflateThreshold = -1, but that hasn't helped.  Here's the stack trace when I deflate it:

Any ideas?

Thanks!

set CompressionLevel to CompressionLevel.None

Apr 17, 2012 at 11:49 PM

Thanks for the suggestion, but I'm not sure how that helps...

Apr 18, 2012 at 3:41 AM
@didge:

I would assume that the command does what it says it does and make no attempt at compression ( as I did not right the code and i haven't tesyed the idea I am not 100% sure but I think it's safe to assume ). Therefore this would only work if your goal is to archive files together in one file and you don't care about having them take up less space. In any case by doing this it will stop currupting your files. As that is what I've been doing and I haven't had a currupted file since.

Credit to vwaaijen who first pointed it out in the 8th comment of this thread.
Jun 4, 2012 at 6:04 PM
Edited Jun 4, 2012 at 6:05 PM
random11 wrote:

I have two files that i need to add to one zip file. One is a .mdf file and the other is a .ldf file. This is my code:

 

                string AppPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + @"\PGC\LandscapeBilling\";

using (ZipFile zip = new ZipFile())
{
zip.AddFile(AppPath + "Billing.mdf", @"C:\");
zip.AddFile(AppPath + "Billing_log.ldf", @"C:\");
zip.Save(path);
}

 

Anyway everything works fine until I try and unzip it. The .ldf file is fine but the .mdf file always says that the CRC has failed and the file is corrupt and it wont work. Whats wrong?

MDF file must be corrupted. You need to professional recovery software. Try to use next one http://www.recoverytoolbox.com/sql.html 

Application will restore your mdf file and save its original structure before extracting

Jun 4, 2012 at 6:10 PM

As said many times in this very thread..... use: zip.ParallelDeflateThreshold = -1;

Jul 31, 2012 at 5:47 PM
sirber wrote:

As said many times in this very thread..... use: zip.ParallelDeflateThreshold = -1;

That's good and fine.. But I'm using the ZipIt.exe tool. Just going thru the source briefly, I don't even see where these tools are being output (they look pre-built).

Same CRC error with Zipit.exe... Thoughts?

May 3, 2013 at 7:46 PM
This is caused by an error in the ParallelDeflate process that sets up a race condition.

Re: buffer size. The documentation (xml comments) says the buffer size is 128k but the code actually sets the buffer size to 64k. The problem appears to be a "race" condition that will mostly likely exhibit the problem if the input is an exact multiple of the buffers size but may occur even if it is not a multiple.

Please see https://dotnetzip.codeplex.com/workitem/14623 for further discussion and http://dotnetzip.codeplex.com/workitem/14087 for a complete analysis and fix for the problem.

As a side note: The "ZipFile" class in ZipFile.cs sets the default for ParallelDeflateThreshold to 512 * 1024 in _InitInstance. The "ZipOutputStream" class in ZipOutputStream.cs sets the default for ParallelDeflateThreshold to -1L in _Init. This leads to different default behavior where similar behavior might be expected.
Aug 9, 2013 at 9:56 AM
i also have same problem when i am creating zip using Iconic.Zip.dll program is running but when extracting .mdf file it gives error file is corrupt
Aug 9, 2013 at 2:52 PM
Edited Aug 9, 2013 at 2:53 PM
@rahulmutreja - then you can probably use the same solution as other people in this thread...

use "zip.ParallelDeflateThreshold = -1;" when creating the zip

Hope this helps.

Cheers,

Mike
Aug 11, 2014 at 8:55 AM
I have same problem when I have a sdf file greater than 2mb.