Best way to compress large archive of files

Jan 21, 2011 at 2:58 AM

Greetings....

I am writing a backup program in c# 3.5.  The basics of the program is to be given a location on a server and the max size of a spanned zip file and go. From there it should traverse all the folder/files from the given location and add them to the archive, keeping the exact structure. It should also compress everything down to a reasonable amount. A given "collection" of folders/files could easily be 10-25gb, while the spanned files would probably be about 1gb each.

I have everything working (using DotNetZip). My only challenge is there is little to no compession actually happening. I chose to use the "AddDirectory" method for simplicity of code and just generally how well it seemed to fit my project. After reading around I am second guessing that decision.

1. Given the below code and the large amount of files in an archive, should I compress each file as it is added to the zip? or should the Adddirectory method provide about the same compression?

2. I have tried every level of compression offered by Ionic.Zlib.CompressionLevel and none seem to help. Should I think about using an outside compression algorithm and stream it into my DotNetZip file?

using (ZipFile zip = new ZipFile())
{
  zip.AddDirectory(root.FullName);

  if (zipPassword.Length > 0)
    zip.Password = zipPassword;

  float size = zipGbSize * 1024 * 1024 * 1024;

  zip.CompressionLevel = Ionic.Zlib.CompressionLevel.BestCompression;
  zip.AddProgress += new EventHandler<AddProgressEventArgs>(Zip_AddProgress);
  zip.ZipError += new EventHandler<ZipErrorEventArgs>(Zip_ZipError);
  zip.Comment = "This zip was created at " + System.DateTime.Now.ToString("G");
  zip.MaxOutputSegmentSize = (int)size;   //in gig
  zip.Name = archiveDir.FullName + @"\Task_" + taskId.ToString() + ".zip"; //from the current dir, put the zips in the archive dir

  zip.Save();
}
Thank you for any help!
Feb 2, 2011 at 8:56 PM

I can't offer a solution but I do have a question.    Does the zip file produced using your code actually require a password when you extract it?

If I add individual files to the zip I can get the password functionality to work however, It does not work when I use the AddDirectory method.

Feb 2, 2011 at 9:06 PM

Actually I never used the password functions. I know it’s in my code but those values are all being taken from a database. I have honestly never tested.

From: MightyAC [email removed]
Sent: Wednesday, February 02, 2011 3:56 PM
To: david@entellect.biz
Subject: Re: Best way to compress large archive of files [DotNetZip:242686]

From: MightyAC

I can't offer a solution but I do have a question. Does the zip file produced using your code actually require a password when you extract it?

If I add individual files to the zip I can get the password functionality to work however, It does not work when I use the AddDirectory method.