This project is read-only.

Multiple Cores

Mar 13, 2009 at 6:25 PM

I have written a small application that uses DotNetZip for zipping files. The program runs on a computer with 2-Core CPU. According to task manager during zipping CPU usage is 50% Does it mean that the library uses only one core? Is it possible for the library to use both cores?

Mar 13, 2009 at 6:44 PM
Yes, that is exactly what you are seeing.  There's an open work item to modify the library so that it exploits multiple cores and multiple processors. 
And also, there is ongoing work to improve the performance of the compressor.  
Mar 14, 2009 at 3:11 PM
What about using AForge.Net for parallel computations? It work on .Net framwork 2.0 I think it would be quite useful for users if the library uses multiple cores as nowadays most of the computers have multicore CPU
Mar 22, 2009 at 7:10 AM
Thanks for the suggestion, I will check it out.
In the meantime, I updated the workitem having to do with multi-core/multi-cpu optimization. 
The short story is that I am doing some R&D work for parallelizing the deflation.  On my dual-core 1p machine I was about to reduce the compression time to roughly 55-58% of the time required in the single-threaded approach used in v1.7 od DotNetZip.

This is a pretty good jump in performance.  It would mean little to no change in the API of DotNetZip. 
Jun 7, 2010 at 3:05 PM
Edited Jun 7, 2010 at 3:14 PM
Cheeso, I do love this library! It took very little time to get a utility up and running. By the way, is the DLL threadsafe? If I call t a dozen times from my own threads will it work correctly? I assume so. While I appreciate the use of multiple threads, it actually becomes a problem when used on servers. I am currently using it on a SQL Server machine with quad cores, and it maxes out all four cores. Obviously this is ... an issue. I am about to upgrade this server to one of those 8 or 12 core AMD procs and I truly do NOT want DotNetZip grabbing all 12 cores for compression. If you think about it, moving from 2 to 3 cores gives you a 50% speed bump (all else equal). Moving from 7 to 8 cores gives you only a 12% speed bump. Much less noticible or useful My suggestion would be to give us a property to set the max threads. Then I could decide how many cores I want used for the zip operation. By the way, I spent the weekend zipping up 200 gigs of stuff using the utility we wrote (using DotNetZip) and for my stuff got about an 80% compression. LOTS of room saved. Further we integrated DotNetZip into a process I run that exports huge SQL Server tables to CSV and imports back in the processed CSVs. These table processes leave behind gigabytes of text files that I need to archive (can't just delete) and now I am getting about 80% compression on those as well. Again though, Kudos on what you have done! Really cool!
Jun 14, 2010 at 11:25 PM

yes, the library is thread safe, but the ZipFile class is not threadsafe.

Each instance of the ZipFile class should be used by only one thread.

To limit the parallel compressing - consider setting ZipFile.ParallelDeflateThreshold to -1.  Check the documentation on that property for more info.

I agree though - it would be nice to have an upper limit.

Jun 14, 2010 at 11:26 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.