Multiple Cores

Mar 13, 2009 at 5:25 PM
Hello,

I have written a small application that uses DotNetZip for zipping files. The program runs on a computer with 2-Core CPU. According to task manager during zipping CPU usage is 50% Does it mean that the library uses only one core? Is it possible for the library to use both cores?

Thanks.
Coordinator
Mar 13, 2009 at 5:44 PM
Yes, that is exactly what you are seeing.  There's an open work item to modify the library so that it exploits multiple cores and multiple processors.  http://dotnetzip.codeplex.com/WorkItem/View.aspx?WorkItemId=5183 
And also, there is ongoing work to improve the performance of the compressor.  
Mar 14, 2009 at 2:11 PM
What about using AForge.Net for parallel computations? It work on .Net framwork 2.0 I think it would be quite useful for users if the library uses multiple cores as nowadays most of the computers have multicore CPU
Coordinator
Mar 22, 2009 at 6:10 AM
Thanks for the suggestion, I will check it out.
In the meantime, I updated the workitem having to do with multi-core/multi-cpu optimization.  http://dotnetzip.codeplex.com/WorkItem/View.aspx?WorkItemId=5183 
The short story is that I am doing some R&D work for parallelizing the deflation.  On my dual-core 1p machine I was about to reduce the compression time to roughly 55-58% of the time required in the single-threaded approach used in v1.7 od DotNetZip.

This is a pretty good jump in performance.  It would mean little to no change in the API of DotNetZip. 
Jun 7, 2010 at 2:05 PM
Edited Jun 7, 2010 at 2:14 PM
Cheeso, I do love this library! It took very little time to get a utility up and running. By the way, is the DLL threadsafe? If I call t a dozen times from my own threads will it work correctly? I assume so. While I appreciate the use of multiple threads, it actually becomes a problem when used on servers. I am currently using it on a SQL Server machine with quad cores, and it maxes out all four cores. Obviously this is ... an issue. I am about to upgrade this server to one of those 8 or 12 core AMD procs and I truly do NOT want DotNetZip grabbing all 12 cores for compression. If you think about it, moving from 2 to 3 cores gives you a 50% speed bump (all else equal). Moving from 7 to 8 cores gives you only a 12% speed bump. Much less noticible or useful My suggestion would be to give us a property to set the max threads. Then I could decide how many cores I want used for the zip operation. By the way, I spent the weekend zipping up 200 gigs of stuff using the utility we wrote (using DotNetZip) and for my stuff got about an 80% compression. LOTS of room saved. Further we integrated DotNetZip into a process I run that exports huge SQL Server tables to CSV and imports back in the processed CSVs. These table processes leave behind gigabytes of text files that I need to archive (can't just delete) and now I am getting about 80% compression on those as well. Again though, Kudos on what you have done! Really cool!
Coordinator
Jun 14, 2010 at 10:25 PM

yes, the library is thread safe, but the ZipFile class is not threadsafe.

Each instance of the ZipFile class should be used by only one thread.

To limit the parallel compressing - consider setting ZipFile.ParallelDeflateThreshold to -1.  Check the documentation on that property for more info.

I agree though - it would be nice to have an upper limit.

Coordinator
Jun 14, 2010 at 10:26 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.