1.9.0.29 multithreading on/off property?

Nov 13, 2009 at 10:40 AM

Hi Cheeso,

Is there a way of turning off the implicit multithreadeed operation in 1.9.0.29 and future releases via some property?

Apologies if this is documented, but I also thought you may be interested in the results I have seen.

If not please can you consider implementing it otherwise I will be stuck at versions prior to 1.9.0.29

Here's why.

I am running sql 2000 Data Transformation Service package (  so I'm in VBScript and taditional ADO :-(   ) that basically fires stored procedures saves the results to a text file then zips the report and deletes the text file.

This dts package is utilising dts threading capability and is currently running 10 threads to generate these reports and happens every day. I am using your library to Zip the generated report to save on ever growing disk space.

I have been getting ready to install your library to production and running our processes on a uat test box for about a week.

I ran all my perf tests and dev tests under 1.9.0.23 and saw the new version and thought great - I'll use that it should help to speed things up. -even though I was getting very reasonable perf anyway.

This is running on an old server HP Proliant DL380 G3 Dual Xeon 2800 with 3Gb Ram.

(I'm sure my use of Sql 2000 DTS threading and the libraries use of threading is having the ill effects seen below)

Interstingly on batches run under 1.9.0.29 I had lots of errors of the same type on calling the save operation: "Exception of type 'System.OutOfMemoryException' was thrown." unfortunately i can give you any more error information i only have err.description

Batch DotNetZip ReportCount ReportsSizeMb ReportsZipSizeMb TotalZipDurationMin BatchTimeMin MemoryExceptions
13-Nov 10 1.9.0.23 1104 821 80 7 47 0
12-Nov 9 1.9.0.23 1104 1056 102 15 52 0
11-Nov 8 1.9.0.23 1104 1033 99 10 47 0
10-Nov 7 1.9.0.23 1104 1028 36 10 47 0
09-Nov 6 1.9.0.29 1104 1035 38 114 107 80
06-Nov 5 1.9.0.29 1104 900 43 370 108 431
05-Nov 4 1.9.0.29 1104 1025 46 761 107 77
04-Nov 3 N/A 1104 1026 N/A N/A 43 N/A
03-Nov 2 N/A 1104 1034 N/A N/A 43 N/A
02-Nov 1 N/A 1104 1039 N/A N/A 44 N/A

Many thanks

Matt

Coordinator
Nov 13, 2009 at 12:57 PM

Wow, that's pretty unhelpful.

You can turn off threading in the library via the ParallelDeflateThreshold property, on the ZipFile or on the ZipOutputStream.  Set that property to -1 to get the single-threaded behavior.

 

Nov 13, 2009 at 1:19 PM

Great Thanks.

Whats not helpful?  -- the feedback?

Coordinator
Nov 13, 2009 at 3:08 PM

no - heh heh,  I mean it's unhelpful of DotNetZip to run so long,  It's not faster at all.

ps: the reason you get memory exceptions is that the parallel deflating allocates a bunch of memory buffers.  That can be large if you run a bunch all at once.

let me know how it turns out for you.

Thanks for reporting this, your feedback is helpful.

Nov 15, 2009 at 8:36 PM

Ah - No dotnetzip is a marvelous thing, as u can see compressing  ~1gig by 90% and only adding ~4mins to my batch reporting exec time. happy days!!!

Many thanks for your great efforts

Coordinator
Nov 16, 2009 at 12:30 PM

Glad you like it.