Arithmetic operation resulted in an overflow

Feb 25, 2009 at 4:57 PM
I created a backup utility that, with the exception of a few issues, seems to work well. The utility zips the My Documents directory.
In live beta testing, the error I'm getting is on a particular person/computer is an outlook pst file inside My Documents. The pst file is 1.76GB, the total size of My Documents is 21.7GB. (She has another pst file that is 5+GB (jeez - don't get me started) but the zipping process had not yet reached that far) Anyway..
All my testing, with the exception of xlsx files in use (another post), zips successfully. I even tested pst files (and pst files in use by outlook) successfully - though they were smaller.

Is the overall size of the archive being created causing an issue?

The code is basically:
zipArchive.AddDirectory(SourceFolder)
zipArchive.CompressionLevel = Zlib.CompressionLevel.LEVEL9_BEST_COMPRESSION
zipArchive.UseZip64WhenSaving = Ionic.Zip.Zip64Option.Always
zipArchive.Save(DestinationFile)

 

Coordinator
Feb 25, 2009 at 5:26 PM
BFJ - large files should work successfully.
some questions. 
What version of DotNetZip, specifically?
Can you provide a stack trace of the exception?
Feb 25, 2009 at 5:40 PM
1.7.2.6
I'll provide more info as soon as I retest...
Thanks,
jeff
PS-The zip process had been running for about an hour or so on the test subjects pc before it bombed. I just assumed it was overall file size of the archive and the pst file pushed it over the cliff.
Coordinator
Feb 25, 2009 at 6:31 PM
Edited Feb 25, 2009 at 6:33 PM
YES - Jeff there was a previous problem reported with the progress events, specifically having to do with an arithmetic overflow. 
I fixed it in v1.7.2.7, which is the latest v1.7 release. 
Can you please try it with that?
Feb 25, 2009 at 8:16 PM
I tried 1.7.2.7 and still got the error.
The error is happening on the e.TotalBytesToTransfer when the SaveProgress event fires.
This may help:
System.OverflowException was unhandled
  Message="Arithmetic operation resulted in an overflow."
  Source="SMG Backup"
  StackTrace:
       at SMG_Backup.MainForm.zipArchive_SaveProgress(Object sender, SaveProgressEventArgs e) in C:\Users\jeffy.SELLETHICS\Documents\Visual Studio 2005\Projects\SMG Backup\SMG Backup\MainForm.vb:line 218
       at Ionic.Zip.ZipFile.OnSaveBlock(ZipEntry entry, Int64 bytesXferred, Int64 totalBytesToXfer) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipFile.cs:line 3777
       at Ionic.Zip.ZipEntry.OnWriteBlock(Int64 bytesXferred, Int64 totalBytesToXfer) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipEntry.cs:line 1717
       at Ionic.Zip.ZipEntry._WriteFileData(Stream s) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipEntry.cs:line 2953
       at Ionic.Zip.ZipEntry._EmitOne(Stream outstream) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipEntry.cs:line 3310
       at Ionic.Zip.ZipEntry.Write(Stream outstream) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipEntry.cs:line 3268
       at Ionic.Zip.ZipFile.Save() in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipFile.cs:line 2915
       at Ionic.Zip.ZipFile.Save(String zipFileName) in c:\dinoch\dev\dotnet\zip\DotNetZip\Zip Partial DLL\ZipFile.cs:line 3086
       at SMG_Backup.MainForm.ZipSave(String SourceFolder, String DestFile) in C:\Users\jeffy.SELLETHICS\Documents\Visual Studio 2005\Projects\SMG Backup\SMG Backup\MainForm.vb:line 81
       at SMG_Backup.MainForm.btnZip_Click(Object sender, EventArgs e) in C:\Users\jeffy.SELLETHICS\Documents\Visual Studio 2005\Projects\SMG Backup\SMG Backup\MainForm.vb:line 66
       at System.Windows.Forms.Control.OnClick(EventArgs e)
       at System.Windows.Forms.Button.OnClick(EventArgs e)
       at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
       at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
       at System.Windows.Forms.Control.WndProc(Message& m)
       at System.Windows.Forms.ButtonBase.WndProc(Message& m)
       at System.Windows.Forms.Button.WndProc(Message& m)
       at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
       at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
       at System.Windows.Forms.NativeWindow.DebuggableCallback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)
       at System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(MSG& msg)
       at System.Windows.Forms.Application.ComponentManager.System.Windows.Forms.UnsafeNativeMethods.IMsoComponentManager.FPushMessageLoop(Int32 dwComponentID, Int32 reason, Int32 pvLoopData)
       at System.Windows.Forms.Application.ThreadContext.RunMessageLoopInner(Int32 reason, ApplicationContext context)
       at System.Windows.Forms.Application.ThreadContext.RunMessageLoop(Int32 reason, ApplicationContext context)
       at System.Windows.Forms.Application.Run(ApplicationContext context)
       at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.OnRun()
       at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.DoApplicationModel()
       at Microsoft.VisualBasic.ApplicationServices.WindowsFormsApplicationBase.Run(String[] commandLine)
       at SMG_Backup.My.MyApplication.Main(String[] Args) in 17d14f5c-a337-4978-8281-53493378c1071.vb:line 81
       at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
       at System.AppDomain.nExecuteAssembly(Assembly assembly, String[] args)
       at System.Runtime.Hosting.ManifestRunner.Run(Boolean checkAptModel)
       at System.Runtime.Hosting.ManifestRunner.ExecuteAsAssembly()
       at System.Runtime.Hosting.ApplicationActivator.CreateInstance(ActivationContext activationContext, String[] activationCustomData)
       at System.Runtime.Hosting.ApplicationActivator.CreateInstance(ActivationContext activationContext)
       at System.Activator.CreateInstance(ActivationContext activationContext)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssemblyDebugInZone()
       at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
Coordinator
Feb 25, 2009 at 8:38 PM

Ok,
It looks like the exception is occuring in your code.

Can you show me line 218 in MainForm.vb?    I suspect you are assigning an Int64 value to an Int32 field, like the Value property on the progressBar ?
Also, regarding the other issue: you can see if the temporary XLSX problem is fixed with v1.7.2.9. 

Feb 25, 2009 at 8:42 PM
That is my saveprogess event, line 218 is: pbarCurrent.Maximum = e.TotalBytesToTransfer

Private

 

Sub zipArchive_SaveProgress(ByVal sender As Object, ByVal e As Ionic.Zip.SaveProgressEventArgs) Handles zipArchive.SaveProgress

 

 

If e.EntriesTotal > 0 Then

 

pbarTotal.Maximum = e.EntriesTotal

 

End If

 

 

If e.EntriesSaved > 0 Then

 

pbarTotal.Value = e.EntriesSaved

 

End If

 

 

If e.TotalBytesToTransfer > 0 Then

 

pbarCurrent.Maximum = e.TotalBytesToTransfer

 

End If

 

 

If e.BytesTransferred > 0 Then

 

pbarCurrent.Value = e.BytesTransferred

lblCurrentFile.ResetText()

lblCurrentFile.Text = e.CurrentEntry.FileName

lblCurrentFile.Update()

 

End If

 

Application.DoEvents()

 

End Sub

 

Feb 25, 2009 at 9:01 PM
I think I see..
Progress bar is Integer, TotalBytesToTransfer is long. So the value exceeded limitations.

I'm testing again (after this test, I'll try the xlsx test), but I change the code to set the progressbar max to 100, convert the long to integer, divide the bytestransferred by totaltotransfer * 100.

    Private Sub zipArchive_SaveProgress(ByVal sender As Object, ByVal e As Ionic.Zip.SaveProgressEventArgs) Handles zipArchive.SaveProgress

        If e.EntriesTotal > 0 Then
            pbarTotal.Maximum = e.EntriesTotal
        End If

        If e.EntriesSaved > 0 Then
            pbarTotal.Value = e.EntriesSaved
        End If

        If e.BytesTransferred > 0 Then
            pbarCurrent.Value = CType((e.BytesTransferred / e.TotalBytesToTransfer) * 100, Integer)
            lblCurrentFile.ResetText()
            lblCurrentFile.Text = e.CurrentEntry.FileName
            lblCurrentFile.Update()
        End If

        Application.DoEvents()

    End Sub

Feb 25, 2009 at 9:37 PM
Edited Feb 25, 2009 at 9:37 PM
I downloaded the  DotNetZipLib-DevKit-v1.7.zip but all that was in there was a chm file...
(In order to get 1.7.2.9)
Coordinator
Feb 26, 2009 at 12:13 AM
ok, I'm having release problems!
Sorry about that! I'll get that fixed pronto.
Coordinator
Feb 26, 2009 at 1:09 AM
OK, Jeff, v1.7.2.10 is up there, it corrects the "missing files".
Coordinator
Feb 26, 2009 at 4:56 AM
Yes, your code looks right now.  The overflow you were seeing was because of the long==>int conversion, as you said.
Feb 26, 2009 at 3:22 PM
Success.

I used 1.7.2.11 to zip a directory, My Documents, and prior to the zip process I opened the following files: .gif, .jpg, .pdf, .doc, .docx, .xls, .xlsx, and .mdb. The zip process completed successfully. I then opened the zip archive and any temp files that were created by opening the original file, were inside the zip archive. I then closed all the files I had open, then ran the update feature I have in my program, which enumerates the files in My Documents and compares to the zip archive, and it updated successfully (replaces an older file with the newer version, adds new files/folders, removes files if they no longer exist).

Now if we could just get that file exception/inclusion feature.... :)

I might not understand your answer, but what was different about the xlsx file compared to the others?
Coordinator
Feb 27, 2009 at 12:27 AM
You will understand just fine.  The xslx file is apparenly opened with Delete permissions by Excel.  When DotNetZip was trying to open the file initially, it did not specify FileShare.Delete, implying that the open would fail if another process (in this case, Excel) had the file open with Delete access on the file pointer.  What I did was further relax the Open call in DotNetZip, so that it would not fail, even if another party held rights to delete the file after I had opened it.  

Previously I had set the Open in DotNetZip to succeed if another process had Read or Write access on the file.  That works for a .xlsx file, but not for a temporary xslx file.  

And prior to that change, DotNetZip   would just fail if the file was open by another process.

So it took me a couple tries to get it relaxed enough for the Excel case.
There still is a chance it will fail of course - Excel could delete the temporary xslx file while your app is zipping.  The library will just throw an exception in that case.  but the point is, the zip process may still fail.