File Pointer bug in v1.8?

May 20, 2009 at 6:30 PM

Cheeso, what's happening in that master programming world of yours?

Some of my users are getting an error when they update their archive.

The error is: "An attempt was made to move the file pointer before the beginning of the file."

I can't duplicate it on my computer in order to give you a stack trace. Any ideas?

(To sum up the entire process, my program compresses their My Documents directory into - let's say. Then I have a scheduler built in which runs their "Partial" update - which updates the archive with any file changes. This process works fine with most everybody, so I'm a little confused as to what might be happening that produces this error on occassion.)


May 20, 2009 at 9:14 PM

Hey Jeff, I'm hanging tight. Watching the Penguins proceed through the Stanley Cup playoffs.

about that error, that is disappointing.

What *could* be happening is an arithmetic or logic error in the DotNetZip code that seeks past the beginning of the zip file. When you modify or update a zip file, the library seeks around in the original file, and grabs chunks of bytes, and copies them to a new file.  Then the new file is closed, and copied over the old file.  If the math is off, it could try to seek beyond the beginning of the original zip file, causing this kind of error.   

It might happen when the size of the thing being backed up shrinks significantly.  OR maybe when the largest file in the archive shrinks significantly.   Between backups I mean.  

I'll try to run some tests to narrow it down.


May 21, 2009 at 12:30 PM

Hurricanes have had a good run, but I think the Penguins are a little stronger. We'll see tonight in game 2. I wouldn't mind seeing the Blackhawks beat Detroit - but I just don't see it happening.

Thanks for the explanation. I hope you can find something. If this helps...On Friday, one user did a "full" (make the initial archive file), on Monday (he was off Sat and Sun) he ran a "Partial" (update the zip archive with any changes) and it was successful. On Tuesday and Wednesday the "Partial" failed. It will run again at 12p today.

If there is anything I can do on my end to help, just let me know. I'm using


Jun 8, 2009 at 2:33 PM

Cheeso, any news on this?

Most of my users are starting to experience this and where I stated most of my users are fine is an incorrect statement. Everyone appears to have the same error.


Jun 9, 2009 at 2:58 PM

Using v1.8.3.17 now, same issue -

Stack Trace:

at System.IO._Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.SeekCore(Int64 offset, SeekOrigin origin)
at System.IO.FileStream.Seek(Int64 offset, SeekOrigin origin)
at Ionic.Zip.ZipEntry.SetFdpLoh()
at Ionic.Zip.ZipEntry.get_LengthOfHeader()
at Ionic.Zip.ZipEntry.CopyThroughOneEntry(Stream outstream)
at Ionic.Zip.ZipEntry.Write(Stream s)
at Ionic.Zip.ZipFile.Save()
at SMG_Backup.MainForm.ZipUpdate(String OriginalFile)

Jun 12, 2009 at 1:28 AM

Sorry for the delay, I still have to do some tests!


Jun 12, 2009 at 12:20 PM

I'm sure you'll work on it tonight, say around, 8p EST. (LOL - yeah right) - 1 game, 1 winner.

Let me know when you get something, I'm hoping the stack trace helps.


Jun 12, 2009 at 12:21 PM

Hey Jeff - When you do a "partial update", your code calls UpdateFile(),  is that right?  Or UpdateDirectory()?

I'm testing different scenarios now. if there's any more insight you can give me., it might be helpful.

  • how many files in the archive (approx)
  • how many files get modified, updated, deleted, and created between initial (full) zip and second (partial) zip
  • how large are the files
  • how common is the problem


Jun 12, 2009 at 1:01 PM

The code calls UpdateFile() - see below.

If I'm not mistaken, the first "partial update" after the "full" is successful. It's definatley the subsequent "partial updates" that fail every time.

  • how many files in the archive (approx)  **I have info on 2 users that are here now. User 1, My Documents directory is 5.81GB, 11251 Files, 4820 Folders. User 2, My Docs dir is 3.03GB, 4154 files, 594 Folders
  • how many files get modified, updated, deleted, and created between initial (full) zip and second (partial) zip **Users usually perform a partial daily or once a week. Above, User 1 is daily, User 2 is every Friday. The files that get updated are minimal, especially the daily partial updates. If I had to guess, 1-2 minimum, 15-20 (over-estimated) maximum for the daily.
  • how large are the files **Most files are typical word, excel, powerpoint, pdf's, jpg's, bmp, etc. that vary in size. Powerpoint and picture files can be a little larger, but not HUGE.
  • how common is the problem **Every time the do a partial update. The exception (and I'm 99% sure) is the Partial after the initial full, which is successful.

    The updatefile() portion of the code:


























































                zipFileUpdate = Zip.ZipFile.Read(OriginalFile)
                'pass 1, enumerate zip entries and mark them for removal if they no longer exist in sourcefolder
                Dim MarkFileRemove As New System.Collections.Generic.List(Of Zip.ZipEntry)
                For Each FileEntryRemove As Zip.ZipEntry In zipFileUpdate
                    Dim PathNamePass1 As String = System.IO.Path.Combine(zipSource, FileEntryRemove.FileName)
                    If Not System.IO.File.Exists(PathNamePass1) Then
                    End If

                'remove entries from pass 1
                For Each RemoveFile As Zip.ZipEntry In MarkFileRemove

                'pass 2, enumerate files in source folder, if file doesn't exist in zip file, add them
                Dim AddNewSourceFile As New System.Collections.Generic.List(Of System.IO.FileInfo)
                Dim SourceDir As New IO.DirectoryInfo(zipSource)
                Dim AllFiles As IO.FileInfo() = SourceDir.GetFiles("*.*", SearchOption.AllDirectories)
                For Each ExistingFile As IO.FileInfo In AllFiles
                    If Not zipFileUpdate.EntryFileNames.Contains(Replace(Replace(ExistingFile.FullName, zipSource & "\", String.Empty), "\", "/")) Then
                    End If

                'add entries from pass 2
                For Each AddNewFile As System.IO.FileInfo In AddNewSourceFile
                    Dim PathToFile As String = AddNewFile.FullName
                    Dim zipDirectoryPath As String = Replace(AddNewFile.Directory.FullName, zipSource, String.Empty)
                    If zipDirectoryPath.StartsWith("\") = True Then
                        zipFileUpdate.AddItem(PathToFile, zipDirectoryPath.Substring(1))
                        zipFileUpdate.AddItem(PathToFile, zipDirectoryPath)
                    End If

                'pass 3, enumerate zip entries, compare date to original file in source folder, update entry if necessary
                Dim CompareFile As New System.Collections.Generic.List(Of Zip.ZipEntry)
                For Each oldFileEntry As Zip.ZipEntry In zipFileUpdate
                    Dim PathName As String = System.IO.Path.Combine(zipSource, oldFileEntry.FileName)
                    If System.IO.File.Exists(PathName) Then
                        Dim infoReader As System.IO.FileInfo
                        infoReader = My.Computer.FileSystem.GetFileInfo(PathName)
                        If (oldFileEntry.LastModified.Date < infoReader.LastWriteTime.Date) Then
                        End If
                    End If

                'add entries from pass 3
                For Each ReplaceFile As Zip.ZipEntry In CompareFile
                    Dim PathToFile As String = System.IO.Path.Combine(zipSource, ReplaceFile.FileName)
                    Dim zipDirectoryPath As String = System.IO.Path.GetDirectoryName(ReplaceFile.FileName)
                    zipFileUpdate.UpdateFile(PathToFile, zipDirectoryPath)

                'save file
                zipFileUpdate.UseZip64WhenSaving = Zip.Zip64Option.Always
                zipFileUpdate.CompressionLevel = Zlib.CompressionLevel.BestCompression
                zipFileUpdate.TempFileFolder = txtParBackup.Text.Substring(0, 3)
                MsgBox("The zip file you selected, " & OriginalFile & ", has been updated successfully.", MsgBoxStyle.Information, "Successful Update")
                pbarTotal.Value = 0
                pbarCurrent.Value = 0
                lblCurrentFile.Text = String.Empty
                lblProcess.Text = String.Empty
                Button_StopPartial.BackColor = Color.Transparent
                Button_StopPartial.ForeColor = Color.Black
            Catch ex As Exception
                If stopPartial = 1 Then
                    MsgBox("Partial Backup process intentionally stopped." & Chr(13) & "The backup file was not updated.", MsgBoxStyle.Critical, "Update Stopped")
                    pbarTotal.Value = 0
                    pbarCurrent.Value = 0
                    lblCurrentFile.Text = String.Empty
                    lblProcess.Text = String.Empty
                    stopPartial = 0
                    Button_StopPartial.BackColor = Color.Transparent
                    Button_StopPartial.ForeColor = Color.Black
                    MsgBox("The attempt to update your existing backup file failed with the following error: " & Chr(13) & ex.Message & Chr(13) & "Please try again or contact the helpdesk for assistance." & Chr(13) & Chr(13) & ex.StackTrace, MsgBoxStyle.Critical, "Update Failed")
                    FileIO.FileSystem.WriteAllText("C:\SMGBackupErrorLog_Stack.txt", ex.StackTrace, True)
                    pbarTotal.Value = 0
                    pbarCurrent.Value = 0
                    lblCurrentFile.Text = String.Empty
                    lblProcess.Text = String.Empty
                    Button_StopPartial.BackColor = Color.Transparent
                    Button_StopPartial.ForeColor = Color.Black
                End If
            End Try

  • Coordinator
    Jun 12, 2009 at 5:25 PM

    Jeff, for the past couple hours I've been running tests trying to reproduce this problem, but no joy.

    I have 162 different tests that I run every time I release an update of the library.  None of these tests are catching what you are seeing.

    I have taken the source code from above, and produced another "BFJ special" test, which I have tweaked and run and tweaked and run, over and over.  I am out of ideas for how to reproduce the problem.   I am just guessing here.

    I figured this was going to be difficult.  I don't doubt that your users are seeing the problem, but I can't figure out how it is happening, and I don't have any bright ideas on how to reproduce it.  If I can't reproduce it, I don't know how to fix it.

    Can you send me a sample zip file produced after the first, "partial" update - the successful update?

    I know there may be data privacy concerns.  You said most of your uses are experiencing this problem now.  If it is straighforward for YOU to reproduce the problem, maybe you could reproduce it on some fake data files on your machine, and send me the zipfile after the 1st partial update from that. 


    Are you using encryption?  If so, Which kind?

    Jun 12, 2009 at 5:45 PM

    OK, Jeff, some news. Getting no joy on the tests I was running, I resorted to a code inspection, where I found a couple of bugs related to Zip64 archives.

    I cannot say for certain that the bugs I found on inspection could lead to the errors your users are seeing, but it's possible.

    I've made fixes, and I am running the release tests now.  In about 20 minutes if the tests all pass, I will give you a new release.  If that's the case I'd like you to test this new release with at least some of your users, and we'll see how it goes.

    stay tuned. 

    Jun 12, 2009 at 5:48 PM

    Sweet. I'm testing too on a spare laptop and of course I can't make it fail. Let me know when the code is there and I'll try it.

    Jun 12, 2009 at 7:11 PM

    ok, try v1.8.3.19.

    it's available now.

    Jun 12, 2009 at 7:14 PM

    Ok, I'm using v1.8.3.19 right now on User 2 from above. This will take some time and it may be next Wed or Thur before I get back to you as I'm off this M and Tu.

    In any event, User 2 is currently doing a Partial with the new code. I'm also working on a test laptop. On my test laptop, I ran a full - success, then I add files to the directory, then run a partial - success. I'm going to keep adding files until it fails. I can send you the archive file if you give me an FTP site. The files I'm working with are just system files, etc.


    Jun 12, 2009 at 7:27 PM

    Ok I am testing also.  If the bugs I found are the source of your problem, then I should be able to to reproduce the problem.  I am testing that right now, with the v1.8.3.17 version.  It will take a good  30 minutes to run the test.   If I can reproduce the problem on v1.8.3.17, then I will run the same test on v1.8.3.19, and if no problem occurs, then we can be pretty certain I found it. Of course final confirmation would have to come from you.

    I'll keep you posted.

    Jun 13, 2009 at 1:59 AM

    Yes, I just checked the test outcome during intermission.  And yes, I was able to reproduce the problem you reported.

    Now I am running the test with the updated library. Will let you know.  My estimate for the length of the test was wrong. It's more like an hour for each run.

    Jun 13, 2009 at 5:18 AM

    OK, just confirming - I've run my test case on both v1.8.3.17 and v1.8.3.19 repeatedly.  The test reproduces the problem reliably on v1.8.3.17.  And the problem does not occur on v1.8.3.19.  That seems to confirm that the change for v1.8.3.19 fixed the problem.

    I'm hoping you will see similar positive results.

    For the record the problem I found and fixed was an incorrect cast, from a long to an int, on the offset within the zip file.  The error showed up only in zip64 files that were being updated. 


    Jun 13, 2009 at 4:20 PM
    Edited Jun 17, 2009 at 2:09 PM

    Hey Jeff, one thing I noticed in your code.  When you are comparing file timestamps... You compare only the date portion of the timestamp. I'm guessing you resorted to that approach because comparing the full timestamp gave you an indication that the file had changed, even when it hadn't.   That can happen with Zip files because of the way the timestamp is encoded for each entry.  It defaults to a 2-second precision, which means if the file changed at 11:34:21, then it is stored as 11:34:20.  

    The way around this is to use a threshold comparison.  Like this:

        'zipFileEntry is an entry in the zip file
        ' filesystemFile is a FileInfo from a filesystem file
        Dim hasBeenUpdated = False
        If (zipFileEntry.LastModified < filesystemFile.LastWriteTime) Then
            hasBeenUpdated = ((filesystemFile.LastWriteTime - zipFileEntry.LastModified) _
                              > New TimeSpan(0,0,2))
        End If
        If (hasBeenUpdated) Then
        End If

    This works because the subtraction operation is supported on DateTime types. The New TimeSpan(0,0,2) represents 2 seconds. So the comparison is asking, is the difference more than 2 seconds?  You may want to try that approach in your code.

    Also, you may want to rely not on LastModified, which gives you 2-second precision, but instead on Mtime, which gives you subsecond precision.  Even if you use Mtime though, you should still use the threshold comparison.


    Jun 17, 2009 at 12:27 PM

    Just got back from mini-vacation playing catch-up. That's great you found the bug and I'll try to test today and tomorrow. Thanks for the code tip too, I really appreciate it.

    I'll let you know how the testing turns out.

    Congrats on the Pens - they earned it.

    Jun 17, 2009 at 3:44 PM

    Ran an "partial" for 2 people and they got an error of:

    Exception seeking entry (<filename>) offset (0xFFFFFFFF800CA496) len(0x45398F701)

    Now, these "partials" were with v1.8.3.19 which was updating an archive created with v1.8.3.17 (or earlier). So I'm making the assumption that is the cause of the error. I am having the users perform a full again, and then I can test the partial. I should know more in the next day or 2 - it takes time to run these things, as you know.

    Jun 17, 2009 at 4:30 PM

    Hey Jeff,

    That "Exception seeking..."  is the same error as you originally reported; I had put in a different error message.  I think you're right about the partial vs Full. 

    I can try to run that kind of test here, too.  So far my tests have been to create and update the zipfile with the same version, and as I told you, the problem did occur with, and did not occur with  I will try to create a zip with and then update with a later version to see if the problem occurs here.

    I have a test that covers your case, it creates a zip and then updates it twice.  The zip has to be over 4gb for the problem to occur, so it takes quite a while.  On my machine it runs for 2+ hours. 


    Jun 17, 2009 at 6:28 PM

    I appreciate all your efforts.

    So some questions came to mind; If you can duplicate (and I think you will) the Exception seeking error using an v1.8.3.17 archive updated with v1.8.3.19, is that something that you can fix? Or will a new archive have to be created? Lastly, will any of this have implications on extracting?

    Jun 17, 2009 at 11:35 PM

    I thought about that - extracting the archives that were produced with v1.8.3.17. 

    A zip file consists of a bunch of zip entries - just blocks of data - in a series.  At the end of the series is a directory, a list of each entry, and a pointer to the place in the file where the entry begins.  The problem you uncovered was a bad pointer in the directory.  The zip files produced by v1.8.3.17 have all the data intact, all the entries are there. But the directory has one or more pointers that refer to incorrect locations.  So I think these files will be partially broken.  The entries in the zip file that have good pointers will be extractable.  The entries that have bad pointers won't be able to be extracted. !! I know that is a significant problem for a backup program.

    Can they be fixed?  Well, the start of each zipentry is marked in the zip file, so rather than reading the directory, a program can just do a full scan of the zip file to learn the locations of the entries in the file.  In fact this is what DotNetZip used to do, in a prior version.  People complained that it was slow, so I modified it to rely on the directory at the end of the zip file.

    It should be possible to create a tool to fix up those zip files.  It would read in the directory, and then do a full scan to read in the individual zipentries, and then compare the two.  (The name of the entry is stored in both places - so that is the key for comparison.)   If they don't match, then there's clearly a bad pointer in the directory.  Fixing it is a simple matter of replacing the bad pointer value with the good one.  There's no checksum or encryption on the directory, so modifications there won't disturb the rest of the zip file.

    Thinking about it now, it would take a really long time to scan through the multi-gigabyte zip files, but it should be possible to fix them.

    The question now is, is it worth the effort to build and test that tool, or is it easier to just create new "full" backups?

    Jun 18, 2009 at 12:21 AM

    Just looked at it again.  If I am correct about the nature of the corruption, it will be simple to enable the ZipFile class to correct those old backups.  It should be very reliable.  Checking each multi-gigabyte zip to see if it needs correction would require 15 minutes or so of scan time for a 4gb file.  Fixing it in the event it needs correction will take 10 minutes or so of write time.  Because it takes a long time, you'd have to make an explicit API call to  *ask* for a correction, if you know what I mean.  

    Jun 18, 2009 at 12:25 PM

    I only have 4 users (that I can think of) that exceed that 4gb size. I'm creating 'fulls' for each of them again so it's not that big of a deal for me. I'm not sure if other people using dotnetzip have had this issue, so my question was more of could you build in a silent fix rather than a tool. You know this stuff better than I (by a long shot!) so I was just thinking out loud. I'll continue testing today and tomorrow - but I think I'm good.

    Although...I have other users that don't have issues, but their backups are not over 4GB. Their backups were created with v1.8.3.17. Is their zip file going to have an issue later on? Or is it only files that meet certain specifications? ie- over 4GB and zip64 and xyz and abc, etc.

    Jun 18, 2009 at 3:40 PM

    I have to test it further, but I believe only zips that are zip64 and over 4gb will have the problem you reported.

    Zips that are zip64 (or not) and under 4gb, can be updated successfully to a zip that is over 4gb by using v1.8.3.19 or later.

    The problem with the automatic fix is it will take 30 minutes to actually run.  15 minutes to check, then 15 minutes to rewrite the file.  It's hard to sneak that in. 


    Jun 18, 2009 at 3:44 PM

    Gotcha. I'm good then, that's only a hand-full of people. I'll have more definitive answers for you on my testing sometime tomorrow. I don't expect any issues because I know you're that good.

    Jun 18, 2009 at 9:49 PM

    heh heh heh

    you've had plenty of issues already!

    Jun 19, 2009 at 8:58 AM


    Could you clarify if this bug affects all Zip64 files over 4GB created with or just files that are updated.  I have a number of large Zip files created with so I'd like to know if I should recreate them with a later version.


    Jun 19, 2009 at 10:02 AM

    yes - I believe it affects all zip64 files over 4gb.

    An easy way to verify is to extract your large zip file.  If you can successfully extract, then the zipfile is not affected by the bug.  If the zip file is affected by the bug, then extraction or update of the zipfile will fail. 

    Jun 19, 2009 at 10:50 AM

    To correct zip files affected by this bug, you have a couple options:

    • re-create the zip file from the original content files
    • call ZipFile.FixZipDirectory or ZipFile.CheckZip(string, bool) on the zip file.
      These methods were added in binary release v1.8.3.29 to help deal with the problem.

    To determine if a zip file is affected, call ZipFile.CheckZip(string). (requires v1.8.3.29)

    Jun 19, 2009 at 6:15 PM

    See, I knew you're that good...This seems to be fixed.

    Out of 4 users (over 4gb), 4 have created zip archives, 2 have updated the archive twice, 1 has updated the archive once, and all successful.

    Thanks for your diligence, much appreciated.

    Jun 19, 2009 at 8:04 PM

    Glad this is working now for you.