"File already exists" error when adding new file to a zip file

Feb 26, 2010 at 3:59 PM

Hi,

We're getting an IO Error exception when we try to add a new file to an existing zip archive.  We're running DotNetZip 1.8.  The IO error is
"Cannot create a file when that file already exists".  However, it doesn't appear that the file is already in the zip archive.

DotNetZip is being run as part of a batch job.  The job has multiple steps, which run in a loop.  One step selects the file to be archived.  The
next step archives the file to the zip file.  The last step deletes the original file.  The following are excerpts from the job's log file, which
show the 'Archive' steps.  I include them all for completeness and to show the pattern.  Each step adds a file to the destination zip file named
'DBReconcile Err Files 2010022510011458.zip'.  The names of the source files to be zipped are of the pattern 'Secmast_xx_2010-02-25.err", where "xx" is
the unique part of the file name.

As can be seen from the log, the following files were successfully added to the zip file:

 Secmast_1a_2010-02-25.err
 Secmast_1b_2010-02-25.err
 Secmast_2a_2010-02-25.err
 Secmast_2b_2010-02-25.err
 Secmast_3a_2010-02-25.err
 Secmast_3b_2010-02-25.err

but, when the job tried to add file 'Secmast_4a_2010-02-25.err', the "file already exists" exception was thrown. The stack trace at the bottom shows
that 'Ionic.Zip.ZipFile.Save' called 'System.IO.File.Move', which threw the exception.

This problem has occurred before, but it is not reproducable on demand. I would appreciate any help you can give us.

Thank you, very much.
 
============================================================================================================================================================

10:01:16.421 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:16.421 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_1a_2010-02-25.err'<ComponentModel.Design>
10:01:16.421 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:16.421 I  Activity 'Archive' started<Workflow.Runtime>
10:01:19.843 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:19.843 V  TrackData('Status')<Workflow.Runtime>
10:01:19.843 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:19.889 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:19.889 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_1b_2010-02-25.err'<ComponentModel.Design>
10:01:19.889 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:19.889 I  Activity 'Archive' started<Workflow.Runtime>
10:01:20.686 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:20.686 V  TrackData('Status')<Workflow.Runtime>
10:01:20.686 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:20.717 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:20.717 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_2a_2010-02-25.err'<ComponentModel.Design>
10:01:20.717 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:20.717 I  Activity 'Archive' started<Workflow.Runtime>
10:01:21.327 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:21.327 V  TrackData('Status')<Workflow.Runtime>
10:01:21.327 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:21.342 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:21.342 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_2b_2010-02-25.err'<ComponentModel.Design>
10:01:21.342 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:21.342 I  Activity 'Archive' started<Workflow.Runtime>
10:01:21.936 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:21.936 V  TrackData('Status')<Workflow.Runtime>
10:01:21.936 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:21.952 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:21.952 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_3a_2010-02-25.err'<ComponentModel.Design>
10:01:21.952 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:21.952 I  Activity 'Archive' started<Workflow.Runtime>
10:01:22.608 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:22.608 V  TrackData('Status')<Workflow.Runtime>
10:01:22.608 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:22.639 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:22.639 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_3b_2010-02-25.err'<ComponentModel.Design>
10:01:22.639 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:22.639 I  Activity 'Archive' started<Workflow.Runtime>
10:01:23.811 V  [ObjectDescriptor] Set: Status='Success'<ComponentModel.Design>
10:01:23.811 V  TrackData('Status')<Workflow.Runtime>
10:01:23.811 I  Activity 'Archive' completed<Workflow.Runtime>

10:01:23.827 V  Prepare Activity 'Archive'<Workflow.Runtime>
10:01:23.827 V  [ObjectDescriptor] Set: Source='\\SYSOPS-SHADAPP2\c$\tnto\dbreconPssiTntdw\output\Secmast_4a_2010-02-25.err'<ComponentModel.Design>
10:01:23.827 V  [ObjectDescriptor] Set: Destination='\\tnt-chi-fs002\IRPT\Shad\SYS\DBA\DBReconcile Err Files 2010022510011458.zip'<ComponentModel.Design>
10:01:23.827 I  Activity 'Archive' started<Workflow.Runtime>
10:01:24.858 I  Activity 'Archive' failed<Workflow.Runtime>
10:01:24.858 V  TrackData('Status')<Workflow.Runtime>
10:01:24.858 E  IOException in step 'Archive': Cannot create a file when that file already exists.
<Workflow.Activity>
10:01:24.858 V  TrackData('Status')<Workflow.Runtime>
10:01:24.858 I  Activity 'ForEachFileERR' failed<Workflow.Runtime>
10:01:24.858 V  TrackData('Status')<Workflow.Runtime>
10:01:24.858 E  IOException in step 'ForEachFileERR': Cannot create a file when that file already exists.
<Workflow.Activity>
10:01:24.858 V  TrackData('Status')<Workflow.Runtime>
10:01:24.858 E  The workflow has faulted<Workflow.Runtime>
10:01:25.061 E     at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.__Error.WinIOError()
   at System.IO.File.Move(String sourceFileName, String destFileName)
   at Ionic.Zip.ZipFile.Save()
   at Asci.ActiveBatch.JobSteps.FileSystem.Archive.NoWildCardZip() in c:\TFS\Builds\ActiveBatch\V70-Release\Sources\DotNet\AddIns\JobSteps\FileSystem\Archive.cs:line 296
   at Asci.ActiveBatch.JobSteps.FileSystem.Archive.Asci.ActiveBatch.Workflow.Runtime.IActivityExecutionHandler.Execute(IActivityExecutionContext executionContext) in c:\TFS\Builds\ActiveBatch\V70-Release\Sources\DotNet\AddIns\JobSteps\FileSystem\Archive.cs:line 189
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.TryInvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.InvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.ExecuteActivity(Activity activity)
   at Asci.ActiveBatch.JobSteps.JobStepSequence.Asci.ActiveBatch.Workflow.Runtime.IActivityExecutionHandler.Execute(IActivityExecutionContext executionContext)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.TryInvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.InvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.ExecuteActivity(Activity activity)
   at Asci.ActiveBatch.JobSteps.FileSystem.ForEachFile.ProcessFiles(String mainPath, FileDateChecker dateChecker, IActivityExecutionContext executionContext) in c:\TFS\Builds\ActiveBatch\V70-Release\Sources\DotNet\AddIns\JobSteps\FileSystem\ForEachFile.cs:line 220
   at Asci.ActiveBatch.JobSteps.FileSystem.ForEachFile.ProcessFilesAndFolders(String mainPath, FileDateChecker dateChecker, IActivityExecutionContext executionContext) in c:\TFS\Builds\ActiveBatch\V70-Release\Sources\DotNet\AddIns\JobSteps\FileSystem\ForEachFile.cs:line 297
   at Asci.ActiveBatch.JobSteps.FileSystem.ForEachFile.Asci.ActiveBatch.Workflow.Runtime.IActivityExecutionHandler.Execute(IActivityExecutionContext executionContext) in c:\TFS\Builds\ActiveBatch\V70-Release\Sources\DotNet\AddIns\JobSteps\FileSystem\ForEachFile.cs:line 163
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.TryInvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.InvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.ExecuteActivity(Activity activity)
   at Asci.ActiveBatch.JobSteps.JobStepSequence.Asci.ActiveBatch.Workflow.Runtime.IActivityExecutionHandler.Execute(IActivityExecutionContext executionContext)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.TryInvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.InvokeHandler(Activity activity, IActivityExecutionHandler executionHandler)
   at Asci.ActiveBatch.Workflow.ActivityExecutionContext.ExecuteActivity(Activity activity)
   at Asci.ActiveBatch.Workflow.WorkflowInstance.StartCallback()<Workflow.Runtime>

Coordinator
Feb 26, 2010 at 6:05 PM

Yes, the error you are seeing indicates a filesystem file that already exists.  It doesn't indicate that an entry in the zipfile already exists.

At the time of the error, the zip file has been saved, to a filename with a temporary name. Whatever files you added to the zip, have been read and compressed and are present in compressed form in the zip file with the temporary name. DotNetZip is getting the error when moving the temporary zip file to the final name - whatever name you chose for the zip file.   The error tells us that the target zip file already exists. 

DotNetZip checks for existence of the zip file *before* beginning to write the temporary file.  The error indicates that sometime between beginning to write the compressed bits into the temporary-named zip file, and renaming that file to the final name, something else has generated a zipfile with the same final name. 

Is it possible that there are multiple workflows occurring simultaneously, and the filenames for the generated zips are being re-used?

I can make DotNetZip more resilient, to handle this case - in other words to delete the ZIP file if it exists at the time of the file rename - and I will do that. The problem is in DotNetZip v1.9 s well.   But with this fix, the error will only shift from DotNetZip code to your code.   Fixing DotNetZip to work this way will expose a logic error on your side, potentially undetected.  It seems you should fix that logic error, independently of the fix to DotNetZip. 

 

Feb 26, 2010 at 7:16 PM

Hi,

Thanks for responding.  There might be simultaneously occurring workflows, but they should not be using identical zip file names.  I will verify that that is the case. 

I want to be sure of one thing, though.  Each of the job steps runs a new instance of DotNetZip.  The steps are sequential, they do not run in parallel.  Our application first checks to see if the zipfile already exists.  If the file doesn't exist, a new ZipFile object is instantiated.  If the file does exist, a ZipFile.Read is issued.  To paraphrase the code:

   ZipFile zip;

   if (!File.Exists(zipFileName))

       zip = new ZipFile (zipFileName);

   else

      zip = ZipFile.Read (zipFileName);

If the zipfile doesn't currently exist, files are added to it using the AddFile method.  If the zipfile does exist, files are added via UpdateFile.  Is this correct?

Thanks for your help.

Coordinator
Feb 26, 2010 at 7:31 PM

You don't have to use File.Exists(zipfilename) - the ZipFile constructor does that for you.  It will read an existing file, or create a new file if none exists.  You can also use UpdateFile() whether the zipfile exists or not, before starting.  The "update" in the UpdateFile()   method refers to the ZipEntry being updated, not to the ZipFile.  The UpdateFile method was originally intended to allow update of an entry that was already present in a zipfile.  But even if the entry isn't present (or if no entry at all is present), UpdateFile() is ok to call. 

So your code can be simplified by always using the ZipFile constructor, and always using UpdateFile(). 

 

Coordinator
Feb 26, 2010 at 7:35 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.
Coordinator
Feb 26, 2010 at 7:38 PM

After thinking about this situation further, I believe the error that you saw in this case is correct.  I don't think you want DotNetZip to delete the file, if it is created by someone else, between the time you call new ZipFile() and the time you call ZipFile.Save().  I think the error could be clearer, but I think you want the error to occur.

I'm interested to hear your opinion on that. 

Feb 26, 2010 at 7:48 PM

If this is indeed what is happening, then I agree with you.  If someone else created a zipfile with the same name, we shouldn't go ahead and delete it.  The group that is getting this error is in a different location, so it will take some time before I can verify if there are doing that.  Our offices are closed today because of heavy snow (we're in New Jersey).  Thanks for following up on this.

Coordinator
Feb 26, 2010 at 9:53 PM

ok, no prob.  I'm glad you agree.  Just based on code inspection, I don't see how the error could occur unless what I described is happening. The Save() method, internally, checks to see if the file exists, and then writes to the temporary file, then deletes the file if it was found to exist originally, then renames the temp file to the final filename. 

Of course, it's possible there are bugs in other places that lead to this problem.  The segmented zip file feature is pretty new, and is less tested.  it's possible that there's some problem there causing a file to be created with the final filename, rather than a temporary name.  But you said you were using v1.8, and segmented zips were added in v1.9.    I'm having difficulty imagining how the problem might be in the library ("the problem" meaning - a file by the final filename being created unexpectedly), if your app is not using segmented zips. 

There is already an open workitem to support a AllowOverwrite parameter on a Save.  Rather than support that, I think it makes sense to take a more general approach for SaveOptions.  I'd like to do what is described in workitem 9031 instead. But that's coming in the future, and it won't address your immediate situation, I don't think.

The best DotNetZip can do in this case, I think, is to deliver an appropriate error message.

Mar 16, 2010 at 8:36 PM

 

 

 

Hi.  Sorry it took so long to get the information about this problem.  From the following job log, it looks like the file that's getting the "already exists" error is the DotNetZip temporary file, not the actual file name.  Is this possible?

Thanks for your help.

========================================================================================================================

03:16:36 I The workflow has started

Start OK File processing.

03:16:36 I Activity 'ForEachFileOK' started

OK File: account_2010-03-06.ok

03:16:37 I Activity 'Archive' started

03:16:37 I Activity 'Archive' failed

03:16:37 E IOException in step 'Archive': The file '\\tnt-chi-fs002\IRPT\Prod\SYS\DotNetZip-aovzxmue.tmp' already exists.

03:16:37 I Activity 'ForEachFileOK' failed

03:16:37 E IOException in step 'ForEachFileOK': The file '\\tnt-chi-fs002\IRPT\Prod\SYS\DotNetZip-aovzxmue.tmp' already exists.

03:16:37 E The workflow has faulted

Coordinator
Mar 16, 2010 at 9:31 PM

ok, that's very interesting.  I'll need to look further.

Coordinator
Mar 16, 2010 at 9:32 PM
This discussion has been copied to a work item. Click here to go to the work item and continue the discussion.