GetNextEntry throwing BadReadException

Jul 13, 2011 at 3:37 PM

Hi,

 

When I call ZipInputStream.GetNextEntry(), it's throwing an exception that says:

+ exception {Ionic.Zip.BadReadException:   ZipEntry::ReadHeader(): Bad signature (0x86635B9F)

 

Now, this only happens when I am on the last entry, so the expected result of GetNextEntry is null, but instead it throws this error.

 

Any ideas on why this is happening when there should be no more entries?

 

Thanks

Jul 13, 2011 at 5:48 PM

Alright so in my zipping up function, I am setting the compression levels

if(compress)
    zipOutStream.CompressionLevel = CompressionLevel.BestSpeed;
else
   zipOutStream.CompressionLevel = CompressionLevel.None;

But if I remove this block of code, I dont get the error mentioned above.
Why is this??

I dont mind removing this code, but some of the files that are going to be added to the archive are already compressed.
Is this going to effect the already compressed files?

Coordinator
Jul 13, 2011 at 7:15 PM

Hi Bowman,

I have a test case that covers this kind of scenario, but it does not fail the way you are reporting.

Seems like you have 2 blocks of code: one that produces the zip and another that reads it. What I understand is that by modifying how you produce the zip file, it causes an error while attempting to read the zip file.  Is that right? Can you provide a  short and sweet code snip that reproduces this behavior ?

Here's the code I just used, it exhibits no problems:

[TestMethod]
public void ZIS_ZOS_VaryCompression()
{
    string testBin = TestUtilities.GetTestBinDir(CurrentDir);
    string resourceDir = Path.Combine(testBin, "Resources");
    var filesToAdd = Directory.GetFiles(resourceDir);

    Func<int, int, bool> chooseCompression = (ix, cycle) => {
        var name = Path.GetFileName(filesToAdd[ix]);
        switch (cycle)
        {
            case 0:
                return !(name.EndsWith(".zip") ||
                         name.EndsWith(".docx") ||
                         name.EndsWith(".xslx"));
            case 1:
                return ((ix%2)==0);

            default:
                return (ix == filesToAdd.Length - 1);
        }
    };

    // Three cycles - three different ways to vary compression
    for (int k=0; k < 3; k++)
    {
        string zipFileToCreate = String.Format("VaryCompression-{0}.zip", k);

        TestContext.WriteLine("");
        TestContext.WriteLine("Creating zip, cycle {0}", k);
        using (var fileStream = File.OpenWrite(zipFileToCreate))
        {
            using (var zos = new ZipOutputStream(fileStream, true))
            {
                for (int i=0; i < filesToAdd.Length; i++)
                {
                    var file = filesToAdd[i];
                    var shortName = Path.GetFileName(file);
                    bool compress = chooseCompression(i, k);

                    if (compress)
                        zos.CompressionLevel = Ionic.Zlib.CompressionLevel.Default;
                    else
                        zos.CompressionLevel = Ionic.Zlib.CompressionLevel.None;

                    zos.PutNextEntry(shortName);
                    using (var input = File.OpenRead(file))
                    {
                        CopyStream(input, zos);
                    }
                }
            }
        }

        TestContext.WriteLine("");
        TestContext.WriteLine("Extracting cycle {0}", k);
        string extractDir = "extract-" + k;
        Directory.CreateDirectory(extractDir);
        using (var raw = File.OpenRead(zipFileToCreate))
        {
            using (var input = new ZipInputStream(raw))
            {
                ZipEntry e;
                while ((e = input.GetNextEntry()) != null)
                {
                    TestContext.WriteLine("entry: {0}", e.FileName);
                    string outputPath = Path.Combine(extractDir, e.FileName);
                    if (e.IsDirectory)
                    {
                        // create the directory
                        Directory.CreateDirectory(outputPath);
                    }
                    else
                    {
                        // create the file
                        using (var output = File.Create(outputPath))
                        {
                            CopyStream(input,output);
                        }
                    }
                }
            }
        }

        string[] filesUnzipped = Directory.GetFiles(extractDir);
        Assert.AreEqual<int>(filesToAdd.Length, filesUnzipped.Length,
                             "Incorrect number of files extracted.");

    }
}

Jul 13, 2011 at 8:44 PM

Alright Im just going to copy and paste the examples you have for zipping and unzipping using zipinputstream and zipoutputstream and add 

significant part of my code in.

im using Ionic.Zip, not Ionic.Zlib

Ok so here is for Zipping:

private void Zipup(bool needsCompression)
{
    if (filesToZip.Count == 0)
    {
        System.Console.WriteLine("Nothing to do.");
        return;
    }

    using (var output= new ZipOutputStream(outputFileName))
    {
        output.Password = "VerySecret!";
        output.Encryption = EncryptionAlgorithm.WinZipAes256;

        foreach (string inputFileName in filesToZip)
        {
            System.Console.WriteLine("file: {0}", inputFileName);

            if(needsCompression)
                 output.CompressionLevel = CompressionLevel.BestSpeed; //<--causing badread exception during unzip (it seems)
            else
                 output.CompressionLevel = CompressionLevel.None;

           //NOTE TO CHEESO:
           //CompressionLevel seems to belong to Zlib, not
           //not Zip, so it wount let me use CompressionMethod
           //instead of CompressionLevel =(

            output.PutNextEntry(inputFileName);
            using (var input = File.Open(inputFileName, FileMode.Open, FileAccess.Read,
                                         FileShare.Read | FileShare.Write ))
            {
                byte[] buffer= new byte[2048];
                int n;
                while ((n= input.Read(buffer,0,buffer.Length)) > 0)
                {
                    output.Write(buffer,0,n);
                }
            }
        }
    }
}

And for Unzipping:

//this method fails when the compression levels are explicitly set
//while zipping, works fine otherwise
private void Unzip(filePath)
{
    byte[] buffer= new byte[2048];
    int n;
    using (var input= new ZipInputStream(inputFileName))
    {
        ZipEntry e;
        while (( e = input.GetNextEntry()) != null)
        {
            if (e.IsDirectory) continue;
            string outputPath = Path.Combine(extractDir, e.FileName);
            using (var output = File.Open(outputPath, FileMode.Create, FileAccess.ReadWrite))
            {
                while ((n= input.Read(buffer, 0, buffer.Length)) > 0)
                {
                    output.Write(buffer,0,n);
                }
            }
            if(Path.GetExtension(fileName).Equals(ZIP_EXTENSION))
            {
                Unzip(filePath); //Recursion here is zip file within zip file
            }
        }
    }
}

I hope this helps 

Coordinator
Jul 14, 2011 at 1:44 AM

ok, I just ran the code you gave me - still no error.  I had to make a few changes to get it to compile.  I tried with v1.9.1.5, and also with v1.9.1.6001 (prelim build for next version). 

I've enclosed the code I used in its entirety, below.  If I can't reproduce the problem you're reporting, then I won't be able to fix it.  Maybe you can have a look at what I produced and see if you can cajole it into something that produces the problem you see.

I noticed that in Unzip() you had some uncompilable parts; also in your code the recursion appears to use the original file.  I'm not sure because as I said it does not compile, the way you gave it to me.  You might want to check that part in your code - confused paths could be  the source of your troubles.

 

// BowmanZis.cs
// ------------------------------------------------------------------
//
// Test something for Bowman
//
// Author     : Dino
// Created    : Wed Jul 13 20:06:47 2011
// Last Saved : <2011-July-13 20:38:50>
//
// ------------------------------------------------------------------
//
// Copyright (c) 2011 by Dino Chiesa
// All rights reserved!
//
// ------------------------------------------------------------------
//
// compile: csc.exe /t:exe /debug+ /r:Ionic.Zip.dll  BowmanZis.cs
//

using System;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using Ionic.Zlib;
using Ionic.Zip;



namespace Ionic.ToolsAndTests
{
    public class BowmanZis
    {
        List<String> filesToZip;
        string basefile = "BowmanZis.zip";
        string baseExtractDir = "BowmanZis";
        string password = "VerySecret!";

        public BowmanZis () {}
        public BowmanZis (string[] args) { }


        private void Zipup(bool needsCompression)
        {
            Console.WriteLine("\nZip up {0}, {1} compression",
                              this.basefile, needsCompression ? "with" : "without");

            if (filesToZip.Count == 0)
            {
                Console.WriteLine("Nothing to do.");
                return;
            }

            using (var output= new ZipOutputStream(this.basefile))
            {
                output.Password = this.password;
                output.Encryption = EncryptionAlgorithm.WinZipAes256;

                foreach (string inputFileName in filesToZip)
                {
                    Console.WriteLine("  file: {0}", inputFileName);

                    if(needsCompression)
                        output.CompressionLevel = CompressionLevel.BestSpeed; //<--causing badread exception during unzip (it seems)
                    else
                        output.CompressionLevel = CompressionLevel.None;

                    //NOTE TO CHEESO:
                    //CompressionLevel seems to belong to Zlib, not
                    //not Zip, so it wount let me use CompressionMethod
                    //instead of CompressionLevel =(

                    output.PutNextEntry(inputFileName);
                    using (var input = File.Open(inputFileName, FileMode.Open, FileAccess.Read,
                                                 FileShare.Read | FileShare.Write ))
                    {
                        byte[] buffer= new byte[2048];
                        int n;
                        while ((n= input.Read(buffer,0,buffer.Length)) > 0)
                        {
                            output.Write(buffer,0,n);
                        }
                    }
                }
            }
        }


        private void Unzip()
        {
            Unzip(this.basefile, this.baseExtractDir);
        }


        private void Unzip(string filePath, string extractDir)
        {
            Console.WriteLine("\nUnzip {0}", filePath);
            byte[] buffer= new byte[2048];
            int n;
            using (var input= new ZipInputStream(filePath))
            {
                input.Password = this.password;
                ZipEntry e;
                while (( e = input.GetNextEntry()) != null)
                {
                    if (e.IsDirectory) continue;
                    string outputPath = Path.Combine(extractDir, e.FileName);
                    Console.WriteLine("  file: {0}", outputPath);

                    string containingDir = Path.GetDirectoryName(outputPath);
                    if (!Directory.Exists(containingDir))
                        Directory.CreateDirectory(containingDir);

                    using (var output = File.Create(outputPath))
                    {
                        while ((n= input.Read(buffer, 0, buffer.Length)) > 0)
                        {
                            output.Write(buffer,0,n);
                        }
                    }
                    if (Path.GetExtension(outputPath).Equals(".zip")) // recurse
                        Unzip(outputPath,
                              Path.Combine(extractDir,
                                           Path.GetFileNameWithoutExtension(outputPath)));
                }
            }
        }


        void Reset()
        {
            if (File.Exists(this.basefile))
            {
                Console.WriteLine("Removing {0}", this.basefile);
                File.Delete(this.basefile);
            }
            if (Directory.Exists(this.baseExtractDir))
            {
                Console.WriteLine("Removing {0}", this.baseExtractDir);
                Directory.Delete(this.baseExtractDir,true);
            }
        }


        public void Run()
        {
            var fileInfos = (from fi in (from fn in Directory.GetFiles(".", "*.cs")
                                         select new FileInfo(fn))
                orderby fi.Length descending
                select fi).Take(12);

            filesToZip = fileInfos.ToList().ConvertAll((fi)=>fi.Name);

            // pass 1: use compression
            Reset();

            Zipup(true);
            Unzip();

            Console.WriteLine("--------------------------------------------");

            // pass 2: no compression
            Reset();

            Zipup(false);
            Unzip();

            Reset();
        }


        public static void Usage()
        {
            Console.WriteLine("\nBowmanZis: test a DotNetZip scenario for bowman.\n");
            Console.WriteLine("Usage:\n  BowmanZis");
        }


        public static void Main(string[] args)
        {
            try
            {
                new BowmanZis(args)
                    .Run();
            }
            catch (System.Exception exc1)
            {
                Console.WriteLine("Exception: {0}", exc1.ToString());
                Usage();
            }
        }

    }
}

 

Jul 14, 2011 at 4:07 PM

Thanks Cheeso, Ill try and reproduce the error with the above code. Did you try zipping/unzipping a zip file with multiple zip files inside>?

Coordinator
Jul 14, 2011 at 6:28 PM

No - i hadn't tried nested zip files.

The use of nested zips should have no bearing on the failure you described  - ZipEntry::ReadHeader(): Bad signature (0x86635B9F) - unless you have a bug in your code.

I just tried nested zips now - it works fine for me. I think maybe you have a bug in your code. If you try to wrap a ZipInputStream over a stream that is already open, you would get the error you're seeing. Check your code again, especially the recursion.