wrong with ZlibStream compress mode

May 7, 2009 at 4:25 PM
        internal static int Deflate(byte[] Input, ref byte[] Output)
        {
            int totalOut;
            using (MemoryStream writer = new MemoryStream())
            {
                using (ZlibStream compress = new ZlibStream(writer, CompressionMode.Compress, CompressionLevel.LEVEL9_BEST_COMPRESSION))
                {
                    compress.Write(Input, 0, Input.Length);
                    compress.Flush();
                    //Output = new byte[writer.Length];
                    totalOut = Convert.ToInt32(writer.Length);
                    writer.Seek(0, SeekOrigin.Begin);
                    writer.Read(Output, 0, totalOut);
                }
                writer.Close();
            }
            return totalOut;
        }


first, please download the sample file from http://files.cnblogs.com/unruledboy/buffer.zip

extract it, then run the following test code:

            byte[] output = new byte[0x8000];
            int result = Deflate(File.ReadAllBytes(@"buffer.bin"), ref output);

you will see the result is 2, which means only 2 bytes are returned.

Coordinator
May 7, 2009 at 9:24 PM
Edited May 8, 2009 at 6:08 AM

UnruledBoy, you are doing something not quite right.
You are reading from the compressed stream (writer) before the compressor has been closed.

When using compressing streams, in DotNetZip and in other compression libraries, the usage pattern generally states that you must Close() the compressing stream before attempting to read the compressed bytes.  This is because the compressing stream often will buffer some data, and the Close() will implicitly flush all the bytes and append a footer where appropriate (where the compression format calls for a footer).  Flush() will not quite suffice in this case.

By reading the stream before it is closed, you are getting an incomplete output.  In the case of the ZlibStream, I believe the buffer size is 32k, and you have read only 2 bytes.  I suspect these two bytes are header bytes, and the actual compressed data has not yet been written to the MemoryStream (writer), at the time you read.

One other minor thing - you have typed the output to be ref, which is not necessary in your usage.

Here's some updated code that should work for you:

      internal static int Deflate(byte[] input, byte[] output)
      {
          int totalOut;
          using (MemoryStream writer = new MemoryStream())
          {
              using (ZlibStream compress = new ZlibStream(writer, CompressionMode.Compress, CompressionLevel.LEVEL9_BEST_COMPRESSION, true))
              {
                  compress.Write(input, 0, input.Length);
              }
              totalOut = Convert.ToInt32(writer.Length);
              writer.Seek(0, SeekOrigin.Begin);
              writer.Read(output, 0, totalOut);
          }
          return totalOut;
      }

When I run this code I get a compressed size of 20216 bytes when compressing your buffer.bin file. 

The main change is that I have moved the call to writer.Read() to outside the using() clause for the compressor. (Using() implicitly calls Close/Dispose upon exit from the using scope). I had to modify the code to employ the ZlibStream constructor that leaves the underlying stream open.

This is not very robust code - there is the obvious problem that the output may not be large enough for the compressed data. But it illustrates the point.

May 8, 2009 at 2:47 AM

It's strange, I could compress other files, so far only this file. But I got your point.

great thanks for your always nice and extensively detail explanation. now it works as expected. thanks for your nice library, again:)