which sftp libraries work with DotNetZip

Apr 15, 2011 at 5:23 PM

 

just wondering which if any SFTP software has been tested with DotNetZip. the current version of DotNetZip in my application is v1.8.

i have tried the .NET FtpWebRequest in initial tests just to test the streaming capabilities of the DotNetZip lib. what i found is that the Zip works fine with an upload stream. the issue is with the unzip using a download stream. the error is that it has some unsupported method. i did some research and FtpWebRequest does not support seek(). im assuming DotNetZip is trying to get the total length of the stream. this is not really my problem, just an example.

i would really like to know if any problems exist using these types of streams before $$$ is dropped on a solution.

 

what im trying to do:

1) upload a file from the users personal PC or a secure share that only the user has access to. then take that file, add it to the zip library, and save() it to the target secure server that the user will not have access to. the SFTP will be using a service account username/password.

2) download a file and extract. i want to do this by using the zip lib, and feeding the extract() method with the SFTP stream of the source, and supply the path to the dest. (something like that)

does any of this seem plausible?

 

 

Coordinator
Apr 15, 2011 at 10:43 PM

What you want to do sounds completely reasonable.

There are some issues with streaming content into and out of the DotNetZip library, using the ZipFile class.

When READING a zip, it is necessary to seek forward and backward in the zip file, to move to the data for the ZipEntry.   This is why you saw problems when trying to read a zipfile from a FtpWebStream - it doesn't seek.  (The v1.9 library attempts to be smarter about non-seekable streams).

When WRITING a zip, ZipFile tries to seek where possible, to fixup metadata associated to each ZipEntry.  If Seek is not possible, then the ZipFile class will skip it.  I know this works in v1.9, not sure about v1.8. (I cannot remember when I inserted the smarts to just skip the Seek() when it is not possible)

In the v1.9 library, there are new classes, ZipInputStream and ZipOutputStream, that are designed for streaming reads and writes.  The ZipInputStream will work with a non-seekable input stream, like your FtpWebStream or whatever else.

 

Apr 18, 2011 at 4:49 PM

ok, thanks ill look into that. are there any issues with streaming and the bytes transferred events? i use these events to operate the progress bars in my application.

Coordinator
Apr 18, 2011 at 5:42 PM

no issues as far as I am aware. It should just work.

If you discover any issues, report them here.

Apr 19, 2011 at 8:54 PM
Edited Apr 19, 2011 at 11:07 PM

 

UPLOAD

i found an error.

"You must call PutNextEntry() before calling Write()."

im experimenting with the latest v1.9 using the Zip stream wrappers ZipInputStream and ZipOutputStream. the SFTP lib im testing is UltimateSftp trial v3.5.404.1253

this is the line it fails on

 "zip1.Save(New ZipOutputStream(oStream, False))"

i removed the ZipOutputStream. the file transfers fine with this code.

"zip1.Save(oStream)"

-----

DOWNLOAD

on download i got the following error. not sure if this one is my fault or not.

A first chance exception of type 'Ionic.Zip.BadReadException' occurred in Ionic.Zip.dll
error occurs on both of these lines

Dim zip1 As Ionic.Zip.ZipFile = Ionic.Zip.ZipFile.Read(zipStream)
'Dim zip1 As Ionic.Zip.ZipFile = Ionic.Zip.ZipFile.Read(New ZipInputStream(zipStream, False))

im trying to download and extract the zip produced in UPLOAD step of my test.

"MsgBox(Ionic.Zip.ZipFile.IsZipFile(zipStream, False))" returns false for my zip file. i downloaded the zip file using filezilla and extracted. its a good zip file. the contents of the zip was one docx file.

the base exception is "Could not read block - no data! (position 0x00000000)"

 

im still testing to find what could be wrong.

Coordinator
Apr 19, 2011 at 11:27 PM

You should not call ZipFile.Save() and pass a ZipOutputStream().

You should not call ZipFile.Read() and pass a ZipInputStream(). 

The ZipFile class offers a metaphor for manipulating zip files; call ZipFile.Read() to read one, and ZipFile.Save() (or one of its variants) to write one.

The ZipInputStream offers a different, stream-based metaphor for reading zip files.  ZipOutputStream offers a stream-based metaphor for writing zip files.

You should not mix the use of ZipFile and ZipInputStream, or ZipFile and ZipOutputStream.

 

Apr 21, 2011 at 6:20 PM
Edited Apr 21, 2011 at 8:14 PM

i copied this code from a code example i found. i had to slightly modify it to get it to compile because it had some assignments using "=" that VB was assuming was for comparison.i replaced do whiles with whiles etc. i hit an error on this line. "Dim e As ZipEntry = input.GetNextEntry()"

A first chance exception of type 'System.IO.IOException' occurred in UltimateSftp.dll "Invalid stream offset"

 

 

Try
            Dim buffer As Byte() = New Byte(2048) {}
            Using raw As Stream = client.GetDownloadStream(inputFileName)
                Using input As ZipInputStream = New ZipInputStream(raw)
                    Dim e As ZipEntry = input.GetNextEntry()
                    While (Not e Is Nothing)
                        If Not e.IsDirectory Then
                            Using output As FileStream = File.Open(Path.Combine(extractDir, e.FileName), _
                                                                   FileMode.Create, IO.FileAccess.ReadWrite)
                                Dim n As Integer = input.Read(buffer, 0, buffer.Length)
                                While (n > 0)
                                    output.Write(buffer, 0, n)

                                    n = input.Read(buffer, 0, buffer.Length)
                                End While
                            End Using
                        End If
                        e = input.GetNextEntry()
                    End While
                End Using
            End Using

        Catch ex As Exception
            MsgBox("main message = " + ex.Message + Environment.NewLine + _
            "base message = " + ex.GetBaseException().Message)


        End Try

 

EDIT: when using the other method above i get:

A first chance exception of type 'Ionic.Zip.BadReadException' occurred in Ionic.Zip.dll

Could not read block - no data! (position 0x00000004)

 

EDIT2:

i just tested the original SFTP stream using this crude code: it works but the SFTP lib throws a end of file exception every time. the file gets downloaded, and i can extract the contents. i think the SFTP stream is fine unless its somehow incompatible with DotNetZip.

Public Function DownloadStreamTest2(ByVal inputFileName As String, ByVal extractDir As String) As Stream

        'Dim s As Stream = client.GetDownloadStream(filePath)
        Dim s As Stream = client.GetDownloadStream(inputFileName)
        If s Is Nothing Then
            MsgBox("object is null")
            Return s
        End If

        If Not s.CanRead Then
            MsgBox("stream cannot read")
        End If

        If Not s.CanSeek Then
            MsgBox("stream cannot seek")
        End If

        If s.Length <= 0 Then
            MsgBox("no data to read")
        End If

        Console.WriteLine("stream length  = " + s.Length.ToString())

        Try
            Dim buffer As Byte() = New Byte(2048) {}
            Using s
                Using output As FileStream = File.Open(Path.Combine(extractDir, "test.zip"), _
                                                                   FileMode.Create, IO.FileAccess.ReadWrite)

                    For i As Long = 0 To s.Length - 1
                        Dim n As Integer = s.Read(buffer, i, i + 1)
                        If n <= 0 Then
                            Exit For
                        End If
                        output.Write(buffer, i, i + 1)
                    Next i
                
                End Using
            End Using

        Catch ex As Exception
            MsgBox("main message = " + ex.Message + Environment.NewLine + _
            "base message = " + ex.GetBaseException().Message)


        End Try

        Return s
    End Function

Apr 22, 2011 at 4:23 PM

not sure what to say here... i tried most everything to get it to work, and that nonstandard code above is the only thing the SFTP lib likes.

i used this code which is pretty standard with all streams across the web examples, but when i try it against the SFTP stream i get "Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection."

this code WORKS for FileStream, but NOT for the SFTP Stream.

Dim buffer As Byte() = New Byte(2048) {}

While (True)
                        Dim n As Integer = s.Read(buffer, 0, buffer.Length)
                        If n <= 0 Then
                            Exit While
                        End If
                        output.Write(buffer, 0, n)
                    End While

i think its time to start looking for another sftp lib. i had high hopes for the Ultimate SFTP by ATP Inc...

 

Apr 25, 2011 at 5:54 PM

as an alternative i tried SharpSSH though the Jcsh classes. but the streams produced do not support seek() so it wont work. i did get it to work with the raw stream example one post above. like before DotNetZip reports False when i use isZipFile(stream) but the zip file is good.

ZipFile method

"A first chance exception of type 'Ionic.Zip.BadReadException' occurred in Ionic.Zip.dll"

ZipEntry::ReadHeader():Bad signature (0xBF613E92) at position 0x00000000

 

ZipInputStream method

hung on getNextEntry()

i left it alone for a long time to see if it would recover, but it did not.

 

please respond