Poor upload speed for many small files (wodSFTP / wodSFTP.NET / wodSFTPdll)

Hello,

I encountered the problem with the uploading speed of the SFTP.Net library. Thus, copying 100 files 5KB each took 23 seconds while copying 5 files 100KB each consumed only 3 seconds.

Could you suggest how we can improve the uploading rate for the case of many little files needed to be put on the server?

Many thanks in advance,
Andrey.

Hi Andrey,

How can we duplicate this? Did you use GetFiles or GetFile Method for each file to download.

What is your server type/vendor/name?

You can check this out if you go in command prompt:

telnet your.hostname 22

and send us first line returned.

Let us know how it goes.

Regards,
Drazen

How can we duplicate this? Did you use GetFiles or GetFile Method for each file to download.

What is your server type/vendor/name?

Regards,
Drazen

Hi Drazen,

I use the PutFile method to upload the file to the server (not to download!).

To reproduce this behavior you can put a small file (5 KB) 100 times to the server with the different names comparing the speed with putting one big file (500 KB).

Telnet shows SSH-2.0-Sun_SSH_1.1 .

Thanks for the quick reply,
Andrey.

Hi Andrey,

I have just finish testing and it worked like a charm on my side. Average upload speed for one file is 1 second (or less). I have test it with 100 5kb size files on our testing SSH-2.0-Sun_SSH_1.1 server.

Are you using latest 3.1.0.50 version of wodSFTP.NET as I did in my test?

In order to get latest version, please go to:

http://www.weonlydo.com/index.asp?update=1

This is automated process. Make sure you use same email address that is registered with our system, one you used when you have purchased the product.

If problem persist I can give you access to our testing server and sample so you can rewrite it and show us your problem. You should just send e-mail to techsupport@weonlydo.com

Hope I helped.

Regards,
Drazen

I have just finish testing and it worked like a charm on my side. Average upload speed for one file is 1 second (or less). I have test it with 100 5kb size files on our testing SSH-2.0-Sun_SSH_1.1 server.

So you uploaded 100 files and that took about a second for each file, taking about 100 seconds for all, right? If so this speed is too slow!
Is it a good speed to upload 5kb in 1 second? If you upload 500 kilos file it wouldn't take more than couple of seconds! What is the reason for this?

Are you using latest 3.1.0.50 version of wodSFTP.NET as I did in my test?

Yes, we use the latest release dated May 30.

If problem persist I can give you access to our testing server and sample so you can rewrite it and show us your problem. You should just send e-mail to techsupport@weonlydo.com

Actually, I sent there a message two days ago.

Thanks,
Andrey.

Andrey,

In you first post you mention that on your side if I understand you correctly each 5kb file to upload take 23sec, is this correct?

Drazen

Andrey,

In you first post you mention that on your side if I understand you correctly each 5kb file to upload take 23sec, is this correct?

Drazen

Sorry for possible misleading. I meant 23 seconds for all the hundred files. But this is pretty slow!

Andrey,

I have just tried to do same with winSCP (free SFTP client). winSCP upload 100 files in 58sec and wodSFTP.NET in 50sec in my environment.

Can you try to compare upload speed with some other SFTP client for a test?

You should remember that it's normal for upload to be slower in such cases because client for each new file which will be uploaded should make request to a server and when there is lots of files to be transferred result is slower transfer time.

In your case that particularly expressed because you are uploading lots of small files and lots of time is lost in file uploading requests.

Let us know how it goes with some other SFTP client(s).

Drazen

You should remember that it's normal for upload to be slower in such cases because client for each new file which will be uploaded should make request to a server and when there is lots of files to be transferred result is slower transfer time.

Hi Drazen,
It seems our clients are satisfied with the current performance, therefore my alarm was false.

Many thanks you for your help,
Andrey.