Transfer Queue items stuck at 99%

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Transfer Queue items stuck at 99%

Post by tenseiken » Sun May 08, 2011 7:32 pm

I've got a long queue of downloads from remote to local going. They're large files and to get my bandwidth to be fully utilized, I'm using the multi-part transfer option. Some (but not all) of my transfers are stopping at 99%, with the status switching to "retry at". Every time they retry, they fail again, unless I disable multi-part for the transfer, then they just erase themselves from the queue at the next retry attempt. This is also leaving the .sections files in my download directories.

Now, none of this would really be much of an issue, but the files are failing hash checks after being downloaded. I was not having this issue with another client I was using recently, though admittedly that client did not have multi-part transfers, which I suspect to be the culprit. I don't really care that they're not deleting from the queue properly, but I can't have files failing hash checks and I really don't want to stop using multi-part as that's the main feature I was looking for when I purchased SmartFTP.

Anyhow, I am using SFTP over SSH and the remote server software reports as SSH-2.0-OpenSSH_5.1p1 Debian-5.

Any ideas?

Here's my system info:

+- System -----------------------------
Microsoft Windows 7
Service Pack 1 (Build 7601)

CPU Speed : 3400 MHz
Total Memory : 6143 MB
Free Memory : 3657 MB

+- SmartFTP ---------------------------
Version : 4.0.1180.0
Time Stamp : 2011-04-17 22:33:50
Platform : x64
Id : 400137753
Maintenance : 2012-05-07
Days in use : 26

+- Language ---------------------------
en-US

+- Internet Explorer ------------------
Version : 9.0.8112.16421

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Transfer Queue items stuck at 99%

Post by mb » Sun May 08, 2011 7:37 pm

Does it happen everytime? Does the protocol matter (SSH or FTP)? Can you send us detailed directions on how to reproduce it with a public FTP server.

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Re: Transfer Queue items stuck at 99%

Post by tenseiken » Sun May 08, 2011 7:46 pm

Not every time, no. I will test it with regular FTP and report back.

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Re: Transfer Queue items stuck at 99%

Post by tenseiken » Sun May 08, 2011 11:20 pm

Alright, it's gone through a couple of files with no transfer queue errors or hash fails over regular FTP. I'll keep an eye on it though. At this point I'm leaning toward using SSH in conjunction with multi-part being the problem. I don't know of any public servers I can connect to with SFTP to try and reproduce it for you, but what I've done is as follows. I established a favorite for the FTP server I'm using, and set the multi-part option for it to 12 parts. Then I connected via STFP over SSH and started adding files to the queue. The queue uses the favorite's setting for multi-part by default, so I left that alone. I'm pretty much just using the default settings aside from ramping multi-part up to 12.

Something I should probably point out--I am using torrent files to perform the hash checks rather than the built-in integrity option, and the files are good server-side. So it's definitely something going on with the transfer.

Edit: I'm pretty convinced it's multi-part using SFTP now as even more files have completed properly. Kind of disappointing, having shelled out the extra money for the version with SFTP. I hope we can come up with a fix. Please let me know if there's anything else I can do to assist. Appreciate the help!
Last edited by tenseiken on Mon May 09, 2011 12:17 am, edited 1 time in total.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Transfer Queue items stuck at 99%

Post by mb » Mon May 09, 2011 8:43 am

Does the problem happen with 2 parts as well?

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Re: Transfer Queue items stuck at 99%

Post by tenseiken » Mon May 09, 2011 11:43 am

2 parts seems to work. Only tested with 3 ~400 meg files but that's a better success rate than what I was seeing before. Going to bump it up to 6 and see what happens. Unfortunately, neither 2 nor 6 are fully utilizing my downstream. 6 is close though, 8 or 10 might be enough.

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Re: Transfer Queue items stuck at 99%

Post by tenseiken » Mon May 09, 2011 12:41 pm

6 parts worked for 3 files in a row. I tried 8 parts on one file and it did the same thing as 12--stuck on retry until I disable multi-part and then fails a hash check.

tenseiken
Posts: 15
Joined: Sat May 07, 2011 9:21 pm

Re: Transfer Queue items stuck at 99%

Post by tenseiken » Fri May 13, 2011 11:55 pm

I hate to keep bumping this, but is there any new info regarding this issue?

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Transfer Queue items stuck at 99%

Post by mb » Sat May 14, 2011 9:29 am

As soon as we have any news we will post it here.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Transfer Queue items stuck at 99%

Post by mb » Mon May 23, 2011 1:15 am

I was trying to reproduce the problem as following:
- 10 parts
- 280 MB file
- Total Transfer Speed: 2 MByte/s.
- Server
[02:40:33] SSH-2.0-OpenSSH_5.1p1 Debian-5
[02:40:33] Starting SSH session. Remote Id: "SSH-2.0-OpenSSH_5.1p1 Debian-5"
[02:40:33] SSH protocol version reply. Client Id: SSH-2.0-SmartFTP
[02:40:36] Key Exchange Algorithm: diffie-hellman-group-exchange-sha256
[02:40:38] Key exchange completed.
[02:40:38] Host Key Algorithm: ssh-rsa
[02:40:38] Client to Server Encryption: aes256-ctr
[02:40:38] Server to Client Encryption: aes256-ctr
[02:40:38] Session MAC: hmac-sha1
[02:40:38] Client to Server Compression: zlib@openssh.com
[02:40:38] Server to Client Compression: zlib@openssh.com

In every case the download was not stuck at 99% and the hash check succeeded. Do you have further instructions on how to reproduce the problem? For example:
- Does your transfer aborted and retried before it reaches 99%?
- Can you post the log from a failed transfer?

Locked