Jump to content


Issue Information

  • #000119

  • 0 - None Assigned

  • Bogus

  • -

  • -

Issue Confirmations

  • Yes (0)No (0)
Photo

SmartFTP will not reconnect connections that timeout

Posted by Roger Garstang on 07 February 2011 - 10:42 PM

I have some issues on servers that timeout/disconnect connections after they are idle...or sometimes during the initial connection (like the worker threads haven't all connected yet)...SmartFTP doesn't realize it has no connection to work with and the log will show whatever the initial command is like switching to Passive, sending Port, sending the file, etc. After this that connection will sit until SmartFTP's timeout and/or just say the client closed the connection. Whatever was being done during this is just aborted. If I drag and drop a file it just pretends like I didn't do it. If it was in the queue then it will go into Retry. My other 3-4 worker threads continue fine like the initial failure alerted SmartFTP to the fact of a disconnect, so they will reconnect and send. Retrying the first thread that failed will also go through as will doing the drag and drop a 2nd time.

This effects other things as well, but not as bad. SmartFTP's Buttons indicate there is still a connection when there is not. SmartFTP should get some type of notification that the TCP port on the other side was closed that it can use to indicate connection was lost (I've wrote my own FTP and HTTP tools in the past and have used Connect, Accept, Read, Write, and Close notifications in TCP). If I click another folder it realizes there was a connection drop and automatically reconnects, but when it gets the file list it is the previous folder and it navigates back to the folder you came from. Refreshing the current folder will also reconnect (Sometimes it will get the directory list, sometimes I may have to click it twice). When I refresh the folders (To make SmartFTP realize connection was lost and reconnect) it doesn't always refresh the worker threads right away and they will often still fail that first time if I send a file right away since they have lost connections too.

The main issue is SmartFTP not reconnecting when it detects a connection loss. This was an improvement of what previously happened that I had reported by email before, but ignoring the errors instead of displaying them and failing didn't make the problem go away. I realize the Worker threads already have a retry, but that should be for when you get an error reply from FTP or when trying to reconnect after the disconnect fails. Same thing with Drag and Drop- it shouldn't ignore the first time on failure like it didn't happen. And, when I click a folder it should remember that folder, so when it has to reconnect it can change the working dir to it and get the file list there instead of going back to the previous folder you were in and getting the file list in it.

Usually I have SmartFTP running all day editing web content then uploading, so it is often disconnected by the server timeout. Since SmartFTP doesn't realize this I waste a lot of time retrying the queue items to test my files and dragging and dropping multiple times (It doesn't even put them in the queue on failure, so the whole process has to be repeated).

Updating status to: Bogus

I understand that the main issue is that SmartFTP does not recognize server disconnects. All the other issues you are seeing are side effects cause by this issue. The problem - as I remember - was that SmartFTP does not receive the TCP close from the server. Therefore it cannot know when a connection has been closed until it sends new data. The source of this might be your network router, a software firewall / AV product, or something else between the server and the client.

Workarounds:
When the keep alive feature in SmartFTP is activated (by default it is), a command (typically NOOP) is sent to the server every 30 seconds. If the connection is still active the server will send a reply. If you decrease the keep alive interval, connection resets will be detected faster.

But again I strongly recommend that you track down the cause which are the lost TCP close messages.


Roger Garstang
Feb 08 2011 03:25 PM
NOOP isn't considered activity by the server, so doesn't stop the timeout. The queue threads/connections to my knowledge aren't sending NOOP either (and if they are it isn't considered activity anyway), so they will still timeout and not reconnect. Even when I disconnect from the server I still show the threads connected to the host until SmartFTP is closed, so they do remain open and timeout. I agree it could tell SmartFTP sooner and I could have it send other commands like LIST every few min instead, but that still doesn't solve the problem of it reconnecting automatically when there is a connection issue. If someone trips over the ethernet cable and connection is lost I'd still expect it to reconnect and continue where it left off. That is the main issue, not what the cause of the disconnect is/was. It isn't that it doesn't see the disconnect since other parts do see it and reconnect. It is how the drag and drop and queue are handled where the problem is. Regardless of If SmartFTP sees a TCP Close or not. There are times when I go to the window and it shows a status of disconnected and the log shows a status of client disconnect if it sits long enough idle like over lunch, etc. So, it does/can get the TCP Close. It still fails that first drag and drop or queued item though and that is the issue is it doesn't re-establish the connection that it knows dropped until the 2nd attempt+. It is an order of what happens issue where currently it attempts, fails connection, aborts, notify self that not connected, future connections know to reconnect and don't fail. It needs to be attempts, fails connection, notify self that not connected, re-establish connection, attempt...any failures now need to abort/put into retry. During drag and drop it is the same thing but needs to put it in the queue no matter what instead of not at all on connection failure.

Perhaps there is a need for reconnect retries as an option and seconds in between retries? A dropdown to select 1-3 or 1-5 tries and maybe an unlimited option for cases like loading at boot when the network connection isn't up and it needs to just retry. There is still the issue when clicking a folder and it forgetting where it was too. If I close the application and reopen it will reconnect and go right back to the folder I was in, so it does remember. It just isn't storing the folder when clicked on and most likely only stores current position after getting a directory listing. A directory listing is almost like a queued transfer. It comes the same way as a file behind the scenes and when I click a new folder it should change working directory to it and get the list no matter what issues happen in between.

Edited by Roger Garstang, 08 February 2011 - 03:35 PM.



Roger Garstang
Feb 08 2011 04:15 PM
I spent some time debugging this and monitoring with Wireshark. When idle and the server times out after 15min the PureFTP server appears to just drop the connection without notification. If I go into CPanel and kill the process/connection a TCP CLOSE is sent and SmartFTP sees the disconnect. When I kill the process and SmartFTP sees the connection drop it handles the Drag and Drop and queue correct. So, it just needs to handle it the same way for when it sees the connection drop during the transfer. It is just an order thing where it needs to handle the connection drops the same way no matter when it sees them.


Roger Garstang
Feb 09 2011 03:11 PM
Just for the heck of it I submitted a ticket to my host to have them look at PureFTP and got this as a reply (I also asked what they used to test and it was x):

====================================================

In my test, after 15 minutes, I got the following after the timeout:

Error: Connection closed by server

My FTP client seems to notice it and, if I just start doing something in it again, the connection is automatically re-established. This functions like a temporary disconnect. Perhaps your client doesn't have a way of noticing that? I am unable to replicate the problem. Everything seems to be working as intended.
Thank you,
Micah
Level II Tech Support Engineer
BlueHost.com

====================================================

I have also used x without issues reconnecting as well as y. All see the disconnect and reconnect automatically as well as completing what was requested whether in queue or Drag and Drop. I'd much rather use SmartFTP though because it is faster and stores passwords better as well as more features (including the Pro ones I paid the upgrade price to get). SmartFTP also actually detects the dropped connections better- When I killed the process on them where SmartFTP sees it and noted the log they did not, but still reconnected. So, the issue is still just how SmartFTP handles disconnects and needs to handle them all the same way with a reconnect and completing what was requested instead of aborting the drag and drop or putting the item in a retry state in the queue.

Bug is still there, but I have had better luck with setting up a few settings-

I've Set Keep Alive to send FEAT, PWD, STAT, and SYST at random every 5min. This keeps the main browser connection active (Sending the same command (ex. only PWD in the list) or NOOP was ignored as activity, so sending this way I get 3 different within 15min and it seems to see it as activity). If I let it sit over lunch it still will disconnect, although I think it is a SmartFTP timeout which it sees as disconnected and as soon as I add to queue or drag and drop it reconnects and proceeds where when the server side disconnects it aborts drag and drop without putting in queue and puts queues in retry state.

I've also turned off reusing the queue connections which keeps them from having the server timeout when idle and not reconnecting on that first attempt to add to queue or drag and drop. I've still got the issue with large files that seem to timeout the connection on the server side and it will retry them from start to finish over and over until they send without timeout. Some type of resume would be nice there instead of trying again from the beginning.

The extra traffic with all of the Keep Alive commands and the time to reconnect for each queue thread/connection is a little annoying still, but at least the drag and drops aren't aborted or queue put into retry on the first try all the time anymore. Probably the same amount of wasted time, but at least I don't have to re drag and drop or manually retry the first queue connection all the time.

Bug is still there, but I have had better luck with setting up a few settings-

I've Set Keep Alive to send FEAT, PWD, STAT, and SYST at random every 5min. This keeps the main browser connection active (Sending the same command (ex. only PWD in the list) or NOOP was ignored as activity, so sending this way I get 3 different within 15min and it seems to see it as activity). If I let it sit over lunch it still will disconnect, although I think it is a SmartFTP timeout which it sees as disconnected and as soon as I add to queue or drag and drop it reconnects and proceeds where when the server side disconnects it aborts drag and drop without putting in queue and puts queues in retry state.

I've also turned off reusing the queue connections which keeps them from having the server timeout when idle and not reconnecting on that first attempt to add to queue or drag and drop. I've still got the issue with large files that seem to timeout the connection on the server side and it will retry them from start to finish over and over until they send without timeout. Some type of resume would be nice there instead of trying again from the beginning.

The extra traffic with all of the Keep Alive commands and the time to reconnect for each queue thread/connection is a little annoying still, but at least the drag and drops aren't aborted or queue put into retry on the first try all the time anymore. Probably the same amount of wasted time, but at least I don't have to re drag and drop or manually retry the first queue connection all the time.





0 user(s) are reading this issue

0 members, 0 guests, 0 anonymous users