Sending Keep-alives during ftp upload

Hi there
I downloaded SmartFTP to try out as I need an FTP client that can send keep-alives during a large upload.
The problem I'm encoutering is with a firewall that is timing out the ftp control port 21 during a large upload even though data is transferring on port 20.
I have found that Smart FTP sends keep-alives while there is no activity, but doesn't send keep-alives during a transfer.
Is there a setting that can turn on keep-alives during a file transfer, in order to maintain some traffic on the control port (21) and thus avoid a timeout.

Thanks

Hello ..

Sending keep alives (NOOP) during transfers is not recommended and usually cause more problems. Use the Transfer Queue for your transfers. It will automatically resume borken transfers.

Regard,s
Mat

Hello ..

Sending keep alives (NOOP) during transfers is not recommended and usually cause more problems. Use the Transfer Queue for your transfers. It will automatically resume borken transfers.

Regard,s
Mat

Thanks for the reply. I'm looking for an FTP client that I can recommend to other organisations who need to upload large files.

I'll try out what you are saying, but the idea of knowing the transfer will fail and hoping it will resume okay doesn't seem elegant to me.

I came across another ftp client that did lets you send NOOP commands during the transfer. It allowed me to overcome the timer issue and send a large file successfully (380 MBytes), using a cellular network connection. It was a user configurable preference to turn on keep-alives during the transfer. It did have the comment that not all servers supported this configuration, but it worked very well for the server I was testing with.

To allow me to compare the two approaches:

Could you provide more detail on what problems can be caused by sending NOOP commands during the transfer? And also how reliable is the resume feature is?

(There may be automatic processes at the server end scanning for newly uploaded content; So I wonder how they will cope when a connection breaks and the file is partially written, but no longer in use - They might use the break in time to start processing the content which would be problematic).

Thanks

Hello ..

>Problems with NOOP command
Servers that do not support will act strangely (e.g. send the NOOP reply at the end of the transfer, don't reply at all, transfer aborts because server doesn't expect a NOOP command, etc).

>Benefit of NOOP during transfer
The only reason you would send NOOP commands during a data transfer is when you are connected through a firewall which is not aware that there is still a data transfer in progress. This was sometimes the case with 1st generation firewalls which didn't inspect the data on the application layer level. I'm a bit surprised that such products still exists as you will end up with numerous other problems whenever you use a protocol which needs a shared state between two or more connections. E.g. FTP protocol: control connection / data connection.

>Resume
There is no general problem with resume unless - as you mentioned - your FTP server triggers some action whenever a transfer finishes, regardless if it's successful or not. However there is no guaruantee that your transfer will finnish the first time. What if it breaks for other reasons?

Depending on your interest and motivation to do some testing we are willing to implement this feature for you within a couple of days.

Regards,
Mat

Hello ..

>Problems with NOOP command
Servers that do not support will act strangely (e.g. send the NOOP reply at the end of the transfer, don't reply at all, transfer aborts because server doesn't expect a NOOP command, etc).

>Benefit of NOOP during transfer
The only reason you would send NOOP commands during a data transfer is when you are connected through a firewall which is not aware that there is still a data transfer in progress. This was sometimes the case with 1st generation firewalls which didn't inspect the data on the application layer level. I'm a bit surprised that such products still exists as you will end up with numerous other problems whenever you use a protocol which needs a shared state between two or more connections. E.g. FTP protocol: control connection / data connection.

>Resume
There is no general problem with resume unless - as you mentioned - your FTP server triggers some action whenever a transfer finishes, regardless if it's successful or not. However there is no guaruantee that your transfer will finnish the first time. What if it breaks for other reasons?

Depending on your interest and motivation to do some testing we are willing to implement this feature for you within a couple of days.

Regards,
Mat

There are definitely appliances out there that don't do application level monitoring. The group running this one is recommending that keep-alives (during transfer) are the standard way of overcoming the issue, and won't turn on application aware monitoring because of the load it will add to the box.

I tried using the transfer queue and saw the ftp fail as expected. The client then continually retried to establish the transfer, but was unsuccessful. In Wireshark I can see the MDTM command from the client and I can see the server respond with what I assume is a count of bytes transferred thus far, but after that the transfer does not restart for some reason (There are FTP packets exchanged ending with a Response: 500 File transfer failed). So in this case the NOOP command during transfer does the trick, but the resume function does not help.

I understand your point about a transfer failing for some other reason, and so I will try to further investigate what is going wrong with the resume feature, but it seems best to have the option to use both features.

I'd be happy to test out the keep-alive during transfer feature for you if you developed it. (To be up-front - I won't be purchasing the client, but I would mention it to groups that need to carry out large ftp uploads and who want/need a client with this feature).

Hello ..

I have implemented it. You can try the latest version 2.5.1008.3 from:
https://www.smartftp.com/download

To enable the option go to the favorite settings. Then in FTP -> Transfer dialog at the bottom. The NOOP interval is 30 seconds and cannot be changed.

Could you post the NOOP reply from the server?

Thanks.
Regards,
Mat

Hi mb

I have tested the new version you supplied.

It sailed past the usual duration when the ftp would timeout - So the NOOP keep-alive during the transfer has done the trick

Thanks for your help with this.

I will send you an excerpt of the Wireshark trace via email.

Will this now become a standard feature of your client build from this point onwards?

Regards

jpmp

Thank you. I have received your email.

The feature is already in the latest official version ;-)

Regards,
Mat
SmartFTP