Everything in transfer queue fails first transfer, then works on retry

This happens with all SmartFTP 4.x versions I have tried.
I regret that I am unsure if this is a SmartFTP or ProFTPd bug, because I upgraded to 4.0 of SmartFTP at the same exact time I reinstalled the control panel on my server. So hopefully this is enough!

The problem

Every time I transfer a file from my local machine to the FTP server (or from server to local) the file is added to the transfer queue with "retry". Then approx 20 seconds later the file finally transfers. I can then transfer anything within the next 30-60 seconds. However if ~60 seconds goes by, any file that I want to transfer is again added to the transfer queue with "retry".

Here is the log in one of the files in the transfer queue when it says "retry"


[17:11:21] Operation Begin

[17:11:21] PASV

[17:11:21] An established connection was aborted by the software in your host machine.

[17:11:21] Server closed connection

[17:11:21] Transfer failed.

[17:11:21] Operation End


Then it finally transfers after 20 seconds, and here is the full SmartFTP log. As you can see there is no indication of anything wrong...


[17:05:36] SmartFTP v4.0.1063.0

[17:05:36] Resolving host name "xyz.com"

[17:05:36] Connecting to xxx.xxx.xxx.xxx Port: 21

[17:05:36] Connected to xyz.com.

[17:05:36] 220 FTP Server Ready

[17:05:36] USER ftp@xyz.com

[17:05:36] 331 Password required for ftp@xyz.com

[17:05:36] PASS (hidden)

[17:05:36] 230 User ftp@xyz.com logged in

[17:05:36] SYST

[17:05:36] 215 UNIX Type: L8

[17:05:36] Detected Server Type: UNIX

[17:05:36] RTT: 54.887 ms

[17:05:36] FEAT

[17:05:36] 211-Features:

[17:05:36]  MDTM

[17:05:36]  MFMT

[17:05:36]  AUTH TLS

[17:05:36]  MFF modify;UNIX.group;UNIX.mode;

[17:05:36]  MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;

[17:05:36]  PBSZ

[17:05:36]  PROT

[17:05:36]  REST STREAM

[17:05:36]  SIZE

[17:05:36] 211 End

[17:05:36] PWD

[17:05:36] 257 "/" is the current directory

[17:05:36] TYPE A

[17:05:36] 200 Type set to A

[17:05:36] PASV

[17:05:36] 227 Entering Passive Mode (208,100,49,160,198,159).

[17:05:36] Opening data connection to 208.100.49.160 Port: 50847

[17:05:36] MLSD

[17:05:36] 150 Opening ASCII mode data connection for MLSD

[17:05:36] 1358 bytes transferred. (21.3 KB/s) (62 ms)

[17:05:37] 226 Transfer complete

[17:05:51] MLST test.txt

[17:05:51] 550 'test.txt' cannot be listed

[17:05:51] The operation has been added to the Transfer Queue. Check the Transfer Queue for the status.

[17:05:53] MLST test.txt

[17:05:53] 250- Start of list for test.txt

[17:05:53]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt

[17:05:53] 250 End of list

[17:06:23] NOOP

[17:06:23] 200 NOOP command successful

[17:06:53] NOOP

[17:06:53] 200 NOOP command successful

[17:07:23] NOOP

[17:07:23] 200 NOOP command successful

[17:07:53] NOOP

[17:07:53] 200 NOOP command successful

[17:08:23] NOOP

[17:08:23] 200 NOOP command successful

[17:08:53] NOOP

[17:08:53] 200 NOOP command successful

[17:09:23] NOOP

[17:09:23] 200 NOOP command successful

[17:09:53] NOOP

[17:09:53] 200 NOOP command successful

[17:10:23] NOOP

[17:10:23] 200 NOOP command successful

[17:10:53] NOOP

[17:10:54] 200 NOOP command successful

[17:11:19] MLST test.txt

[17:11:20] 250- Start of list for test.txt

[17:11:20]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt

[17:11:20] 250 End of list

[17:11:21] The operation has been added to the Transfer Queue. Check the Transfer Queue for the status.

[17:11:21] MLST test.txt

[17:11:21] 250- Start of list for test.txt

[17:11:21]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt

[17:11:21] 250 End of list


******************** THEN AFTER THE FILE FINALLY TRANSFERS....


[17:11:51] NOOP

[17:11:51] 200 NOOP command successful

[17:12:21] NOOP

[17:12:21] 200 NOOP command successful

[17:12:51] NOOP

[17:12:51] 200 NOOP command successful



And here is my ProFTPd 1.3.2 configuration.............. note that this proFTPd is part of "interworx" control panel.


####

## InterWorx-CP Proftpd config

####



##

# Global Environment

##



ServerName	"InterWorx-CP :: FTP Server"

ServerType	standalone

ServerIdent     on "FTP Server Ready"



PassivePorts 50000 51000



ListOptions     -a

DefaultServer	on

DeferWelcome	on



SyslogLevel     debug

UseReverseDNS   off

UseFtpUsers     off

IdentLookups    off

WtmpLog         off

ScoreboardFile  /var/run/proftpd/proftpd-scoreboard

DisplayLogin    /etc/motd



##

# Basic Defaults

##



Port  21

Umask 022



##

# Timeouts

##



TimeoutLogin      120

TimeoutIdle	  600

TimeoutNoTransfer 900

TimeoutStalled	  3600

TimeoutSession    0



##

# security stuff

##



MaxInstances 30

RootLogin    off



##

# The user/group we run as

##



User  proftpd

Group proftpd



##

# Logging formats

##



TransferLog NONE



LogFormat default "%h %l %u %t \"%r\" %s %b"

LogFormat auth    "%v [%P] %h %t \"%r\" %s"

LogFormat xfer    "%h %l %u %t \"%r\" %s %b"

LogFormat byte    "%u %b"



##

# Global settings

##



<Global>



  ##

  # Basic stuff

  ##



  DisplayLogin	    welcome.msg

  DisplayChdir      readme

  ShowSymlinks      on

  AllowOverwrite    on

  TimesGMT	    off

  AuthOrder         mod_sql.c mod_auth_unix.c



  ##

  # File upload/download resume

  ##

  

  AllowRetrieveRestart on

  AllowStoreRestart    on



  ##

  # SQL Authentication

  ##



  SQLAuthenticate     users*

  SQLConnectInfo      iworx_ftp@127.0.0.1:2306 iworx_ftp m8BCPMZEY9so

  SQLAuthTypes        Crypt

  SQLUserWhereClause  "1"

  SQLMinUserUID       500

  SQLMinUserGID       500

  SQLDefaultUID       65533

  SQLDefaultGID       65533

  CreateHome          off

  SQLUserInfo users username password uid gid homedir shell   

  SQLGroupInfo groups groupname gid members



  ##

  # Security stuff

  ##



  MaxLoginAttempts  3

  RequireValidShell off

  MaxClients 	    25



  ## chroot everyone  

  DefaultRoot ~ !wheel



  ## hide root's stuff

  <Directory />

    HideUser  root

    HideGroup root

    HideGroup wheel

  </Directory>



  ##

  # Logging

  ##



  ## xfer log  

  ExtendedLog /var/log/xferlog READ,WRITE xfer

</Global>



<IfModule mod_tls.c>

  TLSEngine on

  TLSLog /var/log/ftp-tlslog

  TLSProtocol SSLv23 # this selects the latest crypt version



  TLSOptions NoCertRequest # this is REALLY important for WinClients



  # Are clients required to use FTP over TLS when talking to this server?

  TLSRequired off



  # Server's certificate

  TLSRSACertificateFile /etc/pki/tls/certs/proftpd.pem

  TLSRSACertificateKeyFile /etc/pki/tls/certs/proftpd.pem



  # CA the server trusts

  # TLSCACertificateFile /usr/share/ssl/cert.pem



  # Authenticate clients that want to use FTP over TLS?

  TLSVerifyClient off

</IfModule>



maxclientsperuser 65336

Should "TimeoutSession" in my configuration be something bigger than 0 ?

I think something (it doesn't look it is the server because it has a much higher timeout than the 60 seconds. See the Timeouts in the proftpd.conf) disconnects the control connection within the first 60s of inactivity. For some reason SmartFTP never receives the connection closed message nor a FIN (maybe blocked by your router/NAT) and therefore doesn't know that connection has been closed. So it tries to reuse this connection for the next transfer in the queue. Now when it sends the first command (PASV) it recognizes (after the timeout) that the connection is already dead.

The best way to solve the problem is to figure out what closes the control connection due to inactivity. Since this might not be easy you can try this workaround:
In SmartFTP, go to the menu: Tools->Settings. Go to the Queue dialog and disable the [x] Reuse connection option.

Please also install the latest version of SmartFTP:
https://www.smartftp.com/download


The best way to solve the problem is to figure out what closes the control connection due to inactivity. Since this might not be easy you can try this workaround:In SmartFTP, go to the menu: Tools->Settings. Go to the Queue dialog and disable the [x] Reuse connection option.
I just wanted to update this topic and say this worked.
I never needed it before but now it works fine with "reuse connection" off.

Thanks!