Jump to content


Photo

Everything in transfer queue fails first transfer, then works on retry


This topic has been archived. This means that you cannot reply to this topic.
4 replies to this topic

#1 degenerate

degenerate
  • Members
  • 9 posts

Posted 10 November 2009 - 10:17 PM

This happens with all SmartFTP 4.x versions I have tried.
I regret that I am unsure if this is a SmartFTP or ProFTPd bug, because I upgraded to 4.0 of SmartFTP at the same exact time I reinstalled the control panel on my server. So hopefully this is enough!

The problem

Every time I transfer a file from my local machine to the FTP server (or from server to local) the file is added to the transfer queue with "retry". Then approx 20 seconds later the file finally transfers. I can then transfer anything within the next 30-60 seconds. However if ~60 seconds goes by, any file that I want to transfer is again added to the transfer queue with "retry".

Here is the log in one of the files in the transfer queue when it says "retry"

[17:11:21] Operation Begin
[17:11:21] PASV
[17:11:21] An established connection was aborted by the software in your host machine.
[17:11:21] Server closed connection
[17:11:21] Transfer failed.
[17:11:21] Operation End

Then it finally transfers after 20 seconds, and here is the full SmartFTP log. As you can see there is no indication of anything wrong...

[17:05:36] SmartFTP v4.0.1063.0
[17:05:36] Resolving host name "xyz.com"
[17:05:36] Connecting to xxx.xxx.xxx.xxx Port: 21
[17:05:36] Connected to xyz.com.
[17:05:36] 220 FTP Server Ready
[17:05:36] USER ftp@xyz.com
[17:05:36] 331 Password required for ftp@xyz.com
[17:05:36] PASS (hidden)
[17:05:36] 230 User ftp@xyz.com logged in
[17:05:36] SYST
[17:05:36] 215 UNIX Type: L8
[17:05:36] Detected Server Type: UNIX
[17:05:36] RTT: 54.887 ms
[17:05:36] FEAT
[17:05:36] 211-Features:
[17:05:36]  MDTM
[17:05:36]  MFMT
[17:05:36]  AUTH TLS
[17:05:36]  MFF modify;UNIX.group;UNIX.mode;
[17:05:36]  MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
[17:05:36]  PBSZ
[17:05:36]  PROT
[17:05:36]  REST STREAM
[17:05:36]  SIZE
[17:05:36] 211 End
[17:05:36] PWD
[17:05:36] 257 "/" is the current directory
[17:05:36] TYPE A
[17:05:36] 200 Type set to A
[17:05:36] PASV
[17:05:36] 227 Entering Passive Mode (208,100,49,160,198,159).
[17:05:36] Opening data connection to 208.100.49.160 Port: 50847
[17:05:36] MLSD
[17:05:36] 150 Opening ASCII mode data connection for MLSD
[17:05:36] 1358 bytes transferred. (21.3 KB/s) (62 ms)
[17:05:37] 226 Transfer complete
[17:05:51] MLST test.txt
[17:05:51] 550 'test.txt' cannot be listed
[17:05:51] The operation has been added to the Transfer Queue. Check the Transfer Queue for the status.
[17:05:53] MLST test.txt
[17:05:53] 250- Start of list for test.txt
[17:05:53]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt
[17:05:53] 250 End of list
[17:06:23] NOOP
[17:06:23] 200 NOOP command successful
[17:06:53] NOOP
[17:06:53] 200 NOOP command successful
[17:07:23] NOOP
[17:07:23] 200 NOOP command successful
[17:07:53] NOOP
[17:07:53] 200 NOOP command successful
[17:08:23] NOOP
[17:08:23] 200 NOOP command successful
[17:08:53] NOOP
[17:08:53] 200 NOOP command successful
[17:09:23] NOOP
[17:09:23] 200 NOOP command successful
[17:09:53] NOOP
[17:09:53] 200 NOOP command successful
[17:10:23] NOOP
[17:10:23] 200 NOOP command successful
[17:10:53] NOOP
[17:10:54] 200 NOOP command successful
[17:11:19] MLST test.txt
[17:11:20] 250- Start of list for test.txt
[17:11:20]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt
[17:11:20] 250 End of list
[17:11:21] The operation has been added to the Transfer Queue. Check the Transfer Queue for the status.
[17:11:21] MLST test.txt
[17:11:21] 250- Start of list for test.txt
[17:11:21]  modify=20091110220544;perm=adfrw;size=0;type=file;unique=815U5D6006A;UNIX.group=501;UNIX.mode=0644;UNIX.owner=501; test.txt
[17:11:21] 250 End of list

******************** THEN AFTER THE FILE FINALLY TRANSFERS....

[17:11:51] NOOP
[17:11:51] 200 NOOP command successful
[17:12:21] NOOP
[17:12:21] 200 NOOP command successful
[17:12:51] NOOP
[17:12:51] 200 NOOP command successful


And here is my ProFTPd 1.3.2 configuration.............. note that this proFTPd is part of "interworx" control panel.

####
## InterWorx-CP Proftpd config
####

##
# Global Environment
##

ServerName	"InterWorx-CP :: FTP Server"
ServerType	standalone
ServerIdent     on "FTP Server Ready"

PassivePorts 50000 51000

ListOptions     -a
DefaultServer	on
DeferWelcome	on

SyslogLevel     debug
UseReverseDNS   off
UseFtpUsers     off
IdentLookups    off
WtmpLog         off
ScoreboardFile  /var/run/proftpd/proftpd-scoreboard
DisplayLogin    /etc/motd

##
# Basic Defaults
##

Port  21
Umask 022

##
# Timeouts
##

TimeoutLogin      120
TimeoutIdle	  600
TimeoutNoTransfer 900
TimeoutStalled	  3600
TimeoutSession    0

##
# security stuff
##

MaxInstances 30
RootLogin    off

##
# The user/group we run as
##

User  proftpd
Group proftpd

##
# Logging formats
##

TransferLog NONE

LogFormat default "%h %l %u %t \"%r\" %s %b"
LogFormat auth    "%v [%P] %h %t \"%r\" %s"
LogFormat xfer    "%h %l %u %t \"%r\" %s %b"
LogFormat byte    "%u %b"

##
# Global settings
##

<Global>

  ##
  # Basic stuff
  ##

  DisplayLogin	    welcome.msg
  DisplayChdir      readme
  ShowSymlinks      on
  AllowOverwrite    on
  TimesGMT	    off
  AuthOrder         mod_sql.c mod_auth_unix.c

  ##
  # File upload/download resume
  ##
  
  AllowRetrieveRestart on
  AllowStoreRestart    on

  ##
  # SQL Authentication
  ##

  SQLAuthenticate     users*
  SQLConnectInfo      iworx_ftp@127.0.0.1:2306 iworx_ftp m8BCPMZEY9so
  SQLAuthTypes        Crypt
  SQLUserWhereClause  "1"
  SQLMinUserUID       500
  SQLMinUserGID       500
  SQLDefaultUID       65533
  SQLDefaultGID       65533
  CreateHome          off
  SQLUserInfo users username password uid gid homedir shell   
  SQLGroupInfo groups groupname gid members

  ##
  # Security stuff
  ##

  MaxLoginAttempts  3
  RequireValidShell off
  MaxClients 	    25

  ## chroot everyone  
  DefaultRoot ~ !wheel

  ## hide root's stuff
  <Directory />
    HideUser  root
    HideGroup root
    HideGroup wheel
  </Directory>

  ##
  # Logging
  ##

  ## xfer log  
  ExtendedLog /var/log/xferlog READ,WRITE xfer
</Global>

<IfModule mod_tls.c>
  TLSEngine on
  TLSLog /var/log/ftp-tlslog
  TLSProtocol SSLv23 # this selects the latest crypt version

  TLSOptions NoCertRequest # this is REALLY important for WinClients

  # Are clients required to use FTP over TLS when talking to this server?
  TLSRequired off

  # Server's certificate
  TLSRSACertificateFile /etc/pki/tls/certs/proftpd.pem
  TLSRSACertificateKeyFile /etc/pki/tls/certs/proftpd.pem

  # CA the server trusts
  # TLSCACertificateFile /usr/share/ssl/cert.pem

  # Authenticate clients that want to use FTP over TLS?
  TLSVerifyClient off
</IfModule>

maxclientsperuser 65336


#2 degenerate

degenerate
  • Members
  • 9 posts

Posted 10 November 2009 - 10:18 PM

Should "TimeoutSession" in my configuration be something bigger than 0 ?

#3 mb

mb

    Developer

  • Administrators
  • 11521 posts

Posted 10 November 2009 - 10:31 PM

I think something (it doesn't look it is the server because it has a much higher timeout than the 60 seconds. See the Timeouts in the proftpd.conf) disconnects the control connection within the first 60s of inactivity. For some reason SmartFTP never receives the connection closed message nor a FIN (maybe blocked by your router/NAT) and therefore doesn't know that connection has been closed. So it tries to reuse this connection for the next transfer in the queue. Now when it sends the first command (PASV) it recognizes (after the timeout) that the connection is already dead.

The best way to solve the problem is to figure out what closes the control connection due to inactivity. Since this might not be easy you can try this workaround:
In SmartFTP, go to the menu: Tools->Settings. Go to the Queue dialog and disable the [x] Reuse connection option.

Please also install the latest version of SmartFTP:
http://www.smartftp.com/download

#4 mb

mb

    Developer

  • Administrators
  • 11521 posts

Posted 10 November 2009 - 10:31 PM

The SessionTimeout 0 is correct. See:
http://www.proftpd.o...outSession.html

#5 degenerate

degenerate
  • Members
  • 9 posts

Posted 08 January 2010 - 09:22 PM

The best way to solve the problem is to figure out what closes the control connection due to inactivity. Since this might not be easy you can try this workaround:In SmartFTP, go to the menu: Tools->Settings. Go to the Queue dialog and disable the [x] Reuse connection option.

I just wanted to update this topic and say this worked.
I never needed it before but now it works fine with "reuse connection" off.

Thanks!

Edited by degenerate, 08 January 2010 - 09:22 PM.