re:serious ftp error condition

Since going up to SmartFTP 1.5 Build 988 Released, I have been experiencing a disturbing result from direct file transfers.
What is happening is I am getting transfer timeout and 'broken pipe' messages when attempting to direct transfer files of any fair size, like around 18mb. Smaller sized files are fine. The larger sized files do not transfer completely.
Following is an example excerpt from my log file:
Opening data connection to 132.170.10.14 Port: 40581
RETR td1lmail.txt
150 ASCII data connection for td1lmail.txt (132.170.10.14,1447) (18926054 bytes).
Transfer Timeout (20s). Closing data connection.
4105652 bytes transferred. (189 KB/s) (00:00:21)
550 td1lmail.txt: Broken pipe.
Transfer failed. Log closed

It looks like a server/connection problem to me. Use the global queue for your transfers. It features auto reconnect and auto resume.

Regards,
-Mat

From aboulay:

Thanks for the response, but I need to clarify.
I have been using smartftp for a few years now with no incident.
I have drag/dropping files from local to server with absolute file integrity intact, with no sudden disconnections.
Now, with the latest version, I am experiencing the 'broken pipe' on larger sized files, resulting in a partial file transfer.
I can only assume that it's the new version that is causing this, since I have not changed any option/user settings or in the way I transfer files.
This is why I am so concerned that it's a bug that is corrupting my transfers.
Thanks.

Please post the system information from Help->About. Make sure you are running the latest version.

Try to use binary instead of ASCII mode. Remove .txt extension in the Settings->Ascii/binary.

Try a different FTP to see if you have similiar problems. Post the log from there as well.

Regards,
-Mat

Posted: 07-08-2005 08:31 PM Post subject: re:serious ftp error condition

--------------------------------------------------------------------------------

I switched transfer mode to binary and transferred the same large file.
This time the whole file transferred successfully with no broken pipe message and all records remained intact.

I did happen to try another ftp client and was successful in transferring the whole file as ASCII.

I know this is a temporary fix, but can it be acknowledged at this point that Smartftp should not work this way? I do have the latest release, which is 1.5.988.

Thank you.

Hello ..

SmartFTP is working the correct way and there is no known bug related to your problem. It's not recommended to use Ascii mode unless you are exactly aware what it does.

Please reply to this thread and do not create a new thread each time you reply Thanks.

-Mat

You have to admit that after all these years using the product with no incidents and all of a sudden after updating these problems occur can cause a little concern on my part.
I am really extremely impressed with smartftp and would hate to use another product in its place. But unfortunately, I transfer important files and can't take any more chances that might affect file integrity.
Thank you all for the responses. Perhaps I will check in later on after more updates are applied and see if the problem has been solved.

I'd like to summarize the problem. If something should be wrong please let me know.

Problem
When transferring a file in ascii mode (TYPE A) the transfer aborts with a "Broken pipe" error from the server. When transferring the same file in binary/image mode (TYPE I) the transfer completes successfully.

Fact:
The code for handling the data connection in the SmartFTP client is not aware of the data transfer mode (ascii or binary/binary) during the data transfer.

My guess:
Some server especially under *nix temper the data stream in ascii mode (e.g. converting CR LF to LF or similar conversations). Therefore taking the above "Fact" into account, the only difference in the whole process is with the server.

Solution:
Use binary/image mode (TYPE I) as suggested before. There is absolutely no reason to use the ascii (TYPE A) mode.

You cannot just compare situation A with situation B without knowing all the facts. e.g. how do you know if the same file has been successfully transferred in ascii mode with the old version? Did you use the same timeout as before? The default in the previous version was higher than in the new version for example.

If you want us to allocate more engineering resources for this issue you need to provide more information. e.g. Complete logs from the smae file with SmartFTP 1.1, SmartFTP 1.5 and other ftp clients. Logs from transfers with different files, logs from transfers from/to different servers, etc.

Thank you.
-Mat

Thanks for pursuing this with me.
I managed to retrieve an earlier version of SmartFTP, v1.0.983 from Dec, 2004.
The same file (ending with .txt) extension transfers successfully after dozens of transfers.
Once I update to the latest release, the same .txt file does not transfer successfully.
I know what you're saying, use BINARY intstead of ASCII, unless one really needs to use ASCII.
But I am using ASCII mode with this older release and it transfers fine the dozens of times I tried it.
There is no premature timeout that I notice in the most recent release's settings thay may be a factor in terminating the transfer.

I notice SmartFTP just released an update that includes more clarification in the Transfer mode section of the settings.
This should help users to use BINARY, unless its necessary to use ASCII.
But I am curious about one thing.
In the earlier version you have .txt as one of the extension that can be used in ASCII. Why was that put in the settings by default, if as you say, that BINARY should have been used for a reliable transfer?
Just curious.

Thanks.

Can you post the logs of the transfers (same file, same data mode and ascii transfer mode) with SmartFTP v1.5 (latest) and an earlier build?

If a file is transferred in ascii mode, a UNIX FTP server converts all CR LF occurrences in the stream to LF. Therefore it tempers the data stream and it's not possible to compare the file size of the transferred file and with th file size on the server to check if the transfer was successful or not. Resuming a file is not possible in ascii mode as well.

It was a mistake to add the .txt extension to the ascii filter list in previous versions.

Regards,
-Mat

Hi,

got the same problem, except for the fact that whatever mode I choose (ASCII / binary), I always get a broken pipe. I tried a number of solutions: PASV mode, not-passive and even installing a different FTP server on my webserver (tried both FreeBSD's default FTP server as well as ProFTPD). I also tried a different port range for the passive transfers (at the server side).

At first I thought it was a problem with my server, but when I took the time to connect with another FTP client, it worked perfectly well. At that time I decided to grab the newest version of SmartFTP (v. 1.5.990), it didn't work either. I almost gave up all hope, but at that point I decided to shut down the firewall (Zonealarm, see below for version info). It worked. For the sake of argument, I removed zonealarm completely and did a clean install. No help there.

Turning of the firewall is, of course, not a very handy solution. I would guess it has something to do with SmartFTP itself.


I get the error with larger files (tried a couple of different sizes including a 13gig file). The problem occurs at random after a certain amount of time (normally within 5 - 60 seconds). I have an uncapped 100mbit internet connection (both ways). The problem does not occur when uploading files, as far as I can tell.


ZoneAlarm version:
ZoneAlarm version:6.1.737.000
TrueVector version:6.1.737.000
Driver version:6.1.737.000

[edit - forgot to mention, I do not have a router or hardware firewall: I have a direct connection via the Dutch SurfNET University Network (www.surfnet.nl)]

Hi, have you tryed with a diferent firewall software? i have tested with few docens using SmartFTP 1.0 and 1.1 without trouble and SmartFTP 1.5 with nforce 4 hardware firewall with sabe sucess.