Intermittent file corruption problem

hkauffeldt@techsysinsights

2009-03-03 21:31:05

I've just updated my client to the latest version (Home Edition v3.0.1026.7) and have been experiencing sporadic cases where my uploaded files appear to have been corrupted.

I am running Windows XP, SP3 and have been using SmartFTP for the exact same transfer operations for more than a year. My transfer involves connecting to a corporate LAN via a VPN (via CheckPoin VPN-1 SecureClient). I then FTP a set of folders each of which contains multiple files to a server on the corporate network. Typically, I just select all of the folders and transfer them, allowing SmartFTP to determine whether they have changed and therefore whether they need to be uploaded or not. The folders contain a mix of file types, some of which are ASCII text files and some of which are binarys (graphics, FrameMaker source files and PDFs).

Last night, I transferred the files and the transfer claimed to complete normally. Today, I was informed that someone received an error when attempting to access a file. I connected to the VPN and used SmartFTP to download a copy of the file to my local drive. When I attempted to access the file, I also received an error that the file was corrupted (it was a PDF file). I then tried to upload the original file to the same file location. SmartFTP's transfer queue showed that the file went through pre-transfer but no actual copy was done, suggesting the files were believed to be identical. I then did a local binary file compare of the corrupted file and the original source file on my local drive which I had just tried to upload - they had a difference in content.

I deleted the remote file and transferred it again using SmartFTP, from scratch. The transfer proceeded normally. Once complete, I download that file back to my local drive and verified that it was now intact and correct.

I then downloaded all of the remaining files I had transferred the previous night and found that almost all of them were corrupted. The exceptions appeared to be new files I had copied - any files which had previously existed and which I expected to have been replaced seemed to have been corrupted.

I remove all of the remote files and re-uploaded them. Then I downloaded them to a local drive area and verified they were now intact and correct.

Since the problem is sporadic, and the most recent transfers all worked correctly, I have not included any logs here. If the problem recurs, I will attempt to capture the logs and post them, but I am hoping there may be enough info here for you to offer some suggestions or insights. Please feel free to contact me if there is any further data I may be able to provide which may be of help to your analysis of this problem.

mb

2009-03-03 22:12:14

Hello..

We are not aware of any data corruption problems caused by SmartFTP. There are many other reasons why your data may end up corrupted on the server and be strongly believe SmartFTP is none of them.

Regards,
Mat

hkauffeldt@techsysinsights

2009-03-06 04:12:11

OK, I've found that the problem is not as intermittent as I first thought.

The problem appears to be that SmartFTP is erroneously doing a RESUME of a file transfer when it should not be doing so. I transferred my files again and checked them after transfer and found they were corrupted. I delete all the files, transfer, and all works fine.

I transferred an old (smaller) copy of a file to the remote site. I then transferred the newer version, which is larger, and see from the transfer log that a RESUME is being done - erroneously. The log is:

[22:02:35] Remote file exist check: "FW7420 F7 TL1.pdf".
[22:02:35] SIZE FW7420 F7 TL1.pdf
[22:02:35] 213 34163504
[22:02:35] MDTM FW7420 F7 TL1.pdf
[22:02:35] 213 20090306040214
[22:02:35] Source File: Size=34394926, SizeUnit=Byte, Time=2009-03-06T01:09:41, TimeFormat=Exact
[22:02:35] Destination File: Size=34163504, SizeUnit=Byte, Time=2009-03-06T04:02:14, TimeFormat=Exact
[22:02:35] RecentTime=2009-03-06T04:02:35
[22:02:35] Rule "IF Destination Time=Recent AND Size=Smaller AND Transfer=No Matter THEN Resume" matched. Action="Resume".
[22:02:35] PASV
[22:02:35] 227 Entering Passive Mode (172,26,2,22,150,156)
[22:02:35] Opening data connection to 172.26.2.22 Port: 38556
[22:02:35] REST 34163504
[22:02:36] 350 Restart position accepted (34163504).
[22:02:36] STOR FW7420 F7 TL1.pdf
[22:02:36] 150 Ok to send data.
[22:02:37] 231422 bytes transferred. (144 KB/s) (00:00:01)
[22:02:38] 226 File receive OK.
[22:02:38] SIZE FW7420 F7 TL1.pdf
[22:02:38] 213 34394926
[22:02:38] Source File Size=34394926, Destination File Size=34394926

I'm guessing that this "rule" came from the recent upgrade, since I did not define such a rule and all worked fine before my last upgrade of the SmartFTP application.

This is really causing me a lot of trouble - please let me know when a fix is available.

Hal.

mb

2009-03-06 15:31:32

It's not a bug per se. SmartFTP uses the file exist rules (which are customizable) to figures out whether it should resume or overwrite the file. On FTP servers which support a way to set the file time of the remote file (MFMT, MDTM) you won't have such problems but on legacy servers like yours there is no "right" way that works in all situations.

To workaround the problem please follow these steps:
1. Connect to the server/favorite
2. Go to the menu: Favorites->Favorite Properties
3. Go to the Transfer->File Exist menu
4. Change the Transfer Queue File Exist Rules from Use Default Settings to Use Favorite Settings
5. Click on Edit Rules
6. Remove the following rule: IF Destination Time=Recent AND Size=Smaller AND Transfer=No Matter THEN Resume
7. Click OK a couple of times