BUG: Busy Files Occupy Queue

jmarti05

2007-04-16 14:05:54

I recently upgraded and I'm not sure if this is a bug or not. I don't seem to think that the previous version did this.

version 2.5.1005.19

I've got a queue going with 6 threads. When a thread finds a locked file (file in use by another process) it sets the file up to retry in 1 minute, but then does not move onto other files. Should I happen to run into a particularily busy folder, all transfers could be shut down while the queue sits there constantly trying to transfer the same files and never getting anywhere. I'd like to see the queue "move on" and come back to these recheduled transfers. I've been using the app to move data for 6 months now, and while its not impossible I diddn't notice this bahavior before the upgrade.

Thanks

2007-04-19 06:39:11

Hello ..

I cannot reproduce the problem you reported because I'm most likely not able to reconstruct the exact same situation. Thus I need more information and I would appreciate if you can answer the following questions:
1. Are you uploading or downloading?
2. Is the remote or the local file locked?
3. Can you post the log of the Transfer Queue item where the error occurs?
https://www.smartftp.com/support/kb/how- ... f2564.html

Thank you for you time.
Regards,
-Mat

2007-05-05 18:32:37

Hello ..

I was wondering if you already had some time to look at my answer?
We are very interested to find and fix the bug.

Thank you
Regards,
-Mat

jmarti05

2007-05-07 21:14:38

Hello ..

I was wondering if you already had some time to look at my answer?
We are very interested to find and fix the bug.

Thank you
Regards,
-Mat

I've been working the issue off and on. It appears to be related to how busy the disk system is. That is, I notice the issue less when the disks on the far end are idle than when they are busy writing other data. I'll try and get you the logs asap.

jmarti05

2007-05-07 22:23:49

I've been working the issue off and on. It appears to be related to how busy the disk system is. That is, I notice the issue less when the disks on the far end are idle than when they are busy writing other data. I'll try and get you the logs asap.

1) Uploading

2) Nothing else writes to this volume / FTP Site / Folder but SmartFTP

3)

[18:20:45] Remote file exist check: "pbcomx02_1178575214_C1_F1.1178575214.img".
[18:20:45] SIZE pbcomx02_1178575214_C1_F1.1178575214.img
[18:20:45] 213 989623732
[18:20:45] MDTM pbcomx02_1178575214_C1_F1.1178575214.img
[18:20:45] 213 20070507220555
[18:20:45] Source File: Size=1022379008, SizeUnit=Byte, Time=2007-05-07T22:01:19, TimeFormat=Exact
[18:20:45] Destination File: Size=989623732, SizeUnit=Byte, Time=2007-05-07T22:05:55, TimeFormat=Exact
[18:20:45] Rule "IF Destination Time=Recent AND Size=Smaller AND Transfer=No Matter THEN Resume" matched. Action="Resume".
[18:20:45] PASV
[18:20:45] 227 Entering Passive Mode (132,158,132,34,13,109)
[18:20:45] Opening data connection to 132.158.132.34 Port: 3437
[18:20:45] REST 989623732
[18:20:45] 350 Restarting at 989623732.
[18:20:45] STOR pbcomx02_1178575214_C1_F1.1178575214.img
[18:20:46] 550 pbcomx02_1178575214_C1_F1.1178575214.img: The process cannot access the file because it is being used by another process.

Here's the end of that transfer - file was already there!

[18:21:16] CWD /DSU-DR-REPL
[18:21:16] 250 CWD command successful.
[18:21:16] PWD
[18:21:16] 257 "/DSU-DR-REPL" is current directory.
[18:21:16] Remote file exist check: "pbcomx02_1178575214_C1_F1.1178575214.img".
[18:21:16] SIZE pbcomx02_1178575214_C1_F1.1178575214.img
[18:21:16] 213 1022379008
[18:21:16] MDTM pbcomx02_1178575214_C1_F1.1178575214.img
[18:21:16] 213 20070507222105
[18:21:16] Source File: Size=1022379008, SizeUnit=Byte, Time=2007-05-07T22:01:19, TimeFormat=Exact
[18:21:16] Destination File: Size=1022379008, SizeUnit=Byte, Time=2007-05-07T22:21:05, TimeFormat=Exact
[18:21:16] Rule "IF Destination Time=Newer AND Size=Equal AND Transfer=No Matter THEN Skip" matched. Action="Skip".
[18:21:16] Skipping file "pbcomx02_1178575214_C1_F1.1178575214.img". Reason: User action.

2007-05-09 20:25:34

Thank you.

I wasn't able to reproduce the problem. I simulated the following error for the first file in the Transfer Queue:
[17:21:27] STOR reg.rar
[17:21:27] 0 bytes transferred. (N/A/s) (0 ms)
[17:21:27] 550 Cannot STOR.

The Transfer Queue then jumps to the next file (from the same server) and starts processing it. The Transfer Queue will retry the first item again after the retry delay elapsed.

Regards,
-Mat

jmarti05

2007-05-10 21:08:41

Thank you.

I wasn't able to reproduce the problem. I simulated the following error for the first file in the Transfer Queue:
[17:21:27] STOR reg.rar
[17:21:27] 0 bytes transferred. (N/A/s) (0 ms)
[17:21:27] 550 Cannot STOR.

The Transfer Queue then jumps to the next file (from the same server) and starts processing it. The Transfer Queue will retry the first item again after the retry delay elapsed.

Regards,
-Mat

I made three changes in an attempt to get this working because it is an important data transfer.

1) I moved SmartFTP to the client end. Previously files were generated in location A and I found run the FTP on Server A and transfer them to FTP server at location B. Now I'm running SmartFTP on Server B and pulling the data from the FTP Server on Location A.

2) I downgraded to version SmartFTP Client (x86) 2.0.1002.2 (I've had trouble w/ retry errors since upgrading to 2.5)

3) I changed the FTP root so that it wasn't the root of a volume / found SmartFTP trying to transfer the System Volume Information folder in Windows 2003 / NTFS (Protected System File.)

Anyhow.. its working a lot better. I think I'll keep away from the upgrades until the synching feature comes out w/ version 3. Thanks for the help.

2007-05-16 23:42:33

Hello ...

>1 and >2
Before we can fix a problem we need to be able to confirm it. As of now we cannot reproduce your reporting.
I'm trying to explain the technical background concerning retry delays a bit:
There are two scopes where retry delays apply to:
1. item (file/folder)
2. favorite/site
In your case it seems the retry delay got applied to the favorite/site instead of the item. Thus all items form the same favorite/site have to wait until the favorite/site retry delay is over. The following list shows in what situation the retry delay is applied:
item: transfer timeout, permission denied, wrong RETR/STOR return codes
favorite/site: control connection failed, resolve of host failed, login failed
What we are trying to figure out is under what circumstances the Transfer Queue wrongly applies the retry delay to the favorite/site.

>3
You can exclude directories/files in the Filter settings in the favorite.

>Version 3.x
What synchronization feature are you looking forward? One way synchronization and One way synchronization with delete is already possible with the latest version (2.5.1005.33).

Regards,
-Mat