queue mixes up file/directory paths

i added a whole site to the queue. i then let it run with 60 (might have been 30) workers. in the morning come back and it is constantly retrying the remaining files. i view the log and the server is saying the files don't exist. i investigate. turns out the queue got these remaining file paths on the server wrong. for example

assume file paths on server

/a/1.html
/b/2.html

it had queued them as

/a/2.html
/b/1.html

Try to reproduce the problem and try to figure out where the problem can possibly be. If we can reproduce it we can fix it.
Thank you.
Regards,
Mat

add a site with a large number of directories/files to the queue then start it with 30 workers and just let it run. at the end itll be retrying files with the wrong path. it just happened again. this is a major bug. it means the program cant be used to reliably mirror a server or just download a large number of files/directories.

Try to reproduce the problem and try to figure out where the problem can possibly be. If we can reproduce it we can fix it.
Thank you.
Regards,
Mat

Well we cannot reproduce the problem. What version are you using?
Help->About System Information

You need to provide detailed instructions on how to reproduce it. Just saying that it doesn't work for you doesn't really help much.

Thank you
Regards,
Mat

version 2.5.1008

my recommendation for reproducing is setup an ftp server and have it serve a big directory on your computer like program files. then add that directory to the queue with 30 workers and see if it happens

Well we cannot reproduce the problem. What version are you using?
Help->About System Information

You need to provide detailed instructions on how to reproduce it. Just saying that it doesn't work for you doesn't really help much.

Thank you
Regards,
Mat

another problem is fie size mismatch. i think with this many threads they are trying to download the same file or something.

[11:24:26] Initializing. Transfer Queue v1.0.9.3.
[11:24:26] CWD /toorcastcom/wowkasts/abhayparekh/2556/_vti_cnf
[11:24:27] 250 CWD command successful.
[11:24:27] PWD
[11:24:27] 257 "/toorcastcom/wowkasts/abhayparekh/2556/_vti_cnf" is current directory.
[11:24:27] SIZE 3.html
[11:24:27] 213 57412
[11:24:27] MDTM 3.html
[11:24:27] 213 20070808175637
[11:24:27] PASV
[11:24:27] 227 Entering Passive Mode (209,67,188,6,11,44).
[11:24:27] Opening data connection to 209.67.188.6 Port: 2860
[11:24:27] RETR 3.html
[11:24:27] 125 Data connection already open; Transfer starting.
[11:24:28] 404 bytes transferred. (663 bytes/s) (609 ms)
[11:24:29] 226 Transfer complete.
[11:24:29] MDTM 3.html
[11:24:29] 213 20070808175637
[11:24:29] File size mismatch.


tried new version it happened again. let me know where i can send the smartftp log directory with all logs

We cannot reproduce the problem. What version are you using?
Help->About System Information

You need to provide detailed instructions on how to reproduce it.

Thank you
Regards,
Mat

i have done a screen video capture of the whole process. im encoding it now to flash. let me know where i should email the link to the video.


We cannot reproduce the problem. What version are you using?
Help->About System Information

You need to provide detailed instructions on how to reproduce it.

Thank you
Regards,
Mat

You can post it here. But please post your system information here as well.
We would rather prefer step by step instruction on how to reproduce it. The video by itself just shows that there is a problem somewhere.
Are you sure the server returns the correct paths for example? How about symbolic links etc?

Thank you
Regards,
Mat

the video is still encoding when it is done i will post it. the video shows the whole process all the steps which i have already summarized. here the steps are again.

1. connect to a ftp server.
2. add lots of files to queue (i just add the root directory)
3. start queue with 60 workers
4. wait for files in queue to say "retry again" status then check the log and the log will say the path doesnt exist
5. now wait check the queue there will be least one set of files with paths like a/b.html b/a.html where it should have been a/a.html and b/b.html

the video shows this. afaik there are no symbolic links and it happens with different directories/files every time. the ftp server is "Microsoft FTP Service" it is possible there is a bug in the server...

system info is

+- System -----------------------------
Microsoft Windows XP Professional
Service Pack 2 (Build 2600)

CPU Speed : 2394 MHz
Total Memory : 2047 MB
Free Memory : 766 MB

+- SmartFTP ---------------------------
Version : 3.0.1016.3
Time Stamp : 2008-05-12 21:59:01
Platform : x86
Id : 400038133
Days in use : 285

+- Application DLL --------------------
sfFTPLib.dll : 1.5.13.24
sfFavorites.dll : 1.0.12.3
sfFavoritesShellExtension.dll : 1.0.1.9
sfTransferQueue.dll : 1.0.9.3
sfFTPShellExtension.dll : 1.0.7.3

+- Language ---------------------------
SmartFTP.exe : 3.0.1016.3

+- Internet Explorer ------------------
Version : 7.0.5730.11

+- Winsock ----------------------------
Winsock : 2.2

You can post it here. But please post your system information here as well.
We would rather prefer step by step instruction on how to reproduce it. The video by itself just shows that there is a problem somewhere.
Are you sure the server returns the correct paths for example? How about symbolic links etc?

Thank you
Regards,
Mat

Thank you. Maybe you find out more details why the directory is wrong.

it doesnt seem to happen localy against servu. does your logging log the directory the server sends down?

Thank you. Maybe you find out more details why the directory is wrong.

Yes but you probably have to get it from the item that enumerates the directory. So you need to enable the log in the settings and then scan through all the files.

i dont have time to go through the logs. at this point i dont know if it is the server or smart ftp. i wasn't able to reproduce it locally with servu serving files. i guess close this till someone else reports it or id recommend setting up a MS ftp server and trying it. then with MS ftp server logs you can probably figure out where the bug is. also do network monitoring.

Yes but you probably have to get it from the item that enumerates the directory. So you need to enable the log in the settings and then scan through all the files.