Improve folder monitoring by implementing delay

Got a good idea? Want to share it? Then please take a moment to submit it.
CraigBee
Posts: 20
Joined: Fri Aug 13, 2010 3:51 pm

Improve folder monitoring by implementing delay

Post by CraigBee » Thu Nov 11, 2010 4:58 pm

When a large file (~2GB) is copied to a folder being monitored by SmartFTP, it may upload several times. I have SmartFTP configured to use 5 upload threads. When a 2GB file is copied to the NAS folder being monitored, SmartFTP will detect the file and start an upload before the file copy is finished. The file then 'changes' as more of the file is copied, and SmartFTP will start a second upload.

I think this might be avoidable by waiting the monitored folder to stop changing before processing the monitored folder. The logic could be something like this:
Step 1: Wait for change in monitored folder
Step 2: Remember current state of monitored folder
Step 3: Wait 20 seconds
Step 4: Check for change in monitored folder since step 2, if changed, goto Step 2
Step 5: Process monitored folder according to rules

Something like this would allow SmartFTP to wait for large files to finish being copied to the monitored folder before starting the upload. It would then only upload 1 time instead of multiple times.

Craig

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Improve folder monitoring by implementing delay

Post by mb » Thu Nov 11, 2010 5:02 pm

Duplicate transfer queue items should automatically be ignored. I will do some investigations why this is apparently not the case.

CraigBee
Posts: 20
Joined: Fri Aug 13, 2010 3:51 pm

Re: Improve folder monitoring by implementing delay

Post by CraigBee » Thu Nov 11, 2010 7:30 pm

I'm on version 4.0.1135.0 (64-bit). I'm also forcing all files to lower case. A bug related to file matching with forced lower case was fixed in 4.0.1134.0. Perhaps it is related.
Last edited by CraigBee on Thu Nov 11, 2010 7:31 pm, edited 1 time in total.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Improve folder monitoring by implementing delay

Post by mb » Thu Nov 11, 2010 7:40 pm

Please try it again with the latest version: https://www.smartftp.com/download

CraigBee
Posts: 20
Joined: Fri Aug 13, 2010 3:51 pm

Re: Improve folder monitoring by implementing delay

Post by CraigBee » Wed Dec 08, 2010 8:29 pm


Please try it again with the latest version: https://www.smartftp.com/download

When I check the virtual machine every few days, I often find it is crashed and a window pops up says "SmartFTP Client has stopped working. Windows can check online for a solution to the problem and try to restart the program.". I did not get this before updating to the most recent version. What log files or data can I provide to help analyze this crash?

The duplicate transfers are gone though

Craig
Last edited by CraigBee on Wed Dec 08, 2010 8:53 pm, edited 1 time in total.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Improve folder monitoring by implementing delay

Post by mb » Wed Dec 08, 2010 9:43 pm

Please install the latest version and try again:
32-bit
https://www.smartftp.com/download

64-bit
https://www.smartftp.com/download

If the problem persits and you have a way to reproduce it we can give you further instructions. Thanks.

CraigBee
Posts: 20
Joined: Fri Aug 13, 2010 3:51 pm

Re: Improve folder monitoring by implementing delay

Post by CraigBee » Tue Dec 14, 2010 2:29 pm

It still crashes. I have isolated it to some extent. The crash only occurs when connecting to a local Windows IIS FTP server. It seems random. Over long enough period, sometimes seconds, sometimes hours, it will eventually crash when uploading to the Windows IIS FTP server. The crash has never occurred when transferring to other FTP servers that are not local and of unknown software.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Improve folder monitoring by implementing delay

Post by mb » Tue Dec 14, 2010 7:18 pm

If you can reproduce it in a relativ short time (minutes) please create a crash dump as described in this article: https://www.smartftp.com/support/kb/how- ... f2594.html
Last edited by mb on Tue Dec 14, 2010 7:20 pm, edited 1 time in total.

CraigBee
Posts: 20
Joined: Fri Aug 13, 2010 3:51 pm

Re: Improve folder monitoring by implementing delay

Post by CraigBee » Wed Dec 15, 2010 7:54 pm


If you can reproduce it in a relativ short time (minutes) please create a crash dump as described in this article: https://www.smartftp.com/support/kb/how ... f2594.html

I was able to reproduce a crash twice within a few minutes of starting SmartFTP and emailed the 10K & 16K C:\Dump directory to support@smartftp.com. If that email can't receive attachments, let me know and I will find a place to upload the file.

After more testing, I am able to reproduce a crash on demand with the following steps using SmartFTP build 1148:
1. Start SmartFTP, ensure the queue and schedule is clear.
2. Add new item to queue to copy a folder with delete from NAS to any FTP server
3. Right click and hit schedule
4. Add Monitor Folder - instant crash

Craig
Last edited by CraigBee on Wed Dec 15, 2010 8:30 pm, edited 1 time in total.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: Improve folder monitoring by implementing delay

Post by mb » Wed Dec 15, 2010 10:24 pm

Thanks. Can you post your system information from the menu: Help->About "System Information"?

I couldn't reproduce it right away. What folder are you monitoring? UNC or "local" folder?
Last edited by mb on Wed Dec 15, 2010 10:25 pm, edited 1 time in total.

Post Reply