overwrite in loop

This forum is not monitored by the SmartFTP staff. To contact technical support see KB: How to request technical support.
BruceLee
Posts: 23
Joined: Fri Feb 14, 2014 9:42 am

Re: overwrite in loop

Post by BruceLee » Tue Nov 08, 2016 5:25 pm

I have tested one random file from the "faulty" directory in the directory were the other files work fine.
And the file is not working there as well while all others do. How can I send you the file in private as ZIP? Thanks

BruceLee
Posts: 23
Joined: Fri Feb 14, 2014 9:42 am

Re: overwrite in loop

Post by BruceLee » Tue Nov 08, 2016 5:38 pm

I made further test. I took one random file from the "faulty" directory and cpoied it three time in the directory were the other files work fine.
one I left unchanged, second was just renamed to something simple like test and in the third one deleted the content.
The files with the name (empty or not) fail. the third one not.
Have you tested a file with Name pattern?
20160923 160000 01 {DC07C408-378E-410E-8062-1709E705FBE6}.log
Thanks

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: overwrite in loop

Post by mb » Tue Nov 08, 2016 5:39 pm

The name is irrelevant.
 
Are you testing everything with 1 worker? If not, test it with 1 worker.

BruceLee
Posts: 23
Joined: Fri Feb 14, 2014 9:42 am

Re: overwrite in loop

Post by BruceLee » Tue Nov 08, 2016 6:48 pm

yes, I'm testing with 1 worker (in favorite settings and transfer queue).
And the file name is somehow relevant. It is reproducable.
It happens whenever there is a space in the filename. But only with the mentioned files.
creating a new files with spaces are working fine.
 
can I track it down further? are there different types of blank spaces in filenames?

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: overwrite in loop

Post by mb » Tue Nov 08, 2016 7:07 pm

The file name is not relevant for SmartFTP, maybe for your file system.
Is the destination E: a local drive with NTFS?
 
We will log the times with milliseconds in the next build.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: overwrite in loop

Post by mb » Tue Nov 08, 2016 9:08 pm

The new build 2263 is available now:
https://www.smartftp.com/download
 
Please test it and post the log.

BruceLee
Posts: 23
Joined: Fri Feb 14, 2014 9:42 am

Re: overwrite in loop

Post by BruceLee » Wed Nov 09, 2016 10:41 am

tested. still the same. attached the logs. I have also send an email with the zipped file.

Please extract the empty logfile, copy it to your server and do a short test. Thanks

1st transfer creates file:

2016-11-09T09:58:38Z Operation begin
2016-11-09T09:58:38Z Copying file. Source="::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log", Destination="E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Getting properties (Size, Date modified) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Item: Size=4742, Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:38Z Transfer restarting at position 0.
2016-11-09T09:58:38Z 4742 bytes transferred. (73,5 KB/s) (63 ms)
2016-11-09T09:58:38Z Getting properties (Date modified, Date created) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Item: Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:38Z Setting properties (Date modified) of "E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Getting properties (Size) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Item: Size=4742
2016-11-09T09:58:38Z Using cached properties
2016-11-09T09:58:38Z Getting properties (Size) of "E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:38Z Item: Size=4742
2016-11-09T09:58:38Z Source File Size=4742, Destination file size=4742
2016-11-09T09:58:38Z Operation end

2nd transfer overwrites it:

2016-11-09T09:58:42Z Operation begin
2016-11-09T09:58:42Z Copying file. Source="::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log", Destination="E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Getting properties (Size, Date modified) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Item: Size=4742, Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:42Z Getting properties (Size, Date modified) of "E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Item: Size=4742, Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:42Z Source file: Size=4742, Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:42Z Destination file: Size=4742, Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:42Z No rule matched. Default action="Overwrite".
2016-11-09T09:58:42Z Transfer restarting at position 0.
2016-11-09T09:58:42Z 4742 bytes transferred. (94,5 KB/s) (49 ms)
2016-11-09T09:58:42Z Getting properties (Date modified, Date created) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Item: Date modified=2016-11-09T09:53:00.000
2016-11-09T09:58:42Z Setting properties (Date modified) of "E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Getting properties (Size) of "::{82AA9188-44E0-40B9-B956-43A10C315B4F}\::{6DDED4A4-3958-49C1-8E87-D9CE657B977A}/LOGS/20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Item: Size=4742
2016-11-09T09:58:42Z Using cached properties
2016-11-09T09:58:42Z Getting properties (Size) of "E:\LOGS\20161109 105301 01 {0B004120-59B8-4770-A035-D1D2DBB770BB}.log"
2016-11-09T09:58:42Z Item: Size=4742
2016-11-09T09:58:42Z Source File Size=4742, Destination file size=4742
2016-11-09T09:58:42Z Operation end

 
Last edited by BruceLee on Wed Nov 09, 2016 11:23 am, edited 1 time in total.

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: overwrite in loop

Post by mb » Wed Nov 09, 2016 11:29 am

It is a bug (not RFC 3659 compliant) with IIS's FTP server. It returns access denied if it the filename starts with a date string, followed by a space:
MDTM 20161109 094300.log
550 Access is denied.
 
It means that SmartFTP is not able to retrieve the exact date/time (STAT, LIST only returns minutes and no seconds) for this file and as a consequence, none of the file exist rules match (time != equal).
Last edited by mb on Wed Nov 09, 2016 11:49 am, edited 1 time in total.

BruceLee
Posts: 23
Joined: Fri Feb 14, 2014 9:42 am

Re: overwrite in loop

Post by BruceLee » Wed Nov 09, 2016 1:41 pm

thanks a lot. "thanks" IIS.
The glory file name :)
So my options are renaming files, changing server or fixing creation of logs to some other pattern.
any chance reagrding this topic:
>stuck in queue with "the system cannot find the file specified"?
You need to rethink your process if you have concurrent writers/readers.
thanks. its just the case if the sync of previous files take so long that another process has deleted some old files that do not have to be synced anymore.
is there an option to kick an item out if queue after X failed retries?
or something else?
thanks a lot

mb
Posts: 12030
Joined: Thu Oct 11, 2001 1:24 pm
Display name: Mat (SmartFTP)
Contact:

Re: overwrite in loop

Post by mb » Wed Nov 09, 2016 3:39 pm

>is there an option to kick an item out if queue after X failed retries?
No. But if they reach the max retry count, their state changes to "failed" and they will be ignored by the transfer queue.

Post Reply