mattt1986
400097040 license number
I have a large number of small files to transfer (thousands). Having multiple workers really helps speed things up. However, since having upgraded to v3 I have noticed that the queue will spend large amounts of time idle even when there are items in it and the queue is running.
Here is how it usually works:
1. Items are cued and cue starts
2. 5-10 workers start working
3. One item fails and is retried, no more workers begin working
4. Once the 5-10 active workers have completed, the cue stops and does not continue until the retried item is completed (30 seconds - 1 min) at which point the cycle repeats.
This leaves LOTS and LOTS of idle time in which I could be uploading other files.
Does this make sense? What can I do to fix this? I have tried in both Active and Passive modes and nothing works.
I have a large number of small files to transfer (thousands). Having multiple workers really helps speed things up. However, since having upgraded to v3 I have noticed that the queue will spend large amounts of time idle even when there are items in it and the queue is running.
Here is how it usually works:
1. Items are cued and cue starts
2. 5-10 workers start working
3. One item fails and is retried, no more workers begin working
4. Once the 5-10 active workers have completed, the cue stops and does not continue until the retried item is completed (30 seconds - 1 min) at which point the cycle repeats.
This leaves LOTS and LOTS of idle time in which I could be uploading other files.
Does this make sense? What can I do to fix this? I have tried in both Active and Passive modes and nothing works.