Global Queue is crashing

I used smartftp for a real long time and I found it's getting worse and worse.

I used to upload a lot of files to the server by using global queue. But I always got error message. I am using winxp pro and running 24 threads at the same time. Here is the message:

+-- OS --------------------------------------------
PlatformID = 2
Version = 5.1.2600

+-- SmartFTP --------------------------------------
Version = 1.0.978.2
Build Date = 2003-07-10 19:55:40

+-- Crash -----------------------------------------
Time Stamp = 20030713201901
hWnd = 0x0 (0)
nMsg = 0x0 (0)
wParam = 0x0 (0)
lParam = 0x0 (0)

Window Class = ''
Window Title = ''

Exception Code = 0xC0000005 (Access Violation)
Exception Flags = 0x0 (0)
Additional Exceptions = no
Address of Exception = 0x475E71 (4677233)
Number of additional parameters = 2
Parameter[0] = 0x0 (0)
Parameter[1] = 0x0 (0)

Call Stack:

0x00475e71

I used to use smartftp and found that it's a great program. But the recent development really make me disappointed.

By the way, your message board is full of problem too. It always said Invalid_session.

Hello ..

Please post a detailed bugreport with step-by-step how to reproduce the problem. Try to decrease the number of threads during your tests.

>By the way, your message board is full of problem too. It always said Invalid_session
For phpbb problems please make a bug report at http://www.phpbb.org

Regards,
SmartFTP.com

I got the same problem. While transfering files, no matter whether downloading or uploading, using global queue i got the same error. i coulnd reproduce the error step by step, but one thing is interessting. smartftp keeps downloading the files in the global queue even the error window is on top. In addition, the error seems to occure when global queue is under heavy stress with several threads on directory structure checking. it goes through all the folders and without any reason it goes pop and crashes.

Yep I saw one crash happen, out of about 10 tests already with 25+ threads.

Not a very easy one to produce, I can't find a pattern to cause the exception.

If any of you guys can find a exact way on how-to crash the G-Queue,
then please by all means reply with the results you have, thank you.

If you put more than 100 files in the GQ and each files are at least 10MB large. And you running 24 or even 30 threads at the same time. I think you will have the same problem. Sometimes, I even got buffer overflow error. But whenever which error, the upload process is still running.

This problem is not only in this version, but also some previous version. I just thought it will be fixed but it seems it never got fixed.

Some very old version don't have this problem.

Hi Maxbear,

We think the problem is nailed, will reply after some extensive testing.

Some buffer overflows were fixed for the "Global Queue".

Please install the 978.4 developer build, to verify that the
crashing is gone. I myself haven't seen any. Thank you.

https://www.smartftp.com/download

But I still got the exception handler error.

Hi maxbear!

I have spent some more time testing, and still can't crash the Global Queue, even with 50+ threads processing.

Can you please supply as much information as you can, that you haven't supplied to us yet.

Read Me:
https://www.smartftp.com/forums/viewtopic.php?t=6046

And what FTP server software is this occuring on?

And last, what file formats are being transferred?

Hi

I've the same problem when I'm downloading large amount of file. File type RAR, ZIP... on 2 servers :

1) Reply "Serv-U FTP Server v4.0 for WinSock ready" on connect

2) Reply of SYST and FEAT commands :
>SYST
215 UNIX Type: L8
>FEAT
211-Extension supported
MDTM
MDTM YYYYMMDDHHMMSS[+-TZ] filename
SIZE
SITE PSWD;EXEC;SET;INDEX;ZONE;CHMOD;MSG
REST STREAM
211 End


This is the exception report :
=============EXCEPTION=============
+-- OS --------------------------------------------
PlatformID = 2
Version = 5.0.2195

+-- SmartFTP --------------------------------------
Version = 1.0.979.0
Build Date = 2003-09-08 16:55:26

+-- Crash -----------------------------------------
Time Stamp = 20030915012815
hWnd = 0x0 (0)
nMsg = 0x0 (0)
wParam = 0x0 (0)
lParam = 0x0 (0)

Window Class = ''
Window Title = ''

Exception Code = 0xC0000005 (Access Violation)
Exception Flags = 0x0 (0)
Additional Exceptions = no
Address of Exception = 0x4751B1 (4673969)
Number of additional parameters = 2
Parameter[0] = 0x0 (0)
Parameter[1] = 0xC (12)

Call Stack:

0x004751b1 0x00474070 0x7c4e987c
==========END OF EXCEPTION===========

My connection is an ADSL connection and pass throw a BlackICE firewall. Downloaded data are verified by the McAfee VirusScan 6 Web Engine.

When I try to continu execution, program close.

My system information :
==============SYSTEM===============
+- System -----------------------------

Microsoft Windows 2000 Professional
Service Pack 4 (Build 2195)

CPU Speed : 1004 MHz
Total Memory : 785952 KB
Free Memory : 540452 KB

+- SmartFTP ---------------------------

Version : 1.0.979.0
Time Stamp : 2003-09-08 16:55:26

+- Application DLL --------------------

controls.dll : 1.6.979.0
ftpapi.dll : 2.0.979.0
language.dll : 1.0.979.0
reseng.dll : 1.3.979.0
smarthook.dll : 1.0.2.1

+- System DLL -------------------------

shell32.dll : 5.00.3700.6705
shlwapi.dll : 6.00.2800.1226
comctl32.dll : 5.81
riched20.dll : 5.30.23.1215
schannel.dll : 5.00.2195.6705

+- Internet Explorer ------------------

Version : 6.0.2800.1106

+- Winsock ----------------------------

Winsock : 2.2
===========END OF SYSTEM=============

If this can help you :

I've tried to debug program using Microsoft Debugging Tools (WinDbg). WinDbg report the following information with an exception on 004751b1 offset and loop on this exception without result :

===============WINDBG==============
(165c.1654): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=01c8fc2c ebx=00000000 ecx=011a651c edx=ffffffff esi=00000000 edi=011a651c
eip=004751b1 esp=01c8fbec ebp=01c8ff68 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010206
*** WARNING: Unable to verify checksum for C:Program FilesInternetSmartFTPSmartFTP.exe
*** ERROR: Module load completed but symbols could not be loaded for C:Program FilesInternetSmartFTPSmartFTP.exe
SmartFTP+0x751b1:
004751b1 8b460c mov eax,[esi+0xc] ds:0023:0000000c=????????
============END OF WINDBG===========

Olivier

PS : sorry for my bad english...

same thing happening here though i'm only running 4-6 threads, though queue is sometimes 200-300 long.

I can't reproduce a crash at all, with normal/stress testing.

This also seems to be the only report I have seen in over
a month, with the "Global Queue" crashing.

You guys have anything more to add that might be
causing this crash for you, within the Global Queue?

Hi

I've tried to run download with only 1 thread and no crash for more than one day...
I've tried to run download with 7 thread and no more than 1 thread per site and no crash...

I'm searching a way to reproduce this bug... I've "had" this bug for a long time and I'm tired...

Olivier

originally i thought mine was in reference to win2k3 as u'll notice the post 3 down, but then with further testing i can have it running for days on end with one thread up the theads and i'll get a crash the more threads the shorter the period between crashes.

also does anyone know why there seems to be a conflict with rdpclip.exe which from what i found is the clipboard monitor for windows, it seems cpu useage on this jumps to 16-20% and during this time sf doesn't repsond till i this process is closed.

Hi

Thank you destrkta, You're right.

So a way to reproduce problem in reasonable time :

Connect ftp.netscape.com
Add to queue all content of pub directory
Configure queue with more than 60 threads by site and same number of running threads
Click play
Wait no more then 15 minutes (after 3 tests on my computer) and CRASH...

Olivier

60 Threads?

We cannot debug the code with more than 3 parallel threads. The reason is very likely our code and/or the VC debugger.

It crashs almost instantly when increasing the threads further.

-Mat

Sorry, but with only 3 threads, SmartFTP can run during hours before crash...

Trying to process 60 or even higher threads is pretty much insane.
The max limit of threads, I would recommend will always be is 25.

Anything higher and your just asking for trouble. Not only in the
SmartFTP program, but in the windows platform all together.

By the way, I even tested your theory, and processed not only 60
threads from the netscape ftp server, but 61 threads, for 3 hours.
I actually took a nap in the whole process, was a good one to

As soon as I returned back to my computer, the threads were still
going strong, over 3 gb was transferred as well. (Not bad at all).

I then wanted to see how much higher I can push the threading process
and I set the threads per site to 80, and it went for a bit, but then it seems
windows and SmartFTP couldn't handle all of them given connections, and
I couldn't access my task bar. Was able to access the task manager though.

Thought I would share that. Anyhow enough with my experience.
If you find a normal way of crashing the Global Queue, please share.

Thank You!

Hi

Ok LM. This problem can be linked to another software or hardware.

Personnaly, I have (which can have conflict with SmartFTP) :

ASUS motherboard with VIA IDE controller
bi-Pentium III 1 Ghz
784 Mo of SDRAM
Seagate hard disk
SpeedTouch USB Modem for ADSL connection

Windows 2000 SP4
BlackICE as firewall
McAfee as anti-virus with system file scan and web download scan

What is the configuration of people who have same problem ???

Olivier

SmartFTP is unstable with a high number of threads (25+). This is a known problem and we're aware of it.

But this issue is rated very low because only a very tiny number of users are affected and a workaround exists (lower the number of threads).

Hi

It's not a problem of high number of threads.

I'm using 6 threads (2 thread per site) and I have the bug. I've told you to try with 60+ threads to have bug in a SHORT time. So, stop tell me it's not a normal way to use 25+ threads because it's stupid to use 25+ threads !!!!

Olivier

I cannot reproduce your bug either.

I have used smartftp for a long time with 6-7 threads at a time and NEVER had a crash.

Maybe it is an issue with SmartFTP but without really easy reproducable case or some pattern it's hard to fix it.

I hope this thread isn't dead yet, but I can consistently crash the Global Queue using SmartFTP 1.0.979.

Just enable the "Split Folder Contents to Separate Items" in tools -> settings ->Queue, and then connect to a linux distribution site and try to download the distribution (I used redhat/fedora mirrors).

I was able to reproduce the crash consistently using 10 and 5 threads (doesn't seem to matter how many threads, I think it's the amount of entries in the queue and not relatedc to the threads?).

A thread startes enumerating the directory contents, and after a minute or so of this, poof, it's dead.

Thanks

Sorry but this thread is pretty old already.

There hasn't been a reply for this thread in two months.

Please upgrade to the latest development build. (980.0)

The Guidelines already state this (very first sticky' thread) on
this forum. www.smartftp.com/phpBB2/viewtopic.php?t=6046