Jump to content


Photo

Memory Leak


  • Please log in to reply
32 replies to this topic

#1 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 27 October 2012 - 02:41 AM

After updating to the latest version of Smartftp, I have been hit with a memory leak which slows down my computer to a crawl. I'm running windows 7 64bit.

#2 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 27 October 2012 - 07:57 AM

Please provide steps on how to reproduce.

#3 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 27 October 2012 - 08:46 AM

Have absolutely no idea. After changing the multi-part transfer rate from 4 parts to 6, the leak sprung up. Now after changing it back to 4, the leak persists.

Edited by YouBringOnTheLulz, 27 October 2012 - 08:47 AM.


#4 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 27 October 2012 - 10:04 AM

If you don't use multi part transfers, do you still have the memory leak?

#5 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 27 October 2012 - 06:00 PM

Hey mb.

I can concur with the poster's memory issue, I'm having the exact same issue. When files are multi-parts of 4 or more, and the file is larger than 6GB, you have a huge memory spike, in my case all the way to 1GB+ and then the software crashed due to being out of memory. High CPU is what I have as well as the memory issue. If the file is 1 or 2GB or smaller, there does not same to be an issue as far as I could diagnose.

First saw this with build 1227 and now 1228. I upgraded to 1228 expecting 1227 issue to just be a fluke.

This high memory/CPU issue is happening on both Windows XP and Windows 7-x64.

Hope the info can help you with tracing the culprit.

Later,

Edited by Purgatory, 27 October 2012 - 06:00 PM.


#6 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 28 October 2012 - 09:49 AM

Please provide sufficient information on how to reproduce the problem:
- What protocol are you using? If FTP, is the data connection protected?
- Does it happen with 2.5GB files?
- Build 1228? Do you mean build 1278?

#7 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 28 October 2012 - 08:29 PM

Please provide sufficient information on how to reproduce the problem:


- What protocol are you using? If FTP, is the data connection protected?


Yes, downloading via SFTP over SSH. 10mins and 600+MB into the download, 1.03GB of RAM used and CPU spike to 42/45%. Either 4 Multi Parts or 6, same results.

- Does it happen with 2.5GB files?


Haven't tested with a file size of that size. Will create one and test shortly and reply with results.

- Build 1228? Do you mean build 1278?


Oh! Sorry about that, I meant to type build 1278 and 1277. Getting old. :P
*
I've just finished downloading that 16GB file via regular FTP port 21, no issues, memory use normal (6 to 10MB), CPU 2 to 3% use. MD5 hash shows the file is 100%.

#8 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 29 October 2012 - 12:54 AM

- Does it happen with 2.5GB files?

Ok, tested with a 2.65GB file, zero issues when downloading via SFTP over SSH with 6 Multi Parts for the file. Speed, memory and CPU all normal and as expected.

/End of report.

#9 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 29 October 2012 - 02:08 AM

I tried to reproduce it with a 92GB file, 4 multi part workers, SFTP over SSH. Unfortunately so far without success.

#10 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 29 October 2012 - 06:16 AM

Is there a trace option or debug I can turn on to test and send you the results?

#11 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 29 October 2012 - 12:20 PM

That won't help. We need to know how to trigger the problem.

#12 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 29 October 2012 - 04:43 PM

I tried to reproduce it with a 92GB file, 4 multi part workers, SFTP over SSH. Unfortunately so far without success.


Have you tried it with 6 parts? After changing my multi-part settings to 6, the problem had begun.

#13 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 30 October 2012 - 02:15 AM

Where can I download build 1274? Would like to test on same system to eliminate that as the cause. I know I would need to uninstall the newer build first.

#14 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 30 October 2012 - 02:20 PM

Did you try to monitor the process with Process Explorer?
http://technet.micro...s/bb896653.aspx

It tells you more about resource allocation (e.g. handles, private memory, etc). Maybe you will see something suspicious.

What you should also be aware of when downloading big files using multi part transfers is that the data is first filled with zeros (NTFS). Depending on the IO performance of your hard drive this may take several minutes. E.g. 100GB file / 30MB/s = 55 minutes.


#15 Purgatory

Purgatory
  • Members
  • 28 posts
  • Gender:
    Male

Posted 01 November 2012 - 07:40 PM

Hey again.

No issues with memory or handles etc that I could see (I use Process Hacker). Nothing else is new or has been installed except for SmartFTP over the last 2 months. Unless it was Windows Updates. Could an update from MS be the possible cause?

I'm going to test on my Win7 system later tonight, from the same download server, same file. That computer still has an older version of SmartFTP installed, version 4.1 build 1265 (64-bit).

The memory issue and high CPU use only came about with build 1277/78 on my my XP system. Only happens when multi parts of 4 or more are active (in my case). No issues when the file is under 6GB. Has anything changed with the SSH part or any other file(s) related to SSH?

I see no other reports of this issue, so besides the other guy above, no one else seems to be affected, or not reporting it here.

I uninstalled (completely) all traces of SmartFTP 4.1 build 1278, registry entries, application folder etc. Installed version 4.1 build 1274, same issues as first stated. Can't say if it is my XP system, perhaps I had build 126X before jumping to the 127X builds and therefore no issue with those builds. ?

I'm going to look for a build 126X, 32-bit and test my XP system again, later tonight. If I get the same high memory/CPU issues with a 126X build, that will convince me that there is something odd with my system.

I'll report back on the outcome/results for both my Win7 system with build 1265 (first test for this problem) and my XP system with build 126X (once I find one to install, 2nd test for this problem) by Saturday.

#16 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 01 November 2012 - 09:42 PM

You should definitively see the memory usage in Process Explorer. If not how do do you monitor the memory usage? Task Managers shows the same information as Process Explorer.

Between 1274 and the latest build there were no significant changes in the SSH implementation.

Thank you for your feedback.

#17 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 04 November 2012 - 07:49 PM

Posted Image
This is just with 2 parts on a 16gb file.

#18 mb

mb

    Developer

  • Administrators
  • 11521 posts
  • Gender:
    Male
  • Location:
    Worldwide

Posted 05 November 2012 - 02:50 PM

If you stop the transfer queue and wait for a minute, is the memory released?

#19 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 05 November 2012 - 08:56 PM

If you stop the transfer queue and wait for a minute, is the memory released?


The memory leak persists after aborting the transfer.

#20 YouBringOnTheLulz

YouBringOnTheLulz
  • Members
  • 16 posts

Posted 06 November 2012 - 08:32 PM

Even with multi-parts disabled, I'm experiencing high memory usage. Slowly the memory increases by about 200 - 400 bytes every 10 seconds or so. After a few hours it reaches levels which I have to restart my computer to fix.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users