Folders with lots of files

ccondrup

2008-07-04 22:02:49

Hi there, thanks for a great product! I am currently using 3.0 and have been using SmartFTP for years, the main reason being the workers. Perfect for lots of small files.

Problem 1:
Displaying folders contents where there are more than 2000 files. I have a /logs folder with 5534 files in it, they range from 0-300 kilobytes in size. No matter what I try, I only get the first 2000 files, alphabetically sorted. I tried to sort by date desc, refreshing and still was presented with the same 2000 files, they were now sorted internally by date, but the other (newer) 2534 files were still not there - the newest visible was dated 2008.04.25. I know the newest file was modified today, so I would expect that if there is a problem with listing more than 2000 files, at least by sorting the columns I would get a say in which 2000 files gets "picked" - well, I didn't.

Problem 2:
Tried to download the entire /logs folder - I got only 2000 files. I then selected another folder above /logs, quit smartftp, restarted (and get my workspace restored, nice!) - hit refresh to be sure not to get a cached version, drag'n'dropped the /logs folder without opening it - still got only 2000 files.

OK, I understand there might be lag/cpu problems when displaying lots of files, but I do expect to get all the files in a folder when downloading it. I also couldn't find anywhere to adjust this setting. I tried to search and browse both the KB and the forums for problems/solutions, but no go. For the record: I have -a (all files) selected, but none of these log files are hidden.

Problem 3:
Problems 1 & 2 might just be me not being able to find the correct setting. Believe me, I looked through all the options several times.
Part of the problem might be that I often find it hard to adjust settings in SmartFTP because the categories under Tools>Settings do not contain the same as when choosing Properties for a favorite;
1. I open properties for my favorite, and go to Transfer>Files - I see that they're all set to default. (there could be a link in this screen: "Edit defaults")
2. OK, so I'll go check those default settings. I open Tools>Settings, and go to Transfer, but it does not at all contain the same items as the Transfer category under favorites did. This is very confusing, please have this in mind when working on the categorization/UI next time, thanks.

Here is my system info
+- System -----------------------------

Microsoft Windows XP Professional 

Service Pack 3 (Build 2600)



CPU Speed		   : 2405 MHz

Total Memory		: 2047 MB

Free Memory		 : 2037 MB



+- SmartFTP ---------------------------

Version			 : 3.0.1019.7

Time Stamp		  : 2008-07-03 21:25:18

Platform			: x86

Id				  : 400032362

Days in use		 : 35



+- Application DLL --------------------

sfFTPLib.dll		: 1.5.13.32

sfFavorites.dll	 : 1.0.15.7

sfFavoritesShellExtension.dll : 1.0.1.13

sfTransferQueue.dll : 1.0.12.7

sfFTPShellExtension.dll : 1.0.10.7



+- Language ---------------------------

SmartFTP.exe		: 3.0.1019.7



+- Internet Explorer ------------------

Version			 : 7.0.5730.13



+- Winsock ----------------------------

Winsock			 : 2.2

mb

2008-07-05 01:13:26

Hello ..

The server only returns the first 2000 files. This is some kind of default setting on the server. You can contact your server administrator and ask him to remove or increase this limit. There is nothing you can do on the client side.

For example pureftpd which is FTP server that is used a lot has the following in its readme:

- '-L <max files>:<max depth>': To avoid stupid denial-of-service attacks
(or just CPU hogs), Pure-FTPd never displays more than 2000 files in response
to an 'ls' command. Also, a recursive 'ls' (-R) never goes further than 5
subdirectories. You can increase/decrease those limits with the '-L' option.

Reference: http://download.pureftpd.org/pub/pure-ftpd/doc/README

Regards,
Mat
SmartFTP

ccondrup

2008-07-05 03:51:32

Thanks for this prompt reply, the server admin worked out a workaround for me and all is well - great!

I hope you still take note of my pt. 3 here and keep it in mind for later - those config-screens are way behind the rest of the application from a usability point of view - meaning the rest is quite superb, but the settings dialogues could use a restructuring!

Thanks again!
- Chris

mb

2008-07-05 04:21:39

While the settings might be confusing for you, it's actually a very logical design:
- The application settings can be found in the menu: Tools->Settings. Settings for the interface, for the transfer queue, for the ident server. These are settings for objects that only exist once. e.g. you only have 1 transfer queue and you only have 1 interface.
- Then there are individual favorite settings, which only affect the Remote Browser part. E.g. the connection timeout, what proxy the favorite uses etc. These settings are not globally because you can have more than one object (Remote Browser respectively favorite) with different preferences and settings.
The favorite settings are inherited from the default favorite settings which you can access in every favorite dialog. The link can be found at the bottom left of each dialog.

Regards,
Mat