Take the 2-minute tour ×

Does it matter how many files I keep in a single directory? If so, how many files in a directory is too many, and what are the impacts of having too many files? (This is on a Linux server.)

Background: I have a photo album website, and every image uploaded is renamed to an 8-hex-digit id (say, a58f375c.jpg). This is to avoid filename conflicts (if lots of "IMG0001.JPG" files are uploaded, for example). The original filename and any useful metadata is stored in a database. Right now, I have somewhere around 1500 files in the images directory. This makes listing the files in the directory (through FTP or SSH client) take a few seconds. But I can't see that it has any affect other than that. In particular, there doesn't seem to be any impact on how quickly an image file is served to the user.

I've thought about reducing the number of images by making 16 subdirectories: 0-9 and a-f. Then I'd move the images into the subdirectories based on what the first hex digit of the filename was. But I'm not sure that there's any reason to do so except for the occasional listing of the directory through FTP/SSH.

share|improve this question

19 Answers 19

up vote 305 down vote accepted


  • Maximum number of files: 268,173,300
  • Maximum number of files per directory: 216 - 1 (65,535)
  • Maximum file size: 2 GiB - 1 without LFS, 4 GiB - 1 with


  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum file size
    • Implementation: 244 - 26 bytes (16 TiB - 64 KiB)
    • Theoretical: 264 - 26 bytes (16 EiB - 64 KiB)
  • Maximum volume size
    • Implementation: 232 - 1 clusters (256 TiB - 64 KiB)
    • Theoretical: 264 - 1 clusters


  • Maximum number of files: 1018
  • Maximum number of files per directory: ~1.3 × 1020 (performance issues past 10,000)
  • Maximum file size
    • 16 GiB (block size of 1 KiB)
    • 256 GiB (block size of 2 KiB)
    • 2 TiB (block size of 4 KiB)
    • 2 TiB (block size of 8 KiB)
  • Maximum volume size
    • 4 TiB (block size of 1 KiB)
    • 8 TiB (block size of 2 KiB)
    • 16 TiB (block size of 4 KiB)
    • 32 TiB (block size of 8 KiB)


  • Maximum number of files: min(volumeSize / 213, numberOfBlocks)
  • Maximum file size: same as ext2
  • Maximum volume size: same as ext2


  • Maximum number of files: 232 - 1 (4,294,967,295)
  • Maximum number of files per directory: unlimited
  • Maximum file size: 244 - 1 bytes (16 TiB - 1)
  • Maximum volume size: 248 - 1 bytes (256 TiB - 1)
share|improve this answer
I assume these are the maximum number of files for the entire partition, not a directory. Thus, this information isn't too useful regarding the problem, because there'd be an equal number of files regardless of the method (unless you count directories as files). –  strager Jan 21 '09 at 19:28
Since we're in 2012 now, I think its time to make clear that ext4 doesn't have any limit concerning the number of subdirectories. Also maximum filesize grew to 16 TB. Furthermore, the overall size of the filesystem may be up to 1 EB = 1,048,576 TB. –  devsnd Jun 25 '12 at 23:13
On some devices, it seems that FAT32 only supports 256 files and folders in the root path. –  Nick T Jun 12 '13 at 21:53
Apparently, ext3 also has a limit of 60,000 files(or directories or links) per directory. I found out the hard way about this. –  stackular Mar 13 at 20:59
please add that ext3 (and ext2?) has a limit of 32k-2 subdirs per dir: en.wikipedia.org/wiki/Ext3 –  qarma Aug 28 at 12:56

I have had over 8 million files in a single ext3 directory. libc readdir() which is used by find, ls, and most of the other methods discussed in this thread to list large directories.

The reason ls and find are slow in this case is that readdir() only reads 32K of directory entries at a time, so on slow disks it will require many many reads to list a directory. There is a solution to this speed problem. I wrote a pretty detailed article about it at: http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/

The key take away is: use getdents() directly -- http://www.kernel.org/doc/man-pages/online/pages/man2/getdents.2.html rather than anything that's based on libc readdir() so you can specify the buffer size when reading directory entries from disk.

share|improve this answer

It depends a bit on the specific filesystem in use on the Linux server. Nowadays the default is ext3 with dir_index, which makes searching large directories very fast.

So speed shouldn't be an issue, other than the one you already noted, which is that listings will take longer.

There is a limit to the total number of files in one directory. I seem to remember it definitely working up to 32000 files.

share|improve this answer
Gnome and KDE load large directories at a snails pace, windows will cache the directory so its reasonable. I love Linux, but kde and gnome are poorly written. –  Rook Apr 20 '10 at 2:26
And ext4 seems to have the equivalent of dir_index on by default. –  Prof. Falken Feb 22 '12 at 13:22
There is a limit of around 32K subdirectories in one directory in ext3, but the OP is talking about image files. There is no (practical?) limit on files in an ext3 file system with Dir Index enabled. –  Peter N Lewis May 31 '12 at 4:41

Keep in mind that on Linux if you have a directory with too many files, the shell may not be able to expand wildcards. I have this issue with a photo album hosted on Linux. It stores all the resized images in a single directory. While the file system can handle many files, the shell can't. Example:

-shell-3.00$ ls A*
-shell: /bin/ls: Argument list too long


-shell-3.00$ chmod 644 *jpg
-shell: /bin/chmod: Argument list too long
share|improve this answer
@Steve, use find(1) and/or xargs(1) for these cases. For the same reason it's a good idea to use such tools in scripts instead of command line expansion. –  Dave C Jan 21 '09 at 21:25
@Steve do you see performance going down when the number of files in a folder increase? Or is there no relation? –  Pacerier Mar 5 '12 at 16:40
This is a good point but to nitpick, the reason given is wrong. The Argument list too long is a limitation not of the shell, but of the system's exec implementation. The shell typically can expand the wildcard just fine - it's the call to exec with that many arguments that returns the error. –  jw013 Nov 30 '12 at 20:34
I had the same error last night (Fedora 15) with "rm" (somefiles*) with about ~400,000 files in a directory. I was able to trim the older files with "find" to the point where I could "rm" with a wildcard. –  PJ Brunet Mar 15 '13 at 3:47

I have a directory with 88,914 files in it. Like yourself this is used for storing thumbnails and on a Linux server.

Listed files via FTP or a php function is slow yes, but there is also a performance hit on displaying the file. e.g. www.website.com/thumbdir/gh3hg4h2b4h234b3h2.jpg has a wait time of 200-400 ms. As a comparison on another site I have with a around 100 files in a directory the image is displayed after just ~40ms of waiting.

I've given this answer as most people have just written how directory search functions will perform, which you won't be using on a thumb folder - just statically displaying files, but will be interested in performance of how the files can actually be used.

share|improve this answer
This is the only useful answer. We've made similar experiences. Our limit is 1.000 files to reduce problems with backups (too much directories slow down, too). –  mgutt Aug 1 '12 at 18:15
It can be useful to mount a drive with noatime as well: howtoforge.com/… and read this, too: serverfault.com/questions/354017/… –  mgutt Aug 1 '12 at 18:29
What filesystem are you using where it slows down so much? XFS, for example, should be able to easily handle 100,000 files in a directory without any noticeable slowdown. –  Ethan Mar 21 '13 at 15:38
Contradicting the opinion of most others, I want to confirm this answer. We have hundreds of thousands of images in our social network website. In order to improve the performance we were forced to have 100 (or 1000 for some files) sub directories and distribute the files into them (ext3 on linux+ Apache for us). –  wmac Jul 24 at 17:58

The biggest issue I've run into is on a 32-bit system. Once you pass a certain number, tools like 'ls' stop working.

Trying to do anything with that directory once you pass that barrier becomes a huge problem.

share|improve this answer

I'm working on a similar problem right now. We have a hierarchichal directory structure and use image ids as filenames. For example, an image with id=1234567 is placed in


using last 4 digits to determine where the file goes.

With a few thousand images, you could use a one-level hierarchy. Our sysadmin suggested no more than couple of thousand files in any given directory (ext3) for efficiency / backup / whatever other reasons he had in mind.

share|improve this answer

It absolutely depends on the filesystem. Many modern filesystems use decent data structures to store the contents of directories, but older filesystems often just added the entries to a list, so retrieving a file was an O(n) operation.

Even if the filesystem does it right, it's still absolutely possible for programs that list directory contents to mess up and do an O(n^2) sort, so to be on the safe side, I'd always limit the number of files per directory to no more than 500.

share|improve this answer

For what it's worth, I just created a directory on an ext4 file system with 1,000,000 files in it, then randomly accessed those files through a web server. I didn't notice any premium on accessing those over (say) only having 10 files there.

This is radically different from my experience doing this on ntfs a few years back.

share|improve this answer

It really depends on the filesystem used, and also some flags.

For example, ext3 can have many thousands of files; but after a couple of thousands, it used to be very slow. Mostly when listing a directory, but also when opening a single file. A few years ago, it gained the 'htree' option, that dramatically shortened the time needed to get an inode given a filename.

Personally, I use subdirectories to keep most levels under a thousand or so items. In your case, I'd create 256 directories, with the two last hex digits of the ID. Use the last and not the first digits, so you get the load balanced.

share|improve this answer
If the filenames were completely random, it wouldn't matter which digits were used. –  strager Jan 21 '09 at 19:30
Indeed, these filenames are generated randomly. –  Kip Jan 21 '09 at 19:58

The question comes down to what you're going to do with the files.

Under Windows, any directory with more than 2k files tends to open slowly for me in Explorer. If they're all image files, more than 1k tend to open very slowly in thumbnail view.

At one time, the system-imposed limit was 32,767. It's higher now, but even that is way too many files to handle at one time under most circumstances.

share|improve this answer

I ran into a similar issue. I was trying to access a directory with over 10,000 files in it. It was taking too long to build the file list and run any type of commands on any of the files.

I thought up a little php script to do this for myself and tried to figure a way to prevent it from time out in the browser.

The following is the php script I wrote to resolve the issue.

Listing Files in a Directory with too many files for FTP

How it helps someone

share|improve this answer

ext3 does in fact have directory size limits, and they depend on the block size of the filesystem. There isn't a per-directory "max number" of files, but a per-directory "max number of blocks used to store file entries". Specifically, the size of the directory itself can't grow beyond a b-tree of height 3, and the fanout of the tree depends on the block size. See this link for some details.


I was bitten by this recently on a filesystem formatted with 2K blocks, which was inexplicably getting directory-full kernel messages warning: ext3_dx_add_entry: Directory index full! when I was copying from another ext3 filesystem. In my case, a directory with a mere 480,000 files was unable to be copied to the destination.

share|improve this answer

I recall running a program that was creating a huge amount of files at the output. The files were sorted at 30000 per directory. I do not recall having any read problems when I had to reuse the produced output. It was on an 32-bit Ubuntu Linux laptop, and even Nautilus displayed the directory contents, albeit after a few seconds.

ext3 filesystem: Similar code on a 64-bit system dealt well with 64000 files per directory.

share|improve this answer

If the time involved in implementing a directory partitioning scheme is minimal, I am in favor of it. The first time you have to debug a problem that involves manipulating a 10000-file directory via the console you will understand.

As an example, F-Spot stores photo files as YYYY\MM\DD\filename.ext, which means the largest directory I have had to deal with while manually manipulating my ~20000-photo collection is about 800 files. This also makes the files more easily browsable from a third party application. Never assume that your software is the only thing that will be accessing your software's files.

share|improve this answer
I advertise against partitioning by date because bulk imports might cluster files at a certain date. –  mdorseif Jan 27 '09 at 21:31
A good point. You should definitely consider your use cases before picking a partitioning scheme. I happen to import photos over many days in a relatively broad distribution, AND when I want to manipulate the photos outside F-Spot date is the easiest way to find them, so it's a double-win for me. –  Sparr Jan 28 '09 at 0:28

I respect this doesn't totally answer your question as to how many is too many, but an idea for solving the long term problem is that in addition to storing the original file metadata, also store which folder on disk it is stored in - normalize out that piece of metadata. Once a folder grows beyond some limit you are comfortable with for performance, aesthetic or whatever reason, you just create a second folder and start dropping files there...

share|improve this answer

Not an answer, but just some suggestions.

Select a more suitable FS (file system). Since from a historic point of view, all your issues were wise enough, to be once central to FSs evolving over decades. I mean more modern FS better support your issues. First make a comparison decision table based on your ultimate purpose from FS list.

I think its time to shift your paradigms. So I personally suggest using a distributed system aware FS, which means no limits at all regarding size, number of files and etc. Otherwise you will sooner or later challenged by new unanticipated problems.

I'm not sure to work, but if you don't mention some experimentation, give AUFS over your current file system a try. I guess it has facilities to mimic multiple folders as a single virtual folder.

To overcome hardware limits you can use RAID-0.

share|improve this answer

There is no single figure that is "too many", as long as it doesn't exceed the limits of the OS. However, the more files in a directory, regardless of the OS, the longer it takes to access any individual file, and on most OS's, the performance is non-linear, so to find one file out of 10,000 takes more then 10 times longer then to find a file in 1,000.

Secondary problems associated with having a lot of files in a directory include wild card expansion failures. To reduce the risks, you might consider ordering your directories by date of upload, or some other useful piece of metadata.

share|improve this answer

I was trying to copy some mp3 files onto a USB stick. After 130 items Windows stopped the copying and displayed the message:

The root of drive D:\ is formatted with the FAT files system and can only store a imited number of items. Store the items in a subfolder or reformat the drive using the NTFS system.

These figures are several orders of magnitude smaller than the ones advised here.

Windows formatted the stick as "FAT(default)". Wondering about this I reformatted using FAT32. That's worked!

On another site I read:

A FAT32 directory can have 65,536 directory entries. Each file and subdirectory takes from two to thirteen entries, depending on the length of its name, so those entries can disappear long before you think you've used them all up."

Don't know how this sits with your numbers (again several orders of magnitude difference). I also haven't been able to find what the limits are of the "basic" FAT system Windows preselects/advises (FAT 8, 16?).

share|improve this answer

Your Answer


