title: Why does os.scandir() slow down/ how to reorganize huge directory?
body: <p>I have a directory with 3 million+ files in it (which I should have avoided creating in the first place). Using os.scandir() to simply print out the names,</p>
  <pre><code>for f in os.scandir():
      print(f)
  </code></pre>
  <p>takes .004 seconds per item for the first ~200,000 files, but drastically slows down to .3 seconds per item. Upon trying it again, it did the same thing- fast for the first ~200,000, then slowed way down.</p>
  <p>After waiting an hour and running it again, this time it was fast for the first ~400,000 files but then slowed down in the same way.</p>
  <p>The files all start with a year between 1908 and 1963, so I've tried reorganizing the files using bash commands like</p>
  <pre><code>for i in {1908..1963}; do 
  &gt; mkdir ../test-folders/$i; 
  &gt; mv $i* ../test-folders/$i/; 
  &gt; done
  </code></pre>
  <p>But it ends up getting hung up and never making it anywhere...</p>
  <p>Any advice on how to reorganize this huge folder or more efficiently list the files in the directory?</p>