title: Why is recursive algorithm adding duplicate files?
body: <p>Once I have all my C drive indexed for fast searching it helps a lot having multiple threads for searching for files! The only problem is that I am finding duplicate files. I will appreciate if someone could explain me why I am having duplicate files in the following algorithm.</p>
  
  <pre><code>using System;
  using System.Collections.Generic;
  using System.Linq;
  using System.Threading;
  using System.Threading.Tasks;
  
  class Program
  {
      static volatile List&lt;System.IO.FileSystemInfo&gt; files = new List&lt;System.IO.FileSystemInfo&gt; ( );
      static readonly object lockFiles = new object ( );  // every time we read or write to files we use this lock
  
      static long numberOfThreadsRuning; // used to determine when we are done. if 0 threads runing it means we are done!
  
      static readonly int max_Number_OF_Threads = 8; 
  
      static void Main (  )
      {
          numberOfThreadsRuning = 0;
  
          // add first task this will increase numberOfThreadsRuning
          AddTask ( () =&gt; findFiles ( @"C:\Users\Antonio" ) );
  
          // while number of threads running is greater than 0
          while ( Interlocked.Read ( ref numberOfThreadsRuning ) &gt; 0 )   
              Thread.Sleep ( 100 );            
      }
  
      // start finding files in a new thread
      static void AddTask (Action task )
      {
          // increment numberOfThreadsRuning with a lock
          Interlocked.Increment ( ref numberOfThreadsRuning );
  
          Task.Factory.StartNew ( task ).ContinueWith ( x =&gt;
          {
              // once we are done executing the task decrement number of threads runing
              Interlocked.Decrement ( ref numberOfThreadsRuning );
          });
      }
  
      // recursively start finding files
      static void findFiles ( string path )
      {
          System.IO.FileSystemInfo[ ] subDirAndFiles;
  
          try {
              subDirAndFiles = new System.IO.DirectoryInfo (path).GetFileSystemInfos ( );
          }
          catch { // sometimes we are not able to access some directories so just ignore the error
              return;
          }
  
          foreach ( var file in subDirAndFiles )
          {
              lock(lockFiles)
                  files.Add ( file );
  
              if ( file is System.IO.DirectoryInfo ) // if it is a directory
              {
                  var numTaskRun = Interlocked.Read (ref  numberOfThreadsRuning );
  
                  if ( numTaskRun &lt; max_Number_OF_Threads )
                      AddTask ( ( ) =&gt; findFiles ( file.FullName ) ); // if there are 8 or less threads running create a new task
                  else
                      findFiles ( file.FullName ); // otherwise continue finding files in current thread
              }                
          }
      }
  }
  </code></pre>
  
  <p>please try and compare how much faster it is when I use 8 threads VS 1 by changing the static variable <code>max_Number_OF_Threads</code> to <code>1</code>.  Another way to prove that my algorithm is wrong, it always finds a different number of files <code>(files.Length = different number)</code> every time I have more than 1 thread.</p>