title: Multithreaded FTP download
body: <p>I'm trying to process a certain FTP directory which holds several directories and in turn those directories have an arbitrary number of files. So what I'm trying to do is to have 1 thread for each of the subdirectories and each thread to be concerned with the respective sub-dir here is what I came up: </p>
  
  <pre><code>private void fetchFilesFromFTP() {
  
      try {
          client.connect("ftp.ncbi.nih.gov");
          client.login("anonymous", "anonymous");
  
          client.changeWorkingDirectory("genomes/Fungi");
  
          FTPFile dirs[] = client.listDirectories();
          dirsToDl.addAndGet(dirs.length); 
  
          for (final FTPFile ftpFile : dirs) {
              exec.execute(new Runnable() {
                  //process each FTP directory in a new thread
                  @Override
                  public void run() {
                      processFTPdir(ftpFile.getName());
                  }
              });
          }
      } catch (SocketException ex) {
          Logger.getLogger(FungiProcessor.class.getName()).log(Level.SEVERE, null, ex);
      } catch (IOException ex) {
          Logger.getLogger(FungiProcessor.class.getName()).log(Level.SEVERE, null, ex);
      } 
  }
  
  private void processFTPdir(String dir) {
  
      File f = new File(destination + File.separator + dir);
      if (!f.mkdirs()) {
          System.out.println("Error creating dir for " + dir);
          return;
      }
  
      FTPFile files[];
      try {
          //we are already in the correct directory
          files = client.listFiles(dir, new FTPFileFilter() {
  
              @Override
              public boolean accept(FTPFile ftpf) {
                  return ftpf.getName().endsWith(".gbk");
              }
          });
  
          for (FTPFile fTPFile : files) {
              FileOutputStream fout =  new FileOutputStream(destination + File.separator + dir + File.separator + fTPFile.getName());
              if (client.retrieveFile(dir + "/" + fTPFile.getName(), fout)) {
  
                  System.out.println("successfully downloaded");
                  fout.flush();
                  fout.close();
              }
              System.out.println(client.getReplyString());
          }
      } catch (IOException ex) {
          Logger.getLogger(FungiProcessor.class.getName()).log(Level.SEVERE, null, ex);
      } finally {
          if(dirsToDl.decrementAndGet() == 0) latch.countDown();
      }
  }
  </code></pre>
  
  <p>The sequential version of the code works - I can see files ending with .gbk actually being downloaded whereas with the multithreaded version only the respective subdirectories are created but no files are downloaded. I don't even get any errors. Is is possible that FTP doesn't support multiple file downloads at once?</p>