title: Hadoop hdfs DistributedFileSystem could not be instantiated
body: <p>I have set up a hadoop hdfs cluster and since I am new to hadoop, I've been trying to follow a simple example to read/write from/to hdfs from a java driver program I am writing in my local machine. The example I am trying to test is the following:</p>
  
  <pre><code>public static void main(String[] args) throws IOException {
  
      args = new String[3];
      args[0] = "add";
      args[1] = "./files/jaildata.csv";
      args[2] = "hdfs://&lt;Namenode-Host&gt;:&lt;Port&gt;/dir1/dir2/";        
      if (args.length &lt; 1) {
          System.out.println("Usage: hdfsclient add/read/delete/mkdir [&lt;local_path&gt; &lt;hdfs_path&gt;]");
          System.exit(1);
      }
  
      FileSystemOperations client = new FileSystemOperations();
      String hdfsPath = "hdfs://&lt;Namenode-Host&gt;:&lt;Port&gt;";
  
      Configuration conf = new Configuration();
      conf.addResource(new Path("file:///user/local/hadoop/etc/hadoop/core-site.xml"));
      conf.addResource(new Path("file:///user/local/hadoop/etc/hadoop/hdfs-site.xml"));
  
      if (args[0].equals("add")) {
          if (args.length &lt; 3) {
              System.out.println("Usage: hdfsclient add &lt;local_path&gt; &lt;hdfs_path&gt;");
              System.exit(1);
          }
          client.addFile(args[1], args[2], conf);
  
      } else {
          System.out.println("Usage: hdfsclient add/read/delete/mkdir [&lt;local_path&gt; &lt;hdfs_path&gt;]");
          System.exit(1);
      }
      System.out.println("Done!");
  }
  </code></pre>
  
  <p>Where the <code>addFile</code> function is the following:</p>
  
  <pre><code>public void addFile(String source, String dest, Configuration conf) throws IOException {
  
      FileSystem fileSystem = FileSystem.get(conf);
  
      // Get the filename out of the file path
      String filename = source.substring(source.lastIndexOf('/') + 1, source.length());
  
      // Create the destination path including the filename.
      if (dest.charAt(dest.length() - 1) != '/') {
          dest = dest + "/" + filename;
      } else {
          dest = dest + filename;
      }
      Path path = new Path(dest);
      if (fileSystem.exists(path)) {
          System.out.println("File " + dest + " already exists");
          return;
      }
  
      // Create a new file and write data to it.
      FSDataOutputStream out = fileSystem.create(path);
      InputStream in = new BufferedInputStream(new FileInputStream(new File(source)));
  
      byte[] b = new byte[1024];
      int numBytes = 0;
      while ((numBytes = in.read(b)) &gt; 0) {
          out.write(b, 0, numBytes);
      }
  
      // Close all the file descriptors
      in.close();
      out.close();
      fileSystem.close();
  }
  </code></pre>
  
  <p>The project is a maven project, with <code>hadoop-common-2.6.5</code>, <code>hadoop-hdfs-2.9.0</code> and <code>hadoop=hdfs-client 2.9.0</code> added to dependencies and configured to build the jar with all the dependencies included.</p>
  
  <p>My problem, no matter the different demo examples I have tried is that I get the following exception at the point where the <code>FileSystem</code> gets created at <code>FileSystem fileSystem = FileSystem.get(conf);</code>:</p>
  
  <pre><code>Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.DistributedFileSystem could not be instantiated
  at java.util.ServiceLoader.fail(ServiceLoader.java:232)
  at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
  at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
  at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
  at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
  at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:2565)
  at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2576)
  at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2593)
  at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
  at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632)
  at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614)
  at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
  at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:169)
  at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:354)
  at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296)
  Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/fs/FSDataOutputStreamBuilder
  </code></pre>
  
  <p>I have no clue on how to go through and I've tried each of the few solutions I've seen on the net, so I would be grateful for any advice on that...</p>
  
  <p>Thanks.</p>