title: Flink - No FileSystem for scheme: hdfs
body: <p>I am currently developing a Flink 1.4 application that reads an Avro file from a Hadoop cluster. However, running it in local mode on my IDE works perfectly fine. But when I submit it to the Jobmanager Flink it always fails with the following message:</p>
  
  <pre><code>java.io.IOException: Error opening the Input Split hdfs://namenode/topics/CaseLocations/partition=0/CaseLocations+0+0000155791+0000255790.avro [0,16549587]: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded.
  at org.apache.flink.api.common.io.FileInputFormat.open(FileInputFormat.java:705)
  at org.apache.flink.formats.avro.AvroInputFormat.open(AvroInputFormat.java:110)
  at org.apache.flink.formats.avro.AvroInputFormat.open(AvroInputFormat.java:54)
  at org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:145)
  at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
  at java.lang.Thread.run(Thread.java:748)
  Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded.
  at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:405)
  at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:320)
  at org.apache.flink.api.common.io.FileInputFormat$InputSplitOpenThread.run(FileInputFormat.java:864)
  Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Hadoop File System abstraction does not support scheme 'hdfs'. Either no file system implementation exists for that scheme, or the relevant classes are missing from the classpath.
  at org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:102)
  at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:401)
  ... 2 more
  Caused by: java.io.IOException: No FileSystem for scheme: hdfs
  at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2798)
  at org.apache.flink.runtime.fs.hdfs.HadoopFsFactory.create(HadoopFsFactory.java:99)
  ... 3 more
  </code></pre>
  
  <p>I am running the cluster with the official Flink Docker image <code>flink:1.4.0-hadoop28-scala_2.11</code> that should already contain a Hadoop distribution.</p>
  
  <p>I have also tried to add the dependencies to my application jar but that did not help either. Here are my sbt dependencies:</p>
  
  <pre><code>val flinkVersion = "1.4.0"
  val hadoopVersion = "2.8.1"
  val providedDependencies = Seq(
      "org.apache.flink" %% "flink-clients" % flinkVersion,
      "org.apache.flink" %% "flink-scala" % flinkVersion,
      "org.apache.flink" %% "flink-streaming-scala" % flinkVersion
  )
  val compiledDependencies = Seq(
      "org.apache.flink" % "flink-hadoop-fs" % flinkVersion,
      "org.apache.hadoop" % "hadoop-hdfs" % hadoopVersion,
      "org.apache.hadoop" % "hadoop-common" % hadoopVersion,
      "org.apache.flink" % "flink-avro" % flinkVersion,
      "org.apache.flink" %% "flink-table" % flinkVersion,
      "org.scalaj" %% "scalaj-http" % "2.2.1"
  )
  </code></pre>
  
  <p>Also, the filesystem classes are included in my <code>META-INF/services/org.apache.hadoop.fs.FileSystem</code>.</p>
  
  <p>Am I missing something? The official documentation couldn't help me.</p>
  
  <p>Thanks in advance</p>