title: Writing data into a text file using spark-submit does not work
body: <p>I have a set of data which is being expanded as per my requirement using spark. The expanded data (running in a loop) is to be written in a text file in HDFS and i am facing issues while doing so. Only the first two records are being written and it fails then on.</p>
  
  <p>Below is the Code i am using to process and write into text file: </p>
  
  <pre><code>
  
           def Hdfswrite(record:String)
          {
          //val timestamp = new java.text.SimpleDateFormat("yyyyMMdd-HH").format(new java.util.Date())
          val file = "/user/bhkp/sparkoutput6/sparkoutput12" + ".txt"
          val line = "\n" + record
          val config = new Configuration()
          val fs = FileSystem.get(config)
  try
          {
          val writer = fs.append(new Path(file))
          writer.write(line.getBytes)
          writer.close
          }
          catch
          {
            case t: Throwable => { val writer = fs.create(new Path(file))
            writer.write(line.getBytes)
            writer.close
            }
          }
          }
    def main(args: Array[String]){
      val tokenized = sc.textFile("/user/bhkp/hv_tables/sparktestdata12.txt").map(rec => (rec.split("\\^",-1)))
      tokenize.foreach(transfer)
  
    }
  </code></pre>
  
  <p>I am getting the following error :
  <strong>org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): ID mismatch. Request id and saved id.</strong></p>
  
  <p>I am not sure where the issue is occurring since the data is getting written when the input file has only 2 records, but starts to fail when the record count is 3 or more.</p>
  
  <p>When i try to print the output, it works perfectly fine.</p>
  
  <p>I am confused.</p>