title: Spark keeps streaming from local directory even when there is no file or data in the directory
body: <p>I have a challenge with my code. spark keeps streaming even when there is no data or files in my local directory .I am running spark on  my local system though I will still deploy to production .I am new to spark anyway. Even when I added three files ,spark is only picking one and  it does not stream the remaining but my concern is the continuous streaming. Please how do I control this and stop this continous streaming when there is no file or data ?I am using spring  boot with java .Below are my source code:</p>
  <pre><code>@Component
  public class FileConverter {
  
      @Value(&quot;${output.path}&quot;)
      protected String source;
      @Autowired
      SparkSessionConfig session;
  
      public void processfileandconverttoexcel() throws TimeoutException, StreamingQueryException {
          StreamingQuery query = null;
          SparkSession sessionconfig = session.getSession();
          System.out.println(sessionconfig.version());
          // define schema type of file data source
          sessionconfig.sparkContext().setLogLevel(&quot;ERROR&quot;);
          StructType schema = new StructType().add(&quot;filedata&quot;, DataTypes.StringType, true);
          // build the streaming data reader from the file source, specifying csv file
          Dataset&lt;Row&gt; rawData = sessionconfig.readStream().format(&quot;csv&quot;).schema(schema)
                  .csv(source);
          rawData.createOrReplaceTempView(&quot;filedata&quot;);
          Dataset&lt;Row&gt; results = sessionconfig.sql(&quot;select * from filedata&quot;);
          StreamingQuery query1 = rawData.writeStream().format(&quot;console&quot;).outputMode(OutputMode.Update()).start();
          query1.awaitTermination();
          System.out.println(&quot;end ofjob ..............&quot;);
      }
        *this test method when tried and called for streaming, is not even reading any files but continues to stream*
      public void test() throws InterruptedException {
          SparkConf conf = new SparkConf().setMaster(&quot;local[5]&quot;).setAppName(&quot;NetworkWordCount&quot;);
          JavaStreamingContext jssc = new JavaStreamingContext(conf, Durations.seconds(1));
          JavaDStream&lt;String&gt; lines = jssc.textFileStream(source);
          lines.print();
          lines.foreachRDD((rdd, time) -&gt; {
              ;
              if (!rdd.isEmpty()) {
                  lines.print();
                  rdd.foreach(f -&gt; {
                      System.out.println(f);
                  });
              }
          });
          jssc.start(); // Start the computation
          jssc.awaitTermination(); // Wait for the computation to terminate
      }
  }
  @Component
  public class SparkSessionConfig {
     
      public SparkSession getSession() {
          System.out.println(&quot;creating spark session ,,,,,,,,,,,,,,, &quot;);
           System.setProperty(&quot;hadoop.home.dir&quot;, &quot;C:/winutils&quot;);
           SparkSession spark  = SparkSession.builder().appName(&quot;File Converter&quot;)
                   .config(&quot;spark.sql.dir.warehouse.dir&quot;,&quot;file:///c:/tmp/&quot;)
                   .master(&quot;local[5]&quot;)
                   .getOrCreate()  ;
           System.out.println(&quot;spark session created ,,,,,,,,,,,,,,, &quot;);
          return spark;
          
      }
  }
  </code></pre>
  <h1>This is my console output in summary</h1>
  <pre><code>2021-01-13 03:09:42.711  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  2021-01-13 03:09:42.723  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  2021-01-13 03:09:42.734  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  2021-01-13 03:09:42.736  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.streaming.MicroBatchExecution  : Streaming query made progress: {
    &quot;id&quot; : &quot;2445367b-fc80-416e-9ff5-0cddd1af7bb2&quot;,
    &quot;runId&quot; : &quot;1dc94f26-0bde-4447-b204-7d949befed27&quot;,
    &quot;name&quot; : null,
    &quot;timestamp&quot; : &quot;2021-01-13T11:09:42.733Z&quot;,
    &quot;batchId&quot; : 1,
    &quot;numInputRows&quot; : 0,
    &quot;inputRowsPerSecond&quot; : 0.0,
    &quot;processedRowsPerSecond&quot; : 0.0,
    &quot;durationMs&quot; : {
      &quot;latestOffset&quot; : 2,
      &quot;triggerExecution&quot; : 2
    },
    &quot;stateOperators&quot; : [ ],
    &quot;sources&quot; : [ {
      &quot;description&quot; : &quot;FileStreamSource[file:/C:/Users/DELL/Desktop/sourcefolders]&quot;,
      &quot;startOffset&quot; : {
        &quot;logOffset&quot; : 0
      },
      &quot;endOffset&quot; : {
        &quot;logOffset&quot; : 0
      },
      &quot;numInputRows&quot; : 0,
      &quot;inputRowsPerSecond&quot; : 0.0,
      &quot;processedRowsPerSecond&quot; : 0.0
    } ],
    &quot;sink&quot; : {
      &quot;description&quot; : &quot;org.apache.spark.sql.execution.streaming.ConsoleTable$@2ab718be&quot;,
      &quot;numOutputRows&quot; : 0
    }
  }
  2021-01-13 03:09:42.748  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  2021-01-13 03:09:42.761  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  2021-01-13 03:09:42.773  INFO 12784 --- [4-7d949befed27]] o.a.s.s.e.datasources.InMemoryFileIndex  : It took 1 ms to list leaf files for 1 paths.
  </code></pre>
  <p>Please guys help look into .</p>