How to use Apache spark as Query Engine? <p>i am using Apache Spark For Big data Processing. The data is loaded to Data frames from a Flat file source or JDBC source. The Job is to search specific records from the data frame using spark sql.</p> <p>So i have to Run the job again and again for new search terms. every time i have to submit the Jar files using spark submit to get the results. <strong>As the size of data is 40.5 GB it becomes tedious to reload the same data every time to data frame to get the results for different queries</strong>.</p> <p>so What i need is, </p> <blockquote> <ul> <li>a way if i can load the data in data frame once and query it multiple time with out submitting the jar multiple times ?</li> <li>if we could use spark as a search engine/ query engine?</li> <li>if we can load the data into data frame once and query the data frame remotely using RestAP</li> </ul> <p><strong>> The current configuration of My Spark Deployment is</strong></p> <ul> <li>5 node cluster.</li> <li>runs on yarn rm.</li> </ul> </blockquote> <p>i have tried to use spark-job server but it also runs the job every time.</p>