title: What is the best practise for querying an big Spark result?
body: <p>I'm trying to build an Recommendation Engine for an Onlineshop with ca. 50000 Articles. I created the frequently itemsets and rules with the FPGrowth an Apache Spark.</p>
  
  <p>My first try was to put the Data (65G Rows) in a Database (PostgreSQL) as intarrays and with the gin Index the performance should be OK. But when Amount of rows is high the query need minutes. Low amounts needs ms.</p>
  
  <p>What are the best practices for querying a big Spark result?</p>