title: How to calculate rolling covariance matrix from a spark dataframe
body: <p>I have a Spark 2.2.0 DataFrame of currency prices where I add the returns to. </p>
  
  <pre><code>import org.apache.spark.sql.SparkSession
  import org.apache.spark.sql.expressions.Window
  import org.apache.spark.sql.functions._
  
  val spark = SparkSession.builder.getOrCreate()
  val prices = spark.read.json("prices.json")
  
  // make a window function and convert prices to returns
  val window = Window.partitionBy("currency").orderBy("time")
  val lagPrice = lag(col("close"), 1).over(window)
  val percentReturn = col("close") / col("lastClose") - 1d
  val logReturn = log(col("close") / col("lastClose"))
  
  val returns = prices.withColumn("lastClose", lagPrice)
                      .withColumn("return", percentReturn)
                      .withColumn("logReturn", logReturn)
  </code></pre>
  
  <p>Now I want to calculate a rolling Covarance Matrix (like a moving average) of all currencies using a window function. But I can not find any documentation or examples.</p>