title: How to use a Scala class inside Pyspark
body: <p>I've been searching for a while if there is any way to use a <code>Scala</code> class in <code>Pyspark</code>, and I haven't found any documentation nor guide about this subject.</p>
  
  <p>Let's say I create a simple class in <code>Scala</code> that uses some libraries of <code>apache-spark</code>, something like:</p>
  
  <pre><code>class SimpleClass(sqlContext: SQLContext, df: DataFrame, column: String) {
    def exe(): DataFrame = {
      import sqlContext.implicits._
  
      df.select(col(column))
    }
  }
  </code></pre>
  
  <ul>
  <li>Is there any possible way to use this class in <code>Pyspark</code>?</li>
  <li>Is it too tough?</li>
  <li>Do I have to create a <code>.py</code> file?  </li>
  <li>Is there any guide that shows how to do that?</li>
  </ul>
  
  <p>By the way I also looked at the <code>spark</code> code and I felt a bit lost, and I was incapable of replicating their functionality for my own purpose.</p>