title: Spark [ScalaTest]: Overall TDD steps for customised Spark-based transformers
body: <p>We all use customised transformers and estimators in our Spark pipelines. Creating a transformer or estimator is pretty strait forward in terms of value/param/method usage. For example in case of transformers: </p>
  
  <ol>
  <li>Use a set of parameters</li>
  <li>Use their set/get methods </li>
  <li>Use transformer, transformerSchema, and copy methods.   </li>
  </ol>
  
  <p>My question here is what are the overall TDD steps for writing transformers? The acceptance test is clear, get this DataFrame and return this DataFrame with this schema; however this does not provide full test coverage.</p>