title: Spark Library Conditional Import
body: <p>Spark - Databricks jobs fail occasionally due to slow cluster library import. The notebook execution starts before the library is fully initialized and the entire process fails. If the same notebook is executed shortly after the failure, import proceeds successfully w/o any other intervention needed.</p>
  <p>I was trying using try/catch approach, but the imported objects are unsurprisingly available only within the try block:</p>
  <pre><code>var tries = 0
  var importSuccessful = false
  
  while(!importSuccessful &amp;&amp; tries &lt; 3){
    try {
      import org.fde.dataFoundation.helper
      import org.fde.dataFoundation.transactionMapping._
      
      importSuccessful = true
    } catch{
      case e:Exception =&gt; println(&quot;Write failed with error: &quot; + e.getMessage)
      tries += 1
      if(tries &gt;= 3) {
        throw new Exception(s&quot;FD&amp;E Data Foundation library couldn't be imported after ${tries - 1} attempts. Check if library ocrDataFoundationS3 is installed on the cluster&quot;)
      }
      
      Thread.sleep(5*60*1000) //wait 5 minutes
    }
  }
  </code></pre>
  <p>Could you please recommend an option how to execute <code>try/catch</code> cycle and make the library available for the entire netbook?</p>
  <p><em>Note: the internal process rules require use of libraries installed on VC.</em></p>