title: PySpark broadcast variables from local functions
body: <p>I'm attempting to create broadcast variables from within Python methods (trying to abstract some utility methods I'm creating that rely on distributed operations). However, I can't seem to access the broadcast variables from within the Spark workers.</p>
  
  <p>Let's say I have this setup:</p>
  
  <pre><code>def main():
      sc = SparkContext()
      SomeMethod(sc)
  
  def SomeMethod(sc):
      someValue = rand()
      V = sc.broadcast(someValue)
      A = sc.parallelize().map(worker)
  
  def worker(element):
      element *= V.value  ### NameError: global name 'V' is not defined ###
  </code></pre>
  
  <p>However, if I instead eliminate the <code>SomeMethod()</code> middleman, it works fine.</p>
  
  <pre><code>def main():
      sc = SparkContext()
      someValue = rand()
      V = sc.broadcast(someValue)
      A = sc.parallelize().map(worker)
  
  def worker(element):
      element *= V.value   # works just fine
  </code></pre>
  
  <p>I'd rather not have to put all my Spark logic in the main method, if I can. Is there any way to broadcast variables from within local functions and have them be globally visible to the Spark workers?</p>
  
  <p>Alternatively, what would be a good design pattern for this kind of situation--e.g., I want to write a method specifically for Spark which is self-contained and performs a specific function I'd like to re-use?</p>