title: Selectively import from another Jupyter Notebook
body: <p>I arranged my Jupyter notebooks into: <code>data.ipynb</code>, <code>methods.ipynb</code> and <code>results.ipynb</code>. How can I selectively import cells from <em>data</em> and <em>methods</em> notebooks for use in the <em>results</em> notebook?</p>
  
  <p>I know of <code>nbimporter</code> and <code>ipynb</code> but neither of those offers selective import of variables. There is an option to import definitions - including variables that are uppercase - but this does not work for me as I would have to convert most of the variables in my notebooks to uppercase.</p>
  
  <p>I would rather import everything except for two or three cells that take a long time to evaluate. Ideally, I would like to defer the execution of some assignments to the very moment I access them (lazy evaluation) - but I understand that it might be difficult to implement.</p>
  
  <p>Here is the overview, in pseudocode (each line repesents a cell):</p>
  
  <p><code>data.ipynb</code>:</p>
  
  <pre><code>raw_data = load_data()
  dataset = munge(raw_data)
  describe(dataset)             # I want this line to be skipped at import
  </code></pre>
  
  <p><code>methods.ipynb</code>:</p>
  
  <pre><code>import data
  method = lambda x: x * x
  # showcase how the method works on a subset of the dataset
  method(data.dataset[:5])      # I want this line to be skipped at import
  </code></pre>
  
  <p><code>results.ipynb</code>:</p>
  
  <pre><code>import data
  import methods
  result = methods.method(data.dataset)
  describe(result)
  </code></pre>
  
  <p>The motivation is that my real <em>data</em> and <em>methods</em> notebooks:</p>
  
  <ul>
  <li>are way much longer and complicated, hence I want to use an import system</li>
  <li>there are only a couple of cells that take more than seconds to evaluate</li>
  </ul>
  
  <p>also, the <em>methods</em> notebook cannot be replaced with <code>methods.py</code> file. In fact, I have such a file which contains the implementation details of my method. The notebook is more of a place to specify default parameters, showcase how my method works and explain example results.</p>
  
  <p>This question is essentially a combination of:</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/questions/19564625/how-can-i-import-from-another-ipython-notebook">How can I import from another ipython-notebook?</a>, and</li>
  <li><a href="https://stackoverflow.com/questions/26494747/simple-way-to-choose-which-cells-to-run-in-ipython-notebook-during-run-all">Simple way to choose which cells to run in ipython notebook during run all</a></li>
  </ul>
  
  <p>I read through answers to both and none satisfied my requirements.</p>
  
  <p>In my answer below I present my solution that uses custom cell magics and monkey-patching. However, I would prefer a solution which allows specifying which cells/expressions to exclude/include not in the notebook of origin (e.g. <code>data.ipynb</code>) but in the target one (e.g. in <code>methods.ipynb</code>).</p>
  
  <p>For example, it could use regular expressions:</p>
  
  <pre><code># all variables starting with 'result' would be ignored
  nbimporter.options['exclude'] = '^result.*'
  </code></pre>
  
  <p>or (even better) lazy evaluation:</p>
  
  <pre><code># only `a` and `b` would be evaluated and imported
  from data import a, b
  </code></pre>
  
  <p>All ideas will be appreciated!</p>