title: Failing to load model using multiprocessing on windows
body: <p>This program works on Unix and I'm trying to transition it to windows.</p>
  <p>It uses multiprocessing and I understand it's an issue with being forced to use spawning on windows opposed to forking on linux for multiprocessing.</p>
  <p>It's to do with the subprocess loading the models that I load on the main thread.</p>
  <p>When <code>Consumer()</code> is called, during the <code>init()</code> it calls a function from another file that loads some tensorflow models.</p>
  <p><strong>Consumer()</strong></p>
  <pre><code>import os
  import time
  import sys
  from PIL import Image
  import subprocess
  from multiprocessing import Process, Queue
  import t2identify
  
  class Consumer:
  
      def __init__(self, frameSource, layer):
          self.directory = os.environ[&quot;directory&quot;]
          self.source = frameSource
          self.task = layer
          if layer == &quot;screen&quot;:
              self.layer = t2identify.identifyImage(&quot;apps&quot;)
          elif layer == &quot;detail&quot;:
          self.layer = &quot;detail&quot;
  
          self.imagesChecked = 0
          self.errorsFound = 0
  
          self.previousApp = &quot;none&quot;
          self.appDetail = {
              &quot;mobile&quot;: t2identify.identifyImage(&quot;mobile&quot;),
          }
  
      def start(self, state, queue):
          self.state = state
      
          consumer1 = Process(target=self.consumeImage, args=(queue,))
          consumer1.start()
      
          consumer2 = Process(target=self.consumeImage, args=(queue,))
          consumer2.start()
  
          consumer1.join()
          consumer2.join()
  </code></pre>
  <p><code>t2identify.identifyImage()</code> involves loading models.</p>
  <p><strong>t2identify.py</strong></p>
  <pre><code>import matplotlib.pyplot as plt
  import numpy as np
  import os
  from PIL import Image
  
  class identifyImage():
  
      def __init__(self, layer):
          import tensorflow as tf
          availableLayers = {
              &quot;apps&quot;:&quot;C:/Users/PycharmProjects/NNs/tf/appModel&quot;,
          }
          self.selectedLayer = availableLayers[layer]
          self.model = tf.saved_model.load(self.selectedLayer)
          self.label = self.loadLabels(availableLayers[layer]+&quot;/labels.txt&quot;)
          self.img_height = 224
          self.img_width = 224
  ...
  </code></pre>
  <p>I'm confident the issue is when the consumer subprocess starts, the models that are loaded here are loaded again, why it says they're not found I'm not sure.</p>
  <p><strong>main.py</strong></p>
  <pre><code>import pathos
  import multiprocessing
  import os
  import time
  import shutil
  from os import path
  from producer import Producer
  from consumer import Consumer
  from controller import Controller
  from tqdm import tqdm
  import re
  import sys
  from env import environmentVariables as Environment
  
  class Main:
  
  def start(self, test):
      self.createDirectories()
      screenQueue = multiprocessing.Queue()
      detailQueue = multiprocessing.Queue()
  
      self.producer = Producer([&quot;SavedFrames&quot;, &quot;ScreenFrames&quot;, &quot;DetailFrames&quot;, &quot;VisualFrames&quot;])
  
      self.producerProcess = multiprocessing.Process(target=self.producer.start,
                                              args=(self.producerEvent, self.producerFrameRate, self.state,
                                                    self.expected, self.iteration, self.testCaseNumber,
                                                    [screenQueue, detailQueue]))
  
      self.screenConsumer = Consumer(&quot;ScreenFrames&quot;, &quot;screen&quot;)
      # MODELS ARE LOADED
      self.detailConsumer = Consumer(&quot;DetailFrames&quot;, &quot;detail&quot;)
  
      self.screenConsumerProcess = multiprocessing.Process(target=self.screenConsumer.start,
                                                           args=(self.state, screenQueue))
      self.detailConsumerProcess = multiprocessing.Process(target=self.detailConsumer.start,
                                                           args=(self.state, detailQueue))
  
      try:
          # Set the new thread to run the controller which performs the test cases
          self.controllerStart = Controller(self.producerEvent, self.producerFrameRate, self.state, self.expected, self.iteration,
                                                  self.testCaseNumber, self.progress)
          self.controllerProcess = multiprocessing.Process(target=self.controllerStart.start, args=(test,))
  
      except:
          print(&quot;ERROR&quot;)
          return False
  
      self.producerProcess.start()
      # FAILS on starting screenConsumerProcess (see error)
      self.screenConsumerProcess.start()
      self.detailConsumerProcess.start()
      self.controllerProcess.start()
  
      self.producerProcess.join()
      self.screenConsumerProcess.join()
      self.detailConsumerProcess.join()
      self.controllerProcess.join()
  
      self.zipFiles()
      self.sendLogs()
      return True
  ...
  if __name__ == &quot;__main__&quot;:
      testing = Main()
      results = testing.start()
  </code></pre>
  <p>The error:</p>
  <pre><code>To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
  Traceback (most recent call last):
    File &quot;&lt;string&gt;&quot;, line 1, in &lt;module&gt;
    File &quot;AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py&quot;, line 105, in spawn_main
      exitcode = _main(fd)
    File &quot;\AppData\Local\Programs\Python\Python37\lib\multiprocessing\spawn.py&quot;, line 115, in _main
      self = reduction.pickle.load(from_parent)
    File &quot;\venv\lib\site-packages\keras\saving\pickle_utils.py&quot;, line 48, in deserialize_model_from_bytecode
      model = save_module.load_model(temp_dir)
    File &quot;\venv\lib\site-packages\keras\utils\traceback_utils.py&quot;, line 67, in error_handler
  Traceback (most recent call last):
    File &quot;C:/Users/PycharmProjects/project/src/main.py&quot;, line 353, in &lt;module&gt;
      raise e.with_traceback(filtered_tb) from None
    File &quot;\venv\lib\site-packages\tensorflow\python\saved_model\load.py&quot;, line 978, in load_internal
      results = testing.start(data)
    File &quot;/PycharmProjects/project/src/main.py&quot;, line 95, in start
      self.screenConsumerProcess.start()
    File &quot;\AppData\Local\Programs\Python\Python37\lib\multiprocessing\process.py&quot;, line 112, in start
      str(err) + &quot;\n You may be trying to load on a different device &quot;
  FileNotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://4e4e1c18-ece9-4d99-88c3-5c2ee965c92a/variables/variables
   You may be trying to load on a different device from the computational device. Consider setting the `experimental_io_device` option in `tf.saved_model.LoadOptions` to the io_device such as '/job:localhost'.
  </code></pre>
  <p>The models that are loaded aren't accessable by subprocesses. As <code>ram://4e4e1c18-ece9-4d99-88c3-5c2ee965c92a/variables/variables</code> is <code>temp_dir</code> in keras/pickle_utils.py <code>model = save_module.load_model(temp_dir)</code>. Which is where the error occured.</p>
  <p>Am I running out of ram? Or do I need to change some multiprocessing code now that I'm on windows.</p>
  <p>EDIT:</p>
  <p>I suspect it's most likely to do with windows reimporting everything once a new process starts (spawning). While doing this, the models are reloaded, and that's where the error occurs. I am still however unsure how to go about resolving this, apart from loading the models in main then passing the models into the subprocesses as parameters... which seems like a subpar solution.</p>
  <p>EDIT2:</p>
  <p>Now looking into using pathos which uses dill and not pickle. As I suspect the issue is with when I start the consumer process, the target is a class, which is not pickleable.</p>