title: How to efficiently run multiple Pytorch Processes / Models at once ? Traceback: The paging file is too small for this operation to complete
body: <p><strong>Background</strong></p>
  <p>I have a very small network which I want to test with different random seeds.
  The network barely uses 1% of my GPUs compute power so i could in theory run 50 processes at once to try many different seeds at once.</p>
  <p><strong>Problem</strong></p>
  <p>Unfortunately i can't even import pytorch in multiple processes. When the <strong>nr of processes</strong> exceeds <strong>4</strong> I get a Traceback regarding a too small paging file.</p>
  <p><strong>Minimal reproducable code§ - dispatcher.py</strong></p>
  <pre><code>from subprocess import Popen
  import sys
  
  procs = []
  for seed in range(50):
      procs.append(Popen([sys.executable, &quot;ml_model.py&quot;, str(seed)]))
  
  for proc in procs:
      proc.wait()
  </code></pre>
  <p>§I increased the number of seeds so people with better machines can also reproduce this.</p>
  <p><strong>Minimal reproducable code - ml_model.py</strong></p>
  <pre><code>import torch
  import time
  time.sleep(10)
  </code></pre>
  <pre><code> 
   Traceback (most recent call last):
    File &quot;ml_model.py&quot;, line 1, in &lt;module&gt;
      import torch
    File &quot;C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\__init__.py&quot;, line 117, in &lt;module&gt;
      import torch
    File &quot;C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\__init__.py&quot;, line 117, in &lt;module&gt;
      raise err
   OSError: [WinError 1455] The paging file is too small for this operation to complete. Error loading &quot;C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\lib\cudnn_cnn_infer64_8.dll&quot; or one of its dependencies.
      raise err
  
  </code></pre>
  <p><strong>Further Investigation</strong></p>
  <p>I noticed that each process loads a lot of dll's into RAM. And when i close all other programs which use a lot of RAM i can get up to 10 procesess instead of 4. So it seems like a resource constraint.</p>
  <p><strong>Questions</strong></p>
  <p>Is there a workaround ?</p>
  <p>What's the recommended way to train many small networks with pytorch on a single gpu ?</p>
  <p>Should i write my own CUDA Kernel instead, or use a different framework to achieve this ?</p>
  <p>My goal would be to run around 50 processes at once (on a 16GB RAM Machine, 8GB GPU RAM)</p>