title: How does the "number of workers" parameter in PyTorch dataloader actually work?
body: <ol>
  <li>If <code>num_workers</code> is 2, Does that mean that it will put 2 batches in the RAM and send 1 of them to the GPU or Does it put 3 batches in the RAM then sends 1 of them to the GPU?</li>
  <li>What does actually happen when the number of workers is higher than the number of CPU cores? I tried it and it worked fine but How does it work? (I thought that the maximum number of workers I can choose is the number of cores).</li>
  <li>If I set <code>num_workers</code> to 3 and during the training there were no batches in the memory for the GPU, Does the main process waits for its workers to read the batches or Does it read a single batch (without waiting for the workers)?</li>
  </ol>