title: Pytorch - Concatenating Datasets before using Dataloader
body: <p>I am trying to load two datasets and use them both for training. </p>
  
  <p>Package versions: python 3.7; 
  pytorch 1.3.1</p>
  
  <p>It is possible to create data_loaders seperately and train on them sequentially: </p>
  
  <pre><code>from torch.utils.data import DataLoader, ConcatDataset
  
  
  train_loader_modelnet = DataLoader(ModelNet(args.modelnet_root, categories=args.modelnet_categories,split='train', transform=transform_modelnet, device=args.device),batch_size=args.batch_size, shuffle=True)
  
  train_loader_mydata = DataLoader(MyDataset(args.customdata_root, categories=args.mydata_categories, split='train', device=args.device),batch_size=args.batch_size, shuffle=True)
  
  for e in range(args.epochs):
      for idx, batch in enumerate(tqdm(train_loader_modelnet)):
          # training on dataset1
      for idx, batch in enumerate(tqdm(train_loader_custom)):
          # training on dataset2
  
  </code></pre>
  
  <p>Note: MyDataset is a custom dataset class which has <code>def __len__(self):</code> <code>def __getitem__(self, index):</code> implemented. As the above configuration works it seems that this is implementation is OK. </p>
  
  <p>But I would ideally like to combine them into a single dataloader object. I attempted this as per the pytorch documentation: </p>
  
  <pre><code>train_modelnet = ModelNet(args.modelnet_root, categories=args.modelnet_categories,
                            split='train', transform=transform_modelnet, device=args.device)
  train_mydata = CloudDataset(args.customdata_root, categories=args.mydata_categories,
                               split='train', device=args.device)
  train_loader = torch.utils.data.ConcatDataset(train_modelnet, train_customdata)
  
  for e in range(args.epochs):
      for idx, batch in enumerate(tqdm(train_loader)):
          # training on combined
  
  </code></pre>
  
  <p>However, on random batches I get the following 'expected a tensor as element X in argument 0, but got a tuple instead' type of error. Any help would be much appreciated!</p>
  
  <pre><code>&gt;   40%|████      | 53/131 [01:03&lt;02:00,  1.55s/it]
  &gt;  Traceback (mostrecent call last):   File
  &gt; "/home/chris/Programs/pycharm-anaconda-2019.3.4/plugins/python/helpers/pydev/pydevd.py",
  &gt; line 1434, in _exec
  &gt;     pydev_imports.execfile(file, globals, locals)  # execute the script   File
  &gt; "/home/chris/Programs/pycharm-anaconda-2019.3.4/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
  &gt;     exec(compile(contents+"\n", file, 'exec'), glob, loc)   File "/home/chris/Documents/4yp/Data/my_kaolin/Classification/pointcloud_classification_combinedset.py",
  &gt; line 83, in &lt;module&gt;
  &gt;     for idx, batch in enumerate(tqdm(train_loader)):   File "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/tqdm/std.py",
  &gt; line 1107, in __iter__
  &gt;     for obj in iterable:   File "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/torch/utils/data/dataloader.py",
  &gt; line 346, in __next__
  &gt;     data = self._dataset_fetcher.fetch(index)  # may raise StopIteration   File
  &gt; "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py",
  &gt; line 47, in fetch
  &gt;     return self.collate_fn(data)   File "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py",
  &gt; line 79, in default_collate
  &gt;     return [default_collate(samples) for samples in transposed]   File "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py",
  &gt; line 79, in &lt;listcomp&gt;
  &gt;     return [default_collate(samples) for samples in transposed]   File "/home/chris/anaconda3/envs/4YP/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py",
  &gt; line 55, in default_collate
  &gt;     return torch.stack(batch, 0, out=out) TypeError: expected Tensor as element 3 in argument 0, but got tuple  
  </code></pre>
  
  <pre><code>
  </code></pre>