title: Saving many values in a MongoDB Collection & node
body: <p>I have an API which I call three times with three different parameters.</p>
  
  <p><code>https://api.developer.com/${param1} // param2, and param3</code></p>
  
  <p>This api returns 30,000+ results each time. (total of around 100,000).</p>
  
  <p>I want to store this data in a single <code>collection</code>. Meaning, I want 100,000+ <code>documents</code> in one collection.</p>
  
  <p>I have a small script that extends the npm <code>request</code> module, which looks like this: </p>
  
  <pre><code>```
  let _request = (urls, cb) =&gt; {
      let results = {}, i = urls.length, c = 0;
      handler = (err, response, body) =&gt; {
          let url = response.request.uri.href;
          results[url] = { err, response, body };
  
          if (++c === urls.length) {
              cb(results);
          }
      };
      while (i--) {
          request(urls[i], handler);
      }
  };
  
  ```
  </code></pre>
  
  <p>But let's exclude that for now. The function which I use to GET a single endpoint, and update the Database after is this:</p>
  
  <pre><code>function update() {
    request(url, (err, response, body) =&gt; {
      if (err) {
        console.log(err);
      } else {
        let json = {};
  
        try {
          json = JSON.parse(body);
        } catch (e) {
          console.log(e);
        }
  
        _.forOwn(json, (price, market_hash_name) =&gt; {
          Price.update(
            { market_hash_name },
            {
              $set: { price }
            },
            { upsert: true },
            err =&gt; {
              if (err) {
                console.log(err);
              }
            }
          );
        });
      }
    });
  }
  </code></pre>
  
  <p>The raw data returned looks like this:</p>
  
  <pre><code>{
    market_hash_name: price,
    market_hash_name: price.. etc
  }
  </code></pre>
  
  <p><strong>The problem:</strong></p>
  
  <p>Even though, <code>Object.keys(data).length === 30000</code>, my MongoDB collection only writes down ~10,000 documents, and the other ~20000 vanish into thin air.
  I've checked a thousand times, with Postman, Browser and even logging the data.keys in the console, and I am sure there are 30k + key:value pairs.</p>
  
  <p>Is it something wrong with my code? Is it bad practice to call Price.update for every key:value pair in the json (probably). But I'm stuck. Any help would be much appreciated.</p>