title: MongoDB aggregation $group is very slow in pipeline but works decently grouping same collection from hard disk
body: <p>I have an aggregation query</p>
  
  <pre><code>db.banners.aggregate(
  [ { '$match':
       { '$and':
          [ { ts: { '$gte': 1448920800000 } },
            { ts: { '$lte': 1449954261885 } },
            { '$or':
               [ { campaignId: { '$exists': true } },
                 { clicks: { '$ne': [] } } ] } ] } },
    { '$sort': { ts: -1 } },
    { '$project':
       { siteId: 1,
         adslotId: 1,
         campaignId: { '$ifNull': [ '$campaignId', null ] },
         adslotElemId: 1,
         ts: 1,
         price: 1,
         adslotName: 1,
         impId: 1,
         siteDomain: 1,
         country: 1,
         browser: 1,
         browserVer: 1,
         os: 1,
         osVer: 1,
         ip: 1,
         kifUserUID: 1,
         wndLocHref: 1,
         cb: 1,
         ps: 1,
         clicks: 1,
         _id: 0,
         clickRev:
          { '$sum':
             { '$cond':
                [ '$campaignId',
                  { '$map': { input: '$clicks', as: 'aux', in: '$$aux.price' } },
                  0 ] } },
         clickCnt:
          { '$cond':
                             { '$cond':
             [ '$campaignId',
               { '$sum': { '$size': { '$ifNull': [ '$clicks', [] ] } } },
               0 ] } } },
    { '$project':
       { siteId: 1,
         adslotId: 1,
         campaignId: 1,
         adslotElemId: 1,
         ts: 1,
         price: 1,
         adslotName: 1,
         impId: 1,
         siteDomain: 1,
         country: 1,
         browser: 1,
         browserVer: 1,
         os: 1,
         osVer: 1,
         ip: 1,
         kifUserUID: 1,
         wndLocHref: 1,
         cb: 1,
         ps: 1,
         clicks: { '$cond': [ '$campaignId', null, '$clicks' ] },
         clickCnt: 1,
         clickRev: 1 } },
    { '$unwind': { path: '$clicks', preserveNullAndEmptyArrays: true } },
    { '$project':
       { siteId: 1,
         adslotId: 1,
         campaignId: { '$ifNull': [ '$campaignId', '$clicks.campaignId' ] },
         adslotElemId: 1,
         ts: 1,
         price: { '$ifNull': [ '$price', '$clicks.price' ] },
         adslotName: 1,
         impId: { '$cond': [ '$campaignId', '$impId', '$clicks.impId' ] },
         siteDomain: 1,
         country: 1,
         browser: 1,
         browserVer: 1,
         os: 1,
         osVer: 1,
         ip: 1,
         wndLocHref: 1,
         cb: 1,
         ps: 1,
         clicks: 1,
         tsRange:
          { '$concat':
             [ { '$cond':
                  [ { '$and':
                       [ { '$gt': [ '$ts', 1448920800000 ] },
                         { '$lte': [ '$ts', 1451512800000 ] } ] },
                    '1448920800000-',
                    '' ] } ] },
         clickCnt: 1,
         clickRev: 1 } },
    { '$out': 'result' }])
  </code></pre>
  
  <p>that outputs 3910424 documents into result collection. 
  If I do for example</p>
  
  <pre><code>db.result.aggregate([{ $group: { _id:{country:'$country'}, c:{$sum: '$price'} }}]);
  </code></pre>
  
  <p>the answer is returned in 7 seconds.</p>
  
  <p>However, when I replace <code>$out</code> stage with <code>$group: { _id:{country:'$country'}, c:{$sum: '$price'}</code> stage, the answer returns after ~3 mins.</p>
  
  <p>My DB stats:</p>
  
  <pre><code>&gt; db.stats()
  {
      "db" : "logs",
      "collections" : 9,
      "objects" : 18904961,
      "avgObjSize" : 713.6041404158411,
      "dataSize" : 13490658444,
      "storageSize" : 17286610928,
      "numExtents" : 83,
      "indexes" : 13,
      "indexSize" : 1810174576,
      "fileSize" : 34276900864,
      "nsSizeMB" : 16,
      "extentFreeList" : {
          "num" : 68,
          "totalSize" : 11930017776
      },
      "dataFileVersion" : {
          "major" : 4,
          "minor" : 22
      },
      "ok" : 1
  }
  </code></pre>
  
  <p>Hardware: 
  32 GB,  Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz</p>
  
  <p>I would use <code>$out</code> as a solution to this slowness in grouping but the $out  itself is slow so the final time spent is about the same.</p>
  
  <p>What's the explanation to the above?</p>