title: Mongo aggregate and $group nested array with $lookup
body: <p>I have the following Product Schema: (partial, using mongoose)</p>
  
  <pre><code> attributes: [
        {
          set: {
            ref: 'AttributeSet',
            type: Schema.Types.ObjectId
          },
          items: [
            {
              attribute: {
                ref: 'Attributes',
                type: Schema.Types.ObjectId
              },
              values: [
                {
                  ref: 'AttributeValues',
                  type: Schema.Types.ObjectId
                }
              ]
            }
          ],
          _id: 0
        }
      ],
  
  </code></pre>
  
  <p><strong>example document 1: (partial)</strong></p>
  
  <pre><code>"attributes" : [
          {
              "set" : ObjectId("5ccc079c846de44116182890"),
              "items" : [
                  {
                      "values" : [
                          ObjectId("5ccc0900846de441161828a1")
                      ],
                      "_id" : ObjectId("5dee638d72fa520f53d0d1c4"),
                      "attribute" : ObjectId("5ccc0900846de441161828a0")
                  },
                  {
                      "values" : [
                          ObjectId("5ccc0a51846de441161828cc")
                      ],
                      "_id" : ObjectId("5dee638d72fa520f53d0d1c3"),
                      "attribute" : ObjectId("5ccc0a51846de441161828cb")
                  },
                  {
                      "values" : [
                          ObjectId("5ccc0c7d846de44116182906")
                      ],
                      "_id" : ObjectId("5dee638d72fa520f53d0d1c2"),
                      "attribute" : ObjectId("5ccc0c7d846de44116182904")
                  },
                  {
                      "values" : [
                          ObjectId("5ccc0d64846de44116182911")
                      ],
                      "_id" : ObjectId("5dee638d72fa520f53d0d1c1"),
                      "attribute" : ObjectId("5ccc0d64846de4411618290f")
                  },
                  {
                      "values" : [
                          ObjectId("5ccc079f846de44116182892")
                      ],
                      "_id" : ObjectId("5def6acf66910405e07e1e9f"),
                      "attribute" : ObjectId("5ccc079f846de44116182891")
                  }
              ]
          }
      ]
  </code></pre>
  
  <p><strong>example document 2: (partial)</strong></p>
  
  <pre><code>"attributes" : [
          {
              "set" : ObjectId("5ccc079c846de44116182890"), 
              "items" : [
                  {
                      "values" : [
                          ObjectId("5ccc079f846de44116182892")
                      ], 
                      "_id" : ObjectId("5dee635c72fa520f53d0d1c0"), 
                      "attribute" : ObjectId("5ccc079f846de44116182891")
                  }, 
                  {
                      "values" : [
                          ObjectId("5ccc0900846de441161828a2")
                      ], 
                      "_id" : ObjectId("5dee635c72fa520f53d0d1bf"), 
                      "attribute" : ObjectId("5ccc0900846de441161828a0")
                  }, 
                  {
                      "values" : [
                          ObjectId("5ccc0ea4846de44116182941")
                      ], 
                      "_id" : ObjectId("5dee635c72fa520f53d0d1be"), 
                      "attribute" : ObjectId("5ccc0ea4846de44116182940")
                  }, 
                  {
                      "values" : [
                          ObjectId("5ccc08ba846de4411618289c")
                      ], 
                      "_id" : ObjectId("5def56c537e877042d5abeb5"), 
                      "attribute" : ObjectId("5ccc08ba846de4411618289a")
                  }, 
                  {
                      "values" : [
                          ObjectId("5ccc09ca846de441161828aa"), 
                          ObjectId("5ccc09ca846de441161828a9")
                      ], 
                      "_id" : ObjectId("5def56c537e877042d5abeb4"), 
                      "attribute" : ObjectId("5ccc09ca846de441161828a7")
                  }
              ]
          }
      ], 
  
  </code></pre>
  
  <p>I want to aggregate and find all products that have attributes and then group the attributes in the output.</p>
  
  <p><strong>Pipeline:</strong> </p>
  
  <pre><code>db.getCollection("products").aggregate(
    [
      { $unwind: "$attributes" },
      {
        $group: {
          _id: "$attributes",
          attributes: { $first: "$attributes.items" }
        }
      },
      { $unwind: "$attributes" },
      {
        $lookup: {
          from: "attributes",
          let: { attribute: "$attributes.attribute" },
          pipeline: [
            {
              $match: {
                $expr: {
                  $eq: ["$_id", "$$attribute"]
                }
              }
            },
            { $project: { display_name: 1, _id: 1 } }
          ],
          as: "attrs"
        }
      },
      {
        $lookup: {
          from: "attributevalues",
          let: { attribute: "$attributes.values" },
          pipeline: [
            {
              $match: {
                $expr: {
                  $in: ["$_id", "$$attribute"]
                }
              }
            }
          ],
          as: "values"
        }
      },
      { $project: { attrs: 1, values: 1, _id: 0 } },
      {
        $group: { _id: "$attrs", items: { $push: "$values" }, total: { $sum: 1 } }
      }
    ],
    {
      allowDiskUse: true
    }
  );
  
  </code></pre>
  
  <p><strong>Pipeline output:</strong></p>
  
  <pre><code>[{ 
      "_id" : [
          {
              "_id" : ObjectId("5ccc079f846de44116182891"), 
              "display_name" : "Caliber (cal.)"
          }
      ], 
      "items" : [
          [
              {
                  "_id" : ObjectId("5ccc079f846de44116182892"), 
                  "sort_order" : NumberInt(0),
                  "label" : "12", 
                  "attribute_id" : ObjectId("5ccc079f846de44116182891")
              }
          ], 
          [
              {
                  "_id" : ObjectId("5ccc079f846de44116182892"), 
                  "sort_order" : NumberInt(0),
                  "label" : "12", 
                  "attribute_id" : ObjectId("5ccc079f846de44116182891")
              }
          ]
      ], 
      "total" : 2.0
  },
  {
      "_id" : [
          {
              "_id" : ObjectId("5ccc0900846de441161828a0"),
              "display_name" : "Mechanism"
          }
      ],
      "items" : [
          [
              {
                  "_id" : ObjectId("5ccc0900846de441161828a2"),
                  "sort_order" : NumberInt(1),
                  "label" : "Inaction",
                  "attribute_id" : ObjectId("5ccc0900846de441161828a0")
              }
          ],
          [
              {
                  "_id" : ObjectId("5ccc0900846de441161828a1"),
                  "sort_order" : NumberInt(0),
                  "label" : "Gas",
                  "attribute_id" : ObjectId("5ccc0900846de441161828a0")
              }
          ]
      ],
      "total" : 2.0
  }]
  
  </code></pre>
  
  <p>Problem is that e.x. in 1st element in the array, I have a duplicate inside the <code>items</code> array.</p>
  
  <p><strong>This is the desired output:</strong> </p>
  
  <pre><code>[{
      "_id" : [
          {
              "_id" : ObjectId("5ccc079f846de44116182891"),
              "display_name" : "Caliber (cal.)"
          }
      ],
      "items" : [
          [
              {
                  "_id" : ObjectId("5ccc079f846de44116182892"),
                  "sort_order" : NumberInt(0),
                  "label" : "12",
                  "attribute_id" : ObjectId("5ccc079f846de44116182891"),
                  "total": 'current _id total, in this case it should be 2'
              }
          ],
          ...other items goes below, grouped as above
      ]
  }]
  
  </code></pre>