title: Why bigquery can't handle a query processing 4TB data?
body: <p>I'm trying to run this query</p>
  <pre><code>SELECT
    id AS id,
    ARRAY_AGG(DISTINCT users_ids) AS users_ids,
    MAX(date) AS date
  FROM
    users,
    UNNEST(users_ids) AS users_ids
  WHERE
    users_ids != &quot; 1111&quot;
    AND users_ids != &quot; 2222&quot;
  GROUP BY
    id;
  </code></pre>
  <p>Where users table is splitted table with id column and user_ids (comma separated) column and date column
  on a +4TB and it give me resources</p>
  <blockquote>
  <p>Resources exceeded during query execution: Your project or organization exceeded the maximum disk and memory limit available for shuffle operations.</p>
  </blockquote>
  <p>.. any idea why?</p>
  <pre><code>id    userids   date
  1     2,3,4     1-10-20
  2     4,5,6     1-10-20   
  1     7,8,4     2-10-20
  
  so the final result I'm trying to reach
  
  id    userids       date
  1     2,3,4,7,8     2-10-20
  2     4,5,6         1-10-20 
  </code></pre>
  <p>Execution details:
  <a href="https://i.stack.imgur.com/0QfAj.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/0QfAj.png" alt="enter image description here" /></a>
  <a href="https://i.stack.imgur.com/PVkRs.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/PVkRs.png" alt="enter image description here" /></a></p>