title: Using jq to extract common prefixes in a JSON data structure
body: <p>I have a JSON data set with around 8.7 million key value pairs extracted from a Redis store, where each key is guaranteed to be an 8 digit number, and the key is an 8 alphanumeric character value i.e.</p>
  
  <pre><code>[{
  "91201544":"INXX0019",
  "90429396":"THXX0020",
  "20140367":"ITXX0043",
   ...
  }]
  </code></pre>
  
  <p>To reduce Redis memory usage, I want to transform this into a hash of hashes, where the hash prefix key is the first 6 characters of the key (see <a href="http://redis.io/topics/memory-optimization" rel="nofollow">this link</a>) and then store this back into Redis.</p>
  
  <p>Specifically, I want my resulting JSON data structure (that I'll then write some code to parse this JSON structure and create a Redis command file consisting of HSET, etc) to look more like</p>
  
  <pre><code>[{
   "000000": { "00000023": "INCD1234",
               "00000027": "INCF1423",
                ....
             },
   ....
   "904293": { "90429300": "THXX0020",
               "90429302": "THXX0024",
               "90429305": "THXY0013"}
   }]
  </code></pre>
  
  <p>Since I've been impressed by <a href="http://stedolan.github.io/jq/manual/#Invokingjq" rel="nofollow">jq</a> and I'm trying to be more proficient at functional style programming, I wanted to use <em>jq</em> for this task.  So far I've come up with the following:</p>
  
  <pre><code>% jq '.[0] | to_entries | map({key: .key, pfx: .key[0:6], value: .value}) | group_by(.pfx)'
  </code></pre>
  
  <p>This gives me something like</p>
  
  <pre><code>[
    [
      {
        "key": "00000130",
        "pfx": "000001",
        "value": "CAXX3231"
      },
      {
        "key": "00000162",
        "pfx": "000001",
        "value": "CAXX4606"
      }
    ],
    [
      {
        "key": "00000238",
        "pfx": "000002",
        "value": "CAXX1967"
      },
      {
        "key": "00000256",
        "pfx": "000002",
        "value": "CAXX0727"
      }
    ],
    ....
  ]
  </code></pre>
  
  <p>I've tried the following:</p>
  
  <pre><code>% jq 'map(map({key: .pfx, value: {key, value}})) 
        | map(reduce .[] as $item ({}; {key: $item.key, value: [.value[], $item.value]} )) 
        | map( {key, value: .value | from_entries} ) 
        | from_entries'
  </code></pre>
  
  <p>which does give me the correct result, but also prints out an error for every reduce (I believe) of </p>
  
  <pre><code>jq: error: Cannot iterate over null
  </code></pre>
  
  <p>The end result is </p>
  
  <pre><code>{
     "000001": {
       "00000130": "CAXX3231",
       "00000162": "CAXX4606"
     },
     "000002": {
       "00000238": "CAXX1967",
       "00000256": "CAXX0727"
     },
     ...
  }
  </code></pre>
  
  <p>which is correct, but how can I avoid getting this stderr warning thrown as well?</p>