title: In PyTables, how to create nested array of variable length?
body: <p>I'm using PyTables 2.2.1 w/ Python 2.6, and I would like to create a table which contains nested arrays of variable length.</p>
  
  <p>I have searched the PyTables documentation, and the tutorial example (<a href="http://www.pytables.org/docs/manual/ch03.html#id332794">PyTables Tutorial 3.8</a>) shows how to create a nested array of length = 1. But for this example, how would I add a variable number of rows to data 'info2/info3/x' and 'info2/info3/y'?</p>
  
  <p>For perhaps an easier to understand table structure, here's my homegrown example:</p>
  
  <pre><code>"""Desired Pytable output:
  
  DIEM    TEMPUS  Temperature             Data
  5       0       100         Category1 &lt;--||--&gt;  Category2
                           x &lt;--| |--&gt; y          z &lt;--|
                          0           0           0
                          2           1           1
                          4           1.33        2.67
                          6           1.5         4.5
                          8           1.6         6.4
  5       1       99
                          2           2           0   
                          4           2           2
                          6           2           4
                          8           2           6
  5       2       96
                          4           4           0
                          6           3           3
                          8           2.67        5.33
  
  
  Note that nested arrays have variable length.
  """
  
  import tables as ts
  
  tableDef =      {'DIEM': ts.Int32Col(pos=0),
                  'TEMPUS': ts.Int32Col(pos=1), 
                  'Temperature' : ts.Float32Col(pos=2),
                  'Data': 
                      {'Category1': 
                          {
                          'x': ts.Float32Col(), 
                          'y': ts.Float32Col()
                          }, 
                      'Category2': 
                          {
                          'z': ts.Float32Col(), 
                          }
                      }
                  }
  
  # create output file
  fpath = 'TestDb.h5'
  fh = ts.openFile(fpath, 'w')
  # define my table
  tableName = 'MyData'
  fh.createTable('/', tableName, tableDef)
  tablePath = '/'+tableName
  table = fh.getNode(tablePath)
  
  # get row iterator
  row = table.row
  for i in xrange(3):
      print '\ni=', i
      # calc some fake data
      row['DIEM'] = 5
      row['TEMPUS'] = i
      row['Temperature'] = 100-i**2
  
      for j in xrange(5-i):
          # Note that nested array has variable number of rows
          print 'j=', j,
          # calc some fake nested data
          val1 = 2.0*(i+j)
          val2 = val1/(j+1.0)
          val3 = val1 - val2
  
          ''' Magic happens here...
          How do I write 'j' rows of data to the elements of 
          Category1 and/or Category2?
  
          In bastardized pseudo-code, I want to do:
  
          row['Data/Category1/x'][j] = val1
          row['Data/Category1/y'][j] = val2
          row['Data/Category2/z'][j] = val3
          '''
  
      row.append()
  table.flush()
  
  fh.close()
  </code></pre>
  
  <p>I have not found any indication in the PyTables docs that such a structure is not possible... but in case such a structure is in fact not possible, what are my alternatives to variable length nested columns?</p>
  
  <ul>
  <li>EArray? VLArray? If so, how to integrate these data types into the above described structure?</li>
  <li>some other idea?</li>
  </ul>
  
  <p>Any assistance is greatly appreciated!</p>
  
  <p>EDIT w/ additional info:
  It appears that the PyTables gurus have already addressed the "is such a structure possible" question:</p>
  
  <p><a href="http://sourceforge.net/mailarchive/message.php?msg_id=18598716">PyTables Mail Forum - Hierachical Datasets</a></p>
  
  <p>So has anyone figured out a way to create an analogous PyTable data structure?</p>
  
  <p>Thanks again!</p>