title: encoding 100,000s variables in each time step of RNN tensorflow keras
body: <p>I’m looking for some pointers on efficiency. I have potentially 100,000s of binary variables that i wish to encode in each time step of an RNN for binary classification of the entire sequence, but I am torn between trying to encode every feature in each step (will be very, very sparse), or just use for a better word a “dodge” effect where if 3 out of the 100,000’s of variables = 1 (the vast majority at each time step will be 0) then these 3 occurrences just take up a separate time step each. For example if i have:</p>
  <pre><code>ID   Event_type (100,000’s of different types)
  —    —————————-
  1.   A
  1.   BX
  1.   ZZA
  2.   YZAE
  3.   C
  3.   FXN
  </code></pre>
  <p>where for each ID, these events all appear in time = 1, I either</p>
  <ol>
  <li>Create a very sparse representation at each time step that includes many 0’s to indicate all the events <em>not</em> experienced at this time step</li>
  <li>Or do something like this:</li>
  </ol>
  <blockquote>
  <pre><code>[
   [[1,[“A”],
       [2,[“YZA”],
       [3,[“C”]] ,
  
   [[1,[“BX”],
       [2,[0],
       [3,[“FXN”]],
  
   [[1,[“ZZA”],
       [2,[0],
       [3,[0]]
  ]
  </code></pre>
  </blockquote>
  <p>i.e. pad out the sequences and effectively turn this 1 time step into 3 time steps (ID A has the largest number, 3 events in this particular time step).</p>
  <p>Has anyone had to deal with many features per time step before, where they are always sparsely populated?</p>