title: Dealing with datasets with repeated multivalued features
body: <p>We have a <a href="http://www.cs.cornell.edu/~adith/Criteo/sample.txt" rel="noreferrer">Dataset</a> that is in sparse representation and has 25 features and 1 binary label. For example, a line of dataset is:</p>
  
  <pre><code>Label: 0
  exid: 24924687
  Features:
  11:0 12:1 13:0 14:6 15:0 17:2 17:2 17:2 17:2 17:2 17:2
  21:11 21:42 21:42 21:42 21:42 21:42 
  22:35 22:76 22:27 22:28 22:25 22:15 24:1888
  25:9 33:322 33:452 33:452 33:452 33:452 33:452 35:14
  </code></pre>
  
  <p>So, sometimes features have multiple values and they can be the same or different, and the website says:</p>
  
  <blockquote>
    <p>Some categorical features are multi-valued (order does not matter)</p>
  </blockquote>
  
  <p>We don't know what is the semantic of features and the value that have been assigned to them (because of some privacy concern they are hidden to public)</p>
  
  <p>We only know:</p>
  
  <ul>
  <li><code>Label</code> means if the user  has clicked on  the recommended ad or not.</li>
  <li><code>Features</code> are describing the product that has been recommended to user.</li>
  <li><code>Task</code> is to predict the probability of getting a click by the user, given an ad of a product.</li>
  </ul>
  
  <p>Any comment on the following problems are appreciated:</p>
  
  <ol>
  <li>What's the best way to import this kind of datasets into a Python data structure.</li>
  <li>How to deal with multi-valued features, specially when they have similar values repeated <code>k</code> times?</li>
  </ol>