title: How to convert pandas DataFrame to a specific json format
body: <p>I have a pandas dataframe in this format : </p>
  
  <pre><code>DataProvider,Id,Name,Time
  
  ABC,f8453e99-516f-4f15-a3bd-8749089b6934,"xyz",43200
  
  ABC,f8453e99-516f-4f15-a3bd-8749089b6934,"123",43200
  
  ABC,00453e99-516f-4f15-a3bd-8749089b6934,"xyz",43200
  </code></pre>
  
  <p>I want to generate the output in this format (json) :</p>
  
  <pre><code>  {
    "dataProviderId":"ABC",
    "items": [
      {
      "Id":"f8453e99-516f-4f15-a3bd-8749089b6934",
      "data": [
        {"Name":"xyz","Time":43200},
        {"Name":"xyz","Time":43200}
      ],
      },
      {
      "Id":"00453e99-516f-4f15-a3bd-8749089b6934",
      "data": [
          {"Name":"xyz","Time":43200}
      ]
      }
  </code></pre>
  
  <hr>
  
  <p>This is what I have tried so far. I am trying to group by "Name" and "Time" fields first to a dictionary and then call it "data". After that I will do one more group by and aggregate "data" and "Id" and call it "items".
  After that, I think, I can download the results to the required json format using df.to_json()</p>
  
  <p>This is the code so far : </p>
  
  <pre><code>import pandas as pd
  import os
  
  df = pd.read_csv('bq-results-20200320-121645-a47v3t1cq9op.csv',encoding='UTF-8')
  
  output = (df.groupby(['dataProviderId', 'Id'], as_index=False)
       .apply(lambda x: dict(zip(['Name','Time''],x[['Name','Time']]))).reset_index(name='data'))
  </code></pre>
  
  <p>However, this is the output that I get : </p>
  
  <pre><code>    dataProviderId  Id  data
  0   lifesight   0016b4a4-c807-497f-b86d-eec8bd53f11f    {'Name': 'Name', 'Time': 'Time'}
  1   lifesight   0057eb21-93f7-4818-af15-871a5f131006    {'Name': 'Name', 'Time': 'Time'}
  2   lifesight   00b86381-c697-496b-b45f-334fc1247914    {'Name': 'Name', 'Time': 'Time'}
  3   lifesight   0163b87e-f59b-4fc5-8c2e-78cb17b47404    {'Name': 'Name', 'Time': 'Time'}
  4   lifesight   01c5d810-18b9-425a-bc2a-03bac44f675f    {'Name': 'Name', 'Time': 'Time'
  </code></pre>