title: Django large queryset return as response efficiently
body: <p>I have a model in django called &quot;Sample&quot;
  I want to query and return a large number of rows ~ 100k based on filters.
  However, it's taking up to 4-5 seconds to return the response and I was wondering whether I could make it faster.</p>
  <p>(Need to improve converting from queryset to df to response json. Not querying from DB)</p>
  <p>My current code looks like this:</p>
  <pre><code>@api_view(['POST'])
  def retrieve_signal_asset_weight_ts_by_signal(request):
      
      #code to get item.id here based on request
  
      qs = Sample.objects.filter(
          data_date__range=[start_date, end_date],
          item__id = item.id).values(*columns_required)
  
      df = pd.DataFrame(list(qs), columns=columns_required)    
      response = df .to_json(orient='records')
  
      return Response(response, status=status.HTTP_200_OK)
  </code></pre>
  <p>Based on multiple test cases -- I've noticed that the slow part isn't actually getting the data from DB, it's converting it to a DataFrame and then returning as JSON.
  It's actually taking about 2 seconds just for this part <code>df = pd.DataFrame(list(qs), columns=columns_required)</code>. Im looking for a faster way to convert queryset to a json which I can send as part of my &quot;response&quot; object!</p>
  <p>Based on this <a href="https://stackoverflow.com/questions/11697887/converting-django-queryset-to-pandas-dataframe">link</a> I've tried other methods including <code>django-pandas</code> and using <code>.values_list()</code> but they seem to be slower than this, and I noticed many of the answers are quite old so I was wondering whether Django 3 has anything to make it faster.</p>
  <p>Thanks</p>
  <p>Django version : 3.2.6</p>