title: Encoding problems while making requests by batches?
body: <p>I am making requests to an api:</p>
  
  <pre><code>def get_data(text, url='api.com'):
      r = requests.get(url,
                       params={'key': '&lt;My KEY&gt;',
                               'hj': text
                               'oi': 'm'})
  
      json_data = json.dumps(r.json())
      data = yaml.load(json_data)                
      return data
  </code></pre>
  
  <p>Then I am applying the function as follows, since my data is in a pandas dataframe:</p>
  
  <pre><code>    data
  0   The quick  fox jumps over the lazy 
  1   The quick  fox  over the lazy dog
  2   The quick brown fox jumps over the lazy dog
  ....
  
  n   The  brown fox jumps over the  dog
  </code></pre>
  
  <p>Then:</p>
  
  <pre><code>df['col'] = df[['data']].apply(get_data, axis=1)
  </code></pre>
  
  <p>The size of the data I am sending and receiving with requests is quite large, thus how can I make the above request by chunks?, let's say 4 by 4?:</p>
  
  <pre><code>for chunk in r.iter_content(chunk_size=5):
         json_data = json.dumps(r.json())
         data = yaml.load(json_data)
         return data
  </code></pre>
  
  <p>However it aint worked, could some one help me to make the request by chunks or split in chunks and concat everything ?.</p>
  
  <p><strong>UPDATE</strong></p>
  
  <p>I also tried to split by chunks the dataframe, and it just doesnt finish:</p>
  
  <p>In:</p>
  
  <pre><code>df.groupby(np.arange(len(df))//10)
  for k,g in df.groupby(np.arange(len(df))//10):
      [g.data.apply(get_data) for _, g in df.groupby(np.arange(len(df))//10)]
  </code></pre>
  
  <p>Out:</p>
  
  <pre><code>----&gt; 7     df = pd.concat(g.data.apply(get_data) for _, g in df2.groupby(np.arange(len(df2))//4))
        8 df
  
  /usr/local/lib/python3.5/site-packages/pandas/core/series.py in apply(self, func, convert_dtype, args, **kwds)
     2290             else:
     2291                 values = self.asobject
  -&gt; 2292                 mapped = lib.map_infer(values, f, convert=convert_dtype)
     2293 
     2294         if len(mapped) and isinstance(mapped[0], Series):
  
  pandas/src/inference.pyx in pandas.lib.map_infer (pandas/lib.c:63307)()
  
  &lt;ipython-input-28-329dbdbb7cdb&gt; in get_data(data)
       62 
       63     r = requests.get('http://api.example.com/api', params=payload, stream = True)
  ---&gt; 64     json_data = json.dumps(r.json())
       65     data = yaml.load(json_data)
       66   
  
  /usr/local/lib/python3.5/site-packages/requests/models.py in json(self, **kwargs)
      848                     # used.
      849                     pass
  --&gt; 850         return complexjson.loads(self.text, **kwargs)
      851 
      852     @property
  
  /usr/local/lib/python3.5/site-packages/simplejson/__init__.py in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
      514             parse_constant is None and object_pairs_hook is None
      515             and not use_decimal and not kw):
  --&gt; 516         return _default_decoder.decode(s)
      517     if cls is None:
      518         cls = JSONDecoder
  
  /usr/local/lib/python3.5/site-packages/simplejson/decoder.py in decode(self, s, _w, _PY3)
      368         if _PY3 and isinstance(s, binary_type):
      369             s = s.decode(self.encoding)
  --&gt; 370         obj, end = self.raw_decode(s)
      371         end = _w(s, end).end()
      372         if end != len(s):
  
  /usr/local/lib/python3.5/site-packages/simplejson/decoder.py in raw_decode(self, s, idx, _w, _PY3)
      398             elif ord0 == 0xef and s[idx:idx + 3] == '\xef\xbb\xbf':
      399                 idx += 3
  --&gt; 400         return self.scan_once(s, idx=_w(s, idx).end())
  
  JSONDecodeError: Expecting value: line 1 column 1 (char 0)
  </code></pre>
  
  <p>However I do not understand to chain everything together after splitting in chunks.</p>