title: 'Compressor detection can only be called on some xcontent bytes or compressed xcontent bytes" error when indexing a list of dictionaries
body: <p>This question is related to this other one:
  <a href="https://stackoverflow.com/questions/62459572/how-can-i-read-data-from-a-list-and-index-specific-values-into-elasticsearch-us/62460512?noredirect=1#comment110837766_62460512">How can I read data from a list and index specific values into Elasticsearch, using python?</a></p>
  <p>I have written a script to read a list (&quot;dummy&quot;) and index it into Elasticsearch.
  I converted the list into a list of dictionaries and used the &quot;Bulk&quot; API to index it into Elasticsearch.
  The script used to work (check the attached link to the related question). But it is no longer working after adding &quot;timestamp&quot; and the function &quot;initialize_elasticsearch&quot;.</p>
  <p>So, what is wrong? Should I be using JSON instead of the list of dictionaries?</p>
  <p>I have also tried using only 1 dictionary of the list. In that case there is no error but nothing gets indexed.</p>
  <p>THIS IS THE ERROR</p>
  <p><a href="https://i.stack.imgur.com/yYTK1.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/yYTK1.png" alt="enter image description here" /></a></p>
  <p>THIS IS THE LIST (dummy)</p>
  <pre><code>[
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: efficientnet-edgetpu-S_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 23.1&quot;,
      &quot;Time(ms): 5.7&quot;,
      &quot;Inference: corkscrew, bottle screw&quot;,
      &quot;Score: 0.03125 &quot;,
      &quot;TPU_temp(°C): 57.05&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: efficientnet-edgetpu-M_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 29.3&quot;,
      &quot;Time(ms): 10.8&quot;,
      &quot;Inference: dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk&quot;,
      &quot;Score: 0.09375 &quot;,
      &quot;TPU_temp(°C): 56.8&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: efficientnet-edgetpu-L_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 45.6&quot;,
      &quot;Time(ms): 31.0&quot;,
      &quot;Inference: pick, plectrum, plectron&quot;,
      &quot;Score: 0.09766 &quot;,
      &quot;TPU_temp(°C): 57.55&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: inception_v3_299_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 68.8&quot;,
      &quot;Time(ms): 51.3&quot;,
      &quot;Inference: ringlet, ringlet butterfly&quot;,
      &quot;Score: 0.48047 &quot;,
      &quot;TPU_temp(°C): 57.3&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: inception_v4_299_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 121.8&quot;,
      &quot;Time(ms): 101.2&quot;,
      &quot;Inference: admiral&quot;,
      &quot;Score: 0.59375 &quot;,
      &quot;TPU_temp(°C): 57.05&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: inception_v2_224_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 34.3&quot;,
      &quot;Time(ms): 16.6&quot;,
      &quot;Inference: lycaenid, lycaenid butterfly&quot;,
      &quot;Score: 0.41406 &quot;,
      &quot;TPU_temp(°C): 57.3&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: mobilenet_v2_1.0_224_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 14.4&quot;,
      &quot;Time(ms): 3.3&quot;,
      &quot;Inference: leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea&quot;,
      &quot;Score: 0.36328 &quot;,
      &quot;TPU_temp(°C): 57.3&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: mobilenet_v1_1.0_224_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 14.5&quot;,
      &quot;Time(ms): 3.0&quot;,
      &quot;Inference: bow tie, bow-tie, bowtie&quot;,
      &quot;Score: 0.33984 &quot;,
      &quot;TPU_temp(°C): 57.3&quot;,
      &quot;labels: imagenet_labels.txt &quot;,
      &quot;Model: inception_v1_224_quant_edgetpu.tflite &quot;,
      &quot;Image: insect.jpg &quot;,
      &quot;Time(ms): 21.2&quot;,
      &quot;Time(ms): 3.6&quot;,
      &quot;Inference: pick, plectrum, plectron&quot;,
      &quot;Score: 0.17578 &quot;,
      &quot;TPU_temp(°C): 57.3&quot;,
  ]
  </code></pre>
  <p>THIS IS THE SCRIPT</p>
  <pre><code>import elasticsearch6  
  from elasticsearch6 import Elasticsearch, helpers
  import datetime
  import re
  
  
  
  ES_DEV_HOST = &quot;http://localhost:9200/&quot;
  INDEX_NAME = &quot;coral_ia&quot; #name of index
  DOC_TYPE = 'coral_edge'  #type of data
  
  
  
  ##This is the list
  dummy = ['labels: imagenet_labels.txt \n', '\n', 'Model: efficientnet-edgetpu-S_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 23.1\n', 'Time(ms): 5.7\n', '\n', '\n', 'Inference: corkscrew, bottle screw\n', 'Score: 0.03125 \n', '\n', 'TPU_temp(°C): 57.05\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: efficientnet-edgetpu-M_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 29.3\n', 'Time(ms): 10.8\n', '\n', '\n', &quot;Inference: dragonfly, darning needle, devil's darning needle, sewing needle, snake feeder, snake doctor, mosquito hawk, skeeter hawk\n&quot;, 'Score: 0.09375 \n', '\n', 'TPU_temp(°C): 56.8\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: efficientnet-edgetpu-L_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 45.6\n', 'Time(ms): 31.0\n', '\n', '\n', 'Inference: pick, plectrum, plectron\n', 'Score: 0.09766 \n', '\n', 'TPU_temp(°C): 57.55\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: inception_v3_299_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 68.8\n', 'Time(ms): 51.3\n', '\n', '\n', 'Inference: ringlet, ringlet butterfly\n', 'Score: 0.48047 \n', '\n', 'TPU_temp(°C): 57.3\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: inception_v4_299_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 121.8\n', 'Time(ms): 101.2\n', '\n', '\n', 'Inference: admiral\n', 'Score: 0.59375 \n', '\n', 'TPU_temp(°C): 57.05\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: inception_v2_224_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 34.3\n', 'Time(ms): 16.6\n', '\n', '\n', 'Inference: lycaenid, lycaenid butterfly\n', 'Score: 0.41406 \n', '\n', 'TPU_temp(°C): 57.3\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: mobilenet_v2_1.0_224_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 14.4\n', 'Time(ms): 3.3\n', '\n', '\n', 'Inference: leatherback turtle, leatherback, leathery turtle, Dermochelys coriacea\n', 'Score: 0.36328 \n', '\n', 'TPU_temp(°C): 57.3\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: mobilenet_v1_1.0_224_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 14.5\n', 'Time(ms): 3.0\n', '\n', '\n', 'Inference: bow tie, bow-tie, bowtie\n', 'Score: 0.33984 \n', '\n', 'TPU_temp(°C): 57.3\n', '##################################### \n', '\n', 'labels: imagenet_labels.txt \n', '\n', 'Model: inception_v1_224_quant_edgetpu.tflite \n', '\n', 'Image: insect.jpg \n', '\n', '*The first inference on Edge TPU is slow because it includes loading the model into Edge TPU memory*\n', 'Time(ms): 21.2\n', 'Time(ms): 3.6\n', '\n', '\n', 'Inference: pick, plectrum, plectron\n', 'Score: 0.17578 \n', '\n', 'TPU_temp(°C): 57.3\n', '##################################### \n', '\n']
  
  #This is to clean data and filter some values
  regex = re.compile(r'(\w+)\((.+)\):\s(.*)|(\w+:)\s(.*)')
  match_regex = list(filter(regex.match, dummy))
  match = [line.strip('\n') for line in match_regex]   
  print(&quot;match list&quot;, match, &quot;\n&quot;)
  
  
  ##Converts the list into a list of dictionaries
  groups = [{}]
  
  for line in match:
      key, value = line.split(&quot;: &quot;, 1)
      if key == &quot;labels&quot;:
          if groups[-1]:
              groups.append({})
      groups[-1][key] = value
  
  
  
  &quot;&quot;&quot;
  Initialize Elasticsearch by server's IP'
  &quot;&quot;&quot;
  def initialize_elasticsearch():
      n = 0
      while n &lt;= 10:
          try:
              es = Elasticsearch(ES_DEV_HOST)
              print(&quot;Initializing Elasticsearch...&quot;)
              return es
          except elasticsearch6.exceptions.ConnectionTimeout as e:  ###elasticsearch
              print(e)
              n += 1
              continue
      raise Exception
  
  
  
  &quot;&quot;&quot;
  Create an index in Elasticsearch if one isn't already there
  &quot;&quot;&quot;
  def initialize_mapping(es):
      mapping_classification = {
          'properties': {
              'timestamp': {'type': 'date'},
              #'type': {'type':'keyword'}, &lt;--- I have removed this 
              'labels': {'type': 'keyword'},
              'Model': {'type': 'keyword'},
              'Image': {'type': 'keyword'},
              'Time(ms)': {'type': 'short'},
              'Inference': {'type': 'text'},
              'Score': {'type': 'short'},
              'TPU_temp(°C)': {'type': 'short'}
          }
      }
      print(&quot;Initializing the mapping ...&quot;)  
      if not es.indices.exists(INDEX_NAME):
          es.indices.create(INDEX_NAME)
          es.indices.put_mapping(body=mapping_classification, doc_type=DOC_TYPE, index=INDEX_NAME)
          
  
  
  
  def generate_actions():
      actions = {
          '_index': INDEX_NAME,
          'timestamp': str(datetime.datetime.utcnow().strftime(&quot;%Y-%m-%d&quot;'T'&quot;%H:%M:%S&quot;)),
          '_type': DOC_TYPE,
          '_source': groups
          }
  
      yield actions
      print(&quot;Generating actions ...&quot;)
      #print(&quot;actions:&quot;, actions)
      #print(type(actions), &quot;\n&quot;)
  
  
  
  def main():
      es=initialize_elasticsearch()
      initialize_mapping(es)  
      
      try:
          res=helpers.bulk(client=es, index = INDEX_NAME, actions = generate_actions())
          print (&quot;\nhelpers.bulk() RESPONSE:&quot;, res)
          print (&quot;RESPONSE TYPE:&quot;, type(res))
          
      except Exception as err:
          print(&quot;\nhelpers.bulk() ERROR:&quot;, err)
  
  
  if __name__ == &quot;__main__&quot;:
      main()
  
  </code></pre>
  <p>THIS IS THE CODE WHEN TESTING WITH ONLY 1 DICTIONARY</p>
  <pre><code>regex = re.compile(r'(\w+)\((.+)\):\s(.*)|(\w+:)\s(.*)')
  match_regex = list(filter(regex.match, dummy))
  match = [line.rstrip('\n') for line in match_regex]   #quita los saltos de linea
  #print(&quot;match list&quot;, match, &quot;\n&quot;)
  
  
  features_wanted='ModelImageTime(ms)InferenceScoreTPU_temp(°C)'
  match_out = {i.replace(' ','').split(':')[0]:i.replace(' ','').split(':')[1] for i in match if i.replace(' ','').split(':')[0] in features_wanted}
  </code></pre>
  <p>-------------------EDIT-------------------------</p>
  <p>No errors, but &quot;Generating actions ...&quot; is not being printed.</p>
  <p><a href="https://i.stack.imgur.com/MbaDb.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/MbaDb.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/HD8KN.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/HD8KN.png" alt="enter image description here" /></a></p>
  <p>THIS IS THE MAPPING</p>
  <p><a href="https://i.stack.imgur.com/z5emq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/z5emq.png" alt="enter image description here" /></a></p>
  <p>THIS APPEARS WHEN I WANT TO SEE IF THE DATA WAS INDEXED</p>
  <p><a href="https://i.stack.imgur.com/iTJdx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/iTJdx.png" alt="enter image description here" /></a></p>
  <p>IT SEEMS THE DATA HAS BEEN INDEXED...</p>
  <p><a href="https://i.stack.imgur.com/LYNfp.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/LYNfp.png" alt="enter image description here" /></a></p>
  <p>----------------------EDIT-----------------------</p>
  <p>I modified the <code>generate_actions</code></p>
  <pre><code>def generate_actions():
      return[{
          '_index': INDEX_NAME,
          '_type': DOC_TYPE,
          '_source': {
              &quot;any&quot;: doc,
              &quot;@timestamp&quot;: str(datetime.datetime.utcnow().strftime(&quot;%Y-%m-%d&quot;'T'&quot;%H:%M:%S&quot;)),}
          }
          for doc in groups]
  </code></pre>
  <p><a href="https://i.stack.imgur.com/foWLx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/foWLx.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.stack.imgur.com/Uh8au.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/Uh8au.png" alt="enter image description here" /></a></p>