title: How to get newly inserted records based on date from AWS ElasticSearch index using query
body: <p>I'm totally new to AWS ElasticSearch Service.</p>
  <p>First I tried to establish connection using python, It was successful.</p>
  <p>Then I tried to get a single record from a specific index and bulk records from an index using query.
  Like:</p>
  <p><code>df = es.get(index=&quot;demo_data&quot;, doc_type=&quot;_doc&quot;, id=&quot;2&quot;) -- For getting single record </code></p>
  <p><code>result_dict = es.search(index=&quot;demo_data&quot;, body={&quot;query&quot;: {&quot;match_all&quot;: {}}}) -- For getting Bulk Records</code></p>
  <p>Now, I've been trying to fetch only newly inserted records i.e.(It can be a single record or a bulk of records)from that same index based on date using query, but I couldn't get an exact reference, I tried several ways like <strong>filter query with sort</strong> etc... Which either gets me the last record of the index or all the records in descending order.</p>
  <p>I couldn't able to get the exact new records based on the date from my previous load.</p>
  <pre><code>from elasticsearch import Elasticsearch, RequestsHttpConnection
  from requests_aws4auth import AWS4Auth
  from pandasticsearch import Select
  from elasticsearch_dsl import Search
  
  spark = SparkSession.builder.appName('abc').getOrCreate()
  
  
  host = 'ENDPOINT' # ES hostname 
  region = 'REGION' # our region ID
  
  service = 'es'
  
  awsauth = AWS4Auth('Secret_Key', 'Secret_ID', region, service)
  
  es = Elasticsearch(
      hosts = [{'host': host, 'port': 443}],
      http_auth = awsauth,
      use_ssl = True,
      verify_certs = True,
      connection_class = RequestsHttpConnection
  )
  
  s = Search(index='demo_data').using(es).query(&quot;match&quot;, date= max(pandas_df.date))
                          
  pandas_df = Select.from_dict(s).to_pandas()
  
  display(pandas_df)
  
  </code></pre>
  <p>Help is much appreciated. Thanks in Advance...</p>