title: How to get the pymongo collections based on updatedtimestamp in unix format
body: <p>I have a MongoDB collection which is having updatedtimestamp column. I am passing from and to date using an external yaml file. But updated timestamp column is in the UNIX timestamp.</p>
  <p>Any suggestions on this? Or anyway to bring the latest records based on updatedtimestamp which is captured in unixtimestamp?</p>
  <p><strong>data format</strong></p>
  <pre><code>id:1223344444444
  createTimestamp:1640845784796
  updateTimestamp:1640845784796
  deleteTimestamp:null
  uuid:&quot;abcdefgh&quot;
  </code></pre>
  <p><strong>YAML file</strong></p>
  <pre><code>collection_name: 
  incremental: true
  date_field: updateTimestamp
  start_date: 2022-01-01
  end_date: 2022-01-02
  hours_interval: 24
  </code></pre>
  <p><strong>Incremental query</strong></p>
  <pre><code># Build interval dates
              initial_dt_t = datetime.combine(day, datetime.min.time()) + timedelta(
                  minutes=lower_limit
              )
              final_dt_t = datetime.combine(day, datetime.min.time()) + timedelta(
                  minutes=upper_limit
              )
  
              initial_dt = mktime(initial_dt_t.timetuple())
              final_dt = mktime(final_dt_t.timetuple())
  
              print(&quot;initial-dt&quot;)
              print(initial_dt_t)
              print(final_dt_t)
  
              print(&quot;unix&quot;)
              print(initial_dt)
              print(final_dt)
  
              # Connect to MongoDB
              mongo_client = MongoClient(MONGODB_URI)
              mongo_db = mongo_client[MONGODB_NAME]
              collection = mongo_db[COLLECTION_NAME]
  
              # Build aggregation pipeline for MongoDB
              or_list = []
              print(&quot;coming to query&quot;)
  
              # Check if there is a single field
              if &quot;date_field&quot; in config.keys():
                  or_list.append(
                      {
                          config[&quot;date_field&quot;]: {
                              &quot;$gte&quot;: initial_dt,
                              &quot;$lt&quot;: final_dt,
                          }
                      }
                  )
  
              # Check if there are multiple date fields
              if &quot;date_fields&quot; in config.keys():
                  for date_field in config[&quot;date_fields&quot;]:
                      or_list.append(
                          {
                              config[&quot;date_field&quot;]: {
                                  &quot;$gte&quot;: initial_dt,
                                  &quot;$lt&quot;: final_dt,
                              }
                          }
                      )
  
              # Get data from MongoDB
              query = {&quot;$or&quot;: or_list}
  
              # Keep track of batch number
              batch_number = 0
              print(&quot;chunk&quot;)
              data = pd.DataFrame(list(collection.find(query)))
              print(&quot;dataframe&quot;)
              print(data)
  </code></pre>