title: python - large and small dataset comparison
body: <p>I have 2 pandas dataframes in python. One is very small <code>df1</code> (&lt;200 rows) and other one is very large <code>df2</code> (>20M rows). I want to compare the small df with the large one and <strong>get all the indexes of larger dataframe <code>df2</code> for all matching rows.</strong> </p>
  
  <p>I have coded the following - </p>
  
  <p><code>df3 = df2[df2['Row Idx'].isin(df1['Row Idx'])]</code></p>
  
  <p>In this all the rows of <code>df2</code> are compared with <code>df1</code> which is not efficient and the execution is very slow. For 200 rows it takes nearly 30 secs. I have tried to run it in Google Colab and AWS also but the performance is similar. </p>
  
  <p>This comparison is done in a loop for ~400K times as per some condition. </p>
  
  <p>Is there way to improve the performance?</p>