title: convert sql code to python pandas dataframe operation
body: <p>So I am having trouble with processing and manipulating large ammount of data.<br>
  My table 1 consist of 2 milions records for example: </p>
  
  <pre><code>TABLE1
  ID; AGE; SEX;
  1; 1; F
  2; 1; F
  3; 1; M
  4; 2; F
  4; 2; m
  </code></pre>
  
  <p>and another table with data: </p>
  
  <pre><code>TABLE2
  TYPE, SEX, AGE_RANGE
  1, F, 1-5
  1, F, 6-10
  1, F, 11-15
  2, F, 1-5
  1, M, 1-5
  2, M, 1-5
  </code></pre>
  
  <p>I am creating a join:</p>
  
  <pre><code>SELECT tbl1.id, tbl1.age, tbl2.sex, table2.type
  FROM table1 tbl1, table2 tbl2
  WHERE 
  tbl1.age &gt;= tbl2.min_age AND tbl1.age &lt;= tbl2.max_age AND
  tbl1.sex = tbl2.sex
  </code></pre>
  
  <p>The result is a really big table (SELECT should return around 40milions rows
  Then I use groupby and filter method to further filter my records. I have a problem because I get MemmoryError when running my code. I was thinking about changing csv to better accomodate pandas dataframe (to avoid using sql) 
  My code example is below: </p>
  
  <pre><code>df_table1 = pd.read_csv(table1)
  df_table2 = pd.read_csv(table2)
  conn = sqlite3.connect(':memory:')
  
  df_table1.to_sql('table1', conn, index=False)
  df_table2.to_sql('table2', conn, index=False)
  
  sql = '''SELECT tbl1.id, tbl1.age, tbl2.sex, table2.type
  FROM table1 tbl1, table2 tbl2
  WHERE 
  tbl1.age &gt;= tbl2.min_age AND tbl1.age &lt;= tbl2.max_age AND
  tbl1.sex = tbl2.sex'''
  
  df_filtered = pd.read_sql_query(sql, conn)
  
  filter = ['sex', 'age_range', 'type']
  grouped_data = df_filtered.groupby(iskanje).apply(select_random_row)
  #select_random_row is my custom_made function which returns just select few rows that groupby returns
  grouped_data .to_csv("result.csv", sep=';', encoding='utf-8') 
  </code></pre>
  
  <p>The biggest problem is my sql because I get out of memmory error but even without the error it is just taking too long to load it. I was thinking about changing my csv and use exact AGE and not AGE_RANGE.... that would allow me to use pandas querying instead of sql</p>