title: Python Pandas - Scrape Multiple Tables from Multiple URLs into One Combined Dataframe
body: <p>I'm new to Python.</p>
  <p>I'm trying to scrape multiple tables from pages on a website. I want to make a dataframe from each table and combine all of the dataframes together into one large, combined dataframe.</p>
  <p>Here are the steps I've completed so far:</p>
  <ol>
  <li>Import packages</li>
  <li>Open website and login</li>
  <li>Jump to the page with the data</li>
  <li>Add numbers to dictionary and loop through them to build the url for each page. The number starting with zeros are strings so the zero doesn't get dropped.</li>
  </ol>
  <p>This is where I'm stuck. I can create one dataframe, but I'm having trouble creating a dataframe for EACH table and combining them together. I've tried the append function, but I end up with one dataframe so I must be missing something.</p>
  <p>To clarify, I have about 700 urls. The urls are at the same website, but each page has a unique 4 digit number at the end of the url. I'm able to loop through each url using the code below. As each url is opened, I want to create a dataframe from the table at that url. Then I want to combine all of the dataframes into one large dataframe.</p>
  <p>I hope this is clear. Any help is appreciated!</p>
  <pre><code>from bs4 import BeautifulSoup
  import pandas as pd
  from selenium import webdriver
  DRIVER_PATH = 'C:\\Path to chromedriver.exe'
  from selenium.webdriver import chrome
  from selenium.webdriver.support.select import Select
  driver = webdriver.Chrome(executable_path='C:\\Path to chromedriver.exe')
  import numpy as np
  from openpyxl import load_workbook
  
  #open website 
  driver.get('https://website.com')
  
  #Login to website
  element = driver.find_element_by_id('txtUID')
  element.click()
  element.clear()
  element.send_keys(&quot;userID&quot;)
  element = driver.find_element_by_id('txtPWD')
  element.click()
  element.clear()
  element.send_keys(&quot;userPW&quot;)
  import time
  time.sleep(1)
  element = driver.find_element_by_id('Send')
  element.click()
  
  #Jump to detail page
  driver.get('https://DetailPage')
     
  numbers = ['0101','0110','0130','0140','0150','0160','0170','0172','0180','0184','0185','0186','0190','0193','0203','0211','0213','0240','0260','0261','0270','0274','0280','0281','0285','0290','0300','0302','0310','0320','0340','0380','0412','0440','0490','0500','0540','0560','0603','0610','0630','0635','0652','0660','0670','0680','0681','0690','0713','0721','0740','0742','0750','0761','0776','0784','0790','0794','0801','0802','0804','0810','0851','0863','0870','0871','0881','0884','0886',1030,1056,1060,1090,1100,1170,1176,1180,1183,1194,1220,1230,1240,1251,1255,1260,1270,1290,1321,1360,1370,1405,1430,1441,1460,1501,1513,1522,1530,1560,1632,1650,1680,1686,1701,1711,1720,1740,1750,1760,1780,1790,1800,1806,1810,1820,1850,1860,1870,1900,1901,1903,1905,1917,1931,1940,1943,1950,1970,1972,1974,2002,2003,2060,2070,2072,2083,2100,2108,2110,2118,2121,2124,2127,2142,2170,2171,2180,2202,2203,2211,2212,2240,2260,2290,2302,2315,2321,2322,2332,2340,2341,2350,2360,2362,2380,2410,2440,2450,2510,2600,2720,2721,2725,2740,2760,2764,2801,2810,2811,2821,2824,2825,2830,2832,2850,2855,2860,2861,2862,2870,2874,2912,2930,2940,2943,2960,2961,2965,2970,2971,2992,3000,3003,3004,3012,3013,3014,3015,3022,3024,3025,3026,3027,3031,3032,3034,3040,3044,3045,3050,3051,3060,3070,3090,3120,3144,3160,3171,3172,3176,3190,3202,3203,3210,3220,3223,3250,3260,3261,3271,3275,3276,3283,3291,3296,3301,3306,3311,3313,3316,3326,3333,3340,3342,3343,3344,3350,3351,3352,3354,3357,3362,3363,3364,3371,3386,3390,3398,3461,3491,3523,3524,3531,3540,3560,3570,3581,3610,3620,3660,3661,3700,3702,3711,3715,3720,3725,3731,3740,3741,3783,3800,3810,3811,3830,3836,3850,3880,3881,3950,3961,3970,4016,4020,4021,4022,4023,4050,4052,4110,4122,4200,4210,4230,4243,4271,4305,4310,4313,4322,4324,4400,4403,4404,4411,4414,4416,4421,4422,4431,4432,4440,4470,4490,4501,4510,4511,4512,4521,4526,4530,4532,4541,4560,4580,4607,4620,4621,4622,4623,4624,4635,4636,4650,4651,4660,4681,4684,4690,4695,4715,4730,4737,4754,4770,4790,4793,4805,4806,4810,4813,4821,4822,4823,4830,4833,4834,4835,4836,4850,4881,4884,4902,4920,4942,4952,4955,5000,5010,5040,5070,5080,5110,5200,5220,5300,5301,5310,5320,5321,5340,5354,5370,5371,5390,5430,5431,5440,5441,5460,5486,5490,5510,5511,5521,5540,5541,5590,5594,5595,5630,5710,5850,5990,6001,6002,6004,6013,6014,6016,6017,6020,6030,6031,6043,6045,6051,6062,6066,6071,6090,6111,6135,6160,6182,6183,6220,6230,6265,6270,6300,6322,6334,6410,6412,6414,6480,6520,6560,6601,6603,6660,6780,6811,6814,6850,6860,6880,7000,7011,7031,7070,7081,7190,7222,7236,7360,7401,7414,7472,7500,7521,7525,7526,7534,7535,7552,7553,7560,7561,7570,7583,7587,7602,7610,7620,7623,7670,7702,7704,7706,7707,7711,7737,7738,7745,7753,7756,7763,7770,7780,7800,7820,7821,7823,7826,7837,7843,7850,7851,7852,7860,7872,7875,7876,7880,7882,7884,7940,7960,7970,7976,7986,7990,8000,8003,8007,8008,8010,8012,8013,8022,8025,8030,8040,8045,8046,8050,8091,8130,8137,8140,8150,8161,8200,8240,8290,8320,8321,8331,8332,8340,8346,8367,8370,8413,8440,8450,8451,8461,8465,8501,8502,8514,8520,8521,8522,8526,8530,8546,8565,8570,8571,8572,8582,8630,8640,8750,8820,8902,8904,8910,8914,8947,8970,9004,9010,9013,9014,9016,9040,9048,9050,9060,9061,9062,9080,9081,9120,9122,9132,9150,9153,9171,9172,9175,9200,9201,9202,9206,9211,9214,9221,9232,9234,9240,9246,9251,9253,9262,9270,9272,9300,9310,9321,9330,9331,9340,9371,9390,9402,9440,9450,9452,9453,9454,9455,9456,9460,9461,9470,9490,9491,9500,9505,9510,9511,9514,9530,9545,9550,9580,9581,9590,9591,9593,9701,9706,9723,9725,9730,9732,9740,9743,9750,9760,9770,9800,9802,9820,9822,9831,9844,9846,9861,9876,9910,9921,9930]
  
  str = 'https://DetailPage={}'
  
  for number in numbers:
      urls = str.format(number)
      print(urls)
      driver.get(urls)
      html = driver.page_source
      df = pd.read_html(html)
  
  print(df)
  </code></pre>