title: Python BeautifulSoup Automatically tracking content table rows andcolumns
body: <p>First let me say that I am new to Stack and to Python. I just started working with it last week. I am however a seasoned PHP/C++/Pascal/ADA/B/Forth (showing my age) programmer. </p>
  
  <p>I have written a script that pulls product pages from a website and stores them in my local MySQL database. I did this so that I can crawl the site late at night when the load is light. I now need to sort through the html of each page and get the product descriptions. These are placed in tables. However, each page may have the needed values in different rows/columns.</p>
  
  <p>The things I can be sure of are:<br></p>
  
  <ul>
  <li>Each table has a heading that defines the data in the rows/columns below it.</li>
  <li>The heading text is consistent for each value i.e. 'Part' always describes the part type and 'Part No.' always describes a part number.</li>
  <li>Not all pages will contain all the data desired. So If not located it must save what it finds.</li>
  </ul>
  
  <blockquote>
    <p>In the section below it is the second part, getting the data values that I am having trouble with. How do I select the nth column from a row?</p>
  </blockquote>
  
  <p>My current approach is:</p>
  
  <h3> To Get Desired Columns </h3>
  
  <ul>
  <li>Get html doc from db</li>
  <li>Grab the table (my table is always contained in the only div on the page.</li>
  <li>Grab all the rows (really only need to do this for the first row)</li>
  <li>For each row grab the row and column index' when I find a desired field names.</li>
  </ul>
  
  <h3> To Get Data Values </h3>
  
  <ul>
  <li>For each row:</li>
  <li>Skip the row if it was a header (save the row counts for those with header fields)</li>
  <li>for each column grab the text value.</li>
  <li>Save the values to db</li>
  </ul>
  
  <p>The important part of my page looks like this: </p>
  
  <pre><code>&lt;div&gt;
     ... 
     &lt;table&gt;
        &lt;tr&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&lt;b&gt;Item&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&lt;b&gt;Description&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&lt;b&gt;Part No.&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&lt;b&gt;Color&lt;/b&gt;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;/tr&gt;
        &lt;tr&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;/tr&gt;
        &lt;tr&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;Toaster&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;2-Slice&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;#25713&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;td&gt;Chorme&lt;/td&gt;&lt;td&gt;&amp;nbsp;&lt;/td&gt;&lt;/tr&gt;
     &lt;/table&gt;
     ...
  &lt;/div&gt;
  </code></pre>
  
  <p>A Big thank you to anyone who responds.</p>