title: Python Pandas Mixed Boolean Yes/True and NaN Columns
body: <p>I am doing a health science course where R or Stata are recommended.  I'm trying to use Python / Numpy / Pandas instead as I wish to use it in future for financial time series analysis. </p>
  
  <p>The data was Stata format so I copied the fields and saved them as a <code>CSV</code>.
  All fields imports are fine except that there are a number of columns of Yes/No some of which have blank fields.</p>
  
  <p>Import command is </p>
  
  <pre><code>fhs = pd.io.parsers.read_csv('F:\\BioStatistics\\fds\\fhs_c2.csv', header=0, index_col=0)
  </code></pre>
  
  <p>If there is a blank field the dtype is object (makes sense)</p>
  
  <p>If there are no blanks some columns convert to <code>TRUE/FALSE</code>, others leave as <code>Yes/No</code> but dtype is bool.  Any idea why?</p>
  
  <p>I want all to by one dtype and expressed one way for viewing + stat analysis.</p>
  
  <p>I have achieve this by adding a row at the beginning with blank cells for the Boolean columns that had no spaces - so everything becomes an object.  Then I use 
  <code>fhs = fhs.drop([1002])</code> to drop that row and data types are still good.</p>
  
  <p>I'd love to save it without this row and just be able to load the data each time with "correct" types but don't know if it possible when some of the columns will have all yes or no, and some will have blank cells.  Is it possible?</p>
  
  <p>Thanks, Sorry about the newbie question.</p>
  
  <h2>Example:</h2>
  
  <p>Importing</p>
  
  <pre><code>      C1    C2    C3
  
  R1   Yes   Yes    No
  
  R2    No    No    No
  
  R3   Yes         Yes
  
  R4   Yes   Yes   Yes
  </code></pre>
  
  <p>first column comes into df as Yes, No, Yes, Yes  type bool xxxx below</p>
  
  <p>2nd   column comes into df as Yes, No, NaN, Yes  type object</p>
  
  <p>3rd   column comes into df as FALSE, FALSE, TRUE, TRUE  type bool</p>
  
  <p>Damn.  Just checked.  I was wrong.  If its yes and no then the column type is object.</p>
  
  <p>I'd like to tell it when importing to make them all object and stick with yes and no because:
  1.  I think the 2nd column must be object (as its mixed otherwise i think)
  2.  The data set is in yes / no and other class members will be looking at yes and no</p>
  
  <h2>What happened when I tried the solution.</h2>
  
  <p>Here's my data:  <a href="http://i.imgur.com/sVRK0.gif" rel="nofollow">link</a>  </p>
  
  <p>Here's the code:<br>
      from pandas import *<br>
      import numpy as np<br>
      import pandas as pd  </p>
  
  <pre><code>def convert_bool(col):
      if str(col).title() ==  "True": #check for nan
          return "Yes"
      elif str(col).title() == "False":
          return "No"
      else:
          return col
  
  fhs = pd.read_csv('F:\\BioStatistics\\fds\\StatExport.csv', converters={"death": lambda x:convert_bool(x)}, header=0, index_col=0)  
  </code></pre>
  
  <p>and output <a href="http://i.imgur.com/oJ42H.gif" rel="nofollow">link</a>  </p>