title: Taking too long to run a python script
body: <p>The script is basically measuring the relationship between students based on the book they have borrowed (code). So I've built a tree for different type of books using ete2 package. Now I'm trying to write a piece of code that takes the data from the tree and a csv file and does some data analysis through the function relationship.The csv file contains more than 50,000 rows. The problem is that it takes to long to run the code (around 7 days), while it uses only 10 to 20% of my computer CPU and memory.</p>
  
  <p>Here is an example of the csv file I've used:</p>
  
  <pre><code>ID Code    Count 
  1    A1...   6
  1    A2...   5
  2    A....   4
  2    D....   1
  2    A1...   2
  3    D....   5
  3    D1...   3
  3    D2...   5
  </code></pre>
  
  <p>Here is the code:</p>
  
  <pre><code>from ete2 import Tree
  import pandas as pd
  import numpy as np
  from __future__ import division
  import math
  
  
  data= pd.read_csv('data.csv', names=['ID','Code', 'Count'])
  codes_list= list (set(data['Code']))
  total_codes= data.shape[0]
  students_list= list (set(data['ID']))
  
  
  
  ####################################
  
  
  # generate the tree
  t = Tree (".....;", format =0)
  for i in codes_list:
      if '....' in i:
          node = t.search_nodes(name = '.....')
          node[0].add_child(name= i)
  for i in codes_list:
      if '...' in i and '....' not in i:
          if i[0]+'....' in codes_list:
              node = t.search_nodes(name = i[0]+'....')
              node[0].add_child(name= i)
          else:
              node = t.search_nodes(name = '.....')
              node[0].add_child(name= i)
  
  # save the tree in a file 
  t.write( outfile= file_path + "Codes_tree.nh", format =3)
  return t.get_ascii(show_internal=True)
  
  ####################################
  
  def relationship(code1,code2):
  
      code1_ancestors= t.search_nodes(name=code1)[0].get_ancestors()
      code2_ancestors=t.search_nodes(name=code2)[0].get_ancestors(
      common_ancestors = []
      for a1 in code1_ancestors:
          for a2 in code2_ancestors:
              if a1==a2:
                  common_ancestors.append(a1)
      IC_values = []
      for ca in common_ancestors:
          code_descendants=[]
          for gd in ca.get_descendants():
              code_descendants.append(gd.name)
          code_descendants.append(ca)
          frequency= 0
          for k in code_descendants:
                  frequency= frequency + code_count.Count[k]
  
          IC = - (math.log (frequency / float (total_codes)))
          IC_values.append (IC)
  
      IC_max= max(IC_values)
      return IC_max
  
  ##################
  
  relationship_matrix = pd.DataFrame(index=[students_list], columns=[students_list])
  for student in students_list:
  p1= list (self.data.Code[data.ID==student])
  for student1 in students_list:
      p2= list data.Code[data.PID==student1])
      student_l=[]
      for l in p1:
          for m in p2:
              student_l.append(relationship(l,m))
  
      max_score = np.max(np.array(student_l).astype(np.float))
      relationship_matrix.loc[student,student1] = max_score
  
  print relationship_matrix
  </code></pre>