text: user10594: I want to create a dataframe in Python starting from a FASTA format file. Given the toy FASTA file that I am attaching, I built this program in Python that returns four colums corresponding to id, sequence length, sequence, animal name and rows corresponding to all the data available.
  However, I am trying to understand how to modify this code in order to create a dataframe in which classes Human and Dog have the same number of data. For example, I want to say to Python: "Append to record (that is the empty list) id, sequence length, sequence and animal for Human, but do it a number of times that is equal to the number of data of the class with minimum number of data (that is Dog)".
  I think that a while loop is needed but I have a bit troubles to understand how to do it. Any suggestion ?
  Below the Python code I wrote and the FASTA format file I used.
  import pandas as pd
  import re
  def read_fasta(file_path, columns) :
      from Bio.SeqIO.FastaIO import SimpleFastaParser 
      with open("Proof.txt") as fasta_file :  
          records = [] # create empty list
          for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘>’ character), and the sequence (with any whitespace removed). 
              record = []
              title_splits=re.findall(r"[\w']+", title) # Data cleaning is needed
            
                   
              
              record.append(title_splits[0])  #First values are ID (Append adds element to a list)
              record.append(len(sequence)) #Second values are sequences lengths
              sequence = " ".join(sequence) #It converts into one line
              record.append(sequence)#Third values are sequences
                  
              #Fourth column will contain the species
              if "Human" in title_splits:
                      record.append("Human")    
              else:
                      record.append("Dog")
                  
             
                
              records.append(record)
      return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe
  
  #Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)        
  #And in the second one the names of columns
  data = read_fasta("Proof.txt", columns=["id","sequence_length", "sequence", "animal"])
  data
  
  The FASTA format file is this:
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  >Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  
  My code prints a dataframe like:
         id  sequence_length                               sequence animal
  0   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  1   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  2   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  3   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  4   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  5   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  6   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  7   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  8   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  9   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  10  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  11  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  
  But I would like that the number of rows for Human is the same for Dog (because, in other words, I would like the same number of data for each class).
  Hoping to have been clear, I thank you in advance.
  P.S.: My program is an adapted version of this notebook .
  
  user3848: An example of contructing a pandas dataframe for fasta on Stack Exchange is here . This is clean code deploying zip outside Biopython. Constructing a pandas dataframe directly via dictionary.
  Are you intending to use split method directly on the sequence, such that each residue is a separate column in a dataframe?
  Anyway,
  title_splits=re.findall(r"[\w']+", title) # Data cleaning is needed
  
  This is a bad idea because you will remove the Genbank unique identifier (its an integer) and this should not leave the sequence.
  You will need to switch out of the Fasta object. Your approacha and @user324810's use of
  sequence = " ".join(sequence)
  
  is okayish. However, I think it should be '', or "" (no space character). I spotted manually removing fasta, well the method should do this automatically. I am not familiar with Fastaparser, but AlignIO and SeqIO will both automatically process fasta without the need deal with ">". Biopython will have a length method and whilst you have the Biopython object it is preferable to use Biopython (OOP) methods.
  from Bio.SeqIO.FastaIO import SimpleFastaParser 
  
  This should be at the top of the code.
  Overall, you manually open the data, pass it through the Bioparser then immediately dump the object. I do agree to write a pandas dataframe from a Biopython object these conversion to string, but Biopython will provide a method to print the object (pass it to a string) without all the parsing. Some of the parsing is removing key data (Genbank ID).
  Personally I would look at the Stackoverflow example and base your code around this.
  You clearly have an understanding of pandas, but I get the impression Biopython is something your not familiar with. My suspicion is that Biopython would infact answer the entirety of your question (whatever it might be) without passing to pandas.
  pandas is cool and powerful for non-molecular data, but unless your using machine/deep learning (where a dataframe is necessary input) I think Biopython will provide most of the answers.
  Important Terminology
  The class is FastaIO and the method is SimpleFastaParser. Humans and dogs are instances of this method and are therefore object(s).
  Please don't write 'class' of dogs and humans in a formal report because it is incorrect OOP and will cause alarm if your reviewer is a coder. Your code is functional programming rather than OOP. You could say the human/dog fasta object, or just refer to it as human/dog dataframes and avoid any OOP terminology.
  
  user4413: Assuming what I said in the comments is true. Here's my attempt, there may be an easier solution but this should do it:
  import pandas as pd
  import re
  
  # a function to return the count number of animals in the fasta in a dictionary
  def count_animals(file_path, separator, animal_index):
    animal_dictionary={}
    with open("Proof.txt") as fasta_file:
      fasta_contents = fasta_file.readlines()
      for line in fasta_contents:
        if line.startswith(">"):
          animal = line.split(separator)[animal_index]
          if animal not in animal_dictionary:
            animal_dictionary[animal] = 1
          else:
            animal_dictionary[animal] = animal_dictionary[animal] + 1
    return animal_dictionary
  
  def read_fasta(file_path, columns) :
    counter = 0  # counter used to print no more than number of times 'Dog'
    animal_d = count_animals(file_path, '|', 1) # execute function
    max_animal =  max(animal_d.values()) # get max number of animals
    min_animal =  min(animal_d.values()) # get min number of animals
  
    from Bio.SeqIO.FastaIO import SimpleFastaParser 
    with open("Proof.txt") as fasta_file :
      records = [] # create empty list
      for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘>’ character), and the sequence (with any whitespace removed). 
        record = []
        title_splits=re.findall(r"[\w']+", title) # Data cleaning is needed
  
        record.append(title_splits[0])  #First values are ID (Append adds element to a list)
        record.append(len(sequence)) #Second values are sequences lengths
        sequence = " ".join(sequence) #It converts into one line
        record.append(sequence)#Third values are sequences
        # if 'Human' exists in title and the counter is less than min_animal('Dog'), add the record
        if 'Human' in title and counter < min_animal:
          counter += 1 # add +1 to counter
  
          #Fourth column will contain the species
          record.append("Human")  
            
          records.append(record)
        elif 'Dog' in title:
          record.append("Dog")  
            
          records.append(record)
  
      # print(records)
    return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe
  
  #Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)    
  #And in the second one the names of columns
  data = read_fasta("Proof.txt", columns=["id","sequence_length", "sequence", "animal"])
  
  print(data)
  
  Should give:
        id  sequence_length                               sequence animal
  0  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  1  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  2  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  3  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  4  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  5  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  6  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  7  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  
  
original_text: 10594: <p>I want to create a dataframe in Python starting from a FASTA format file. Given the toy FASTA file that I am attaching, I built this program in Python that returns four colums corresponding to id, sequence length, sequence, animal name and rows corresponding to all the data available.</p>
  <p>However, I am trying to understand how to modify this code in order to create a dataframe in which classes Human and Dog have the same number of data. For example, I want to say to Python: &quot;Append to record (that is the empty list) id, sequence length, sequence and animal for Human, but do it a number of times that is equal to the number of data of the class with minimum number of data (that is Dog)&quot;.</p>
  <p>I think that a <code>while</code> loop is needed but I have a bit troubles to understand how to do it. Any suggestion ?</p>
  <p>Below the Python code I wrote and the FASTA format file I used.</p>
  <pre><code>import pandas as pd
  import re
  def read_fasta(file_path, columns) :
      from Bio.SeqIO.FastaIO import SimpleFastaParser 
      with open(&quot;Proof.txt&quot;) as fasta_file :  
          records = [] # create empty list
          for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘&gt;’ character), and the sequence (with any whitespace removed). 
              record = []
              title_splits=re.findall(r&quot;[\w']+&quot;, title) # Data cleaning is needed
            
                   
              
              record.append(title_splits[0])  #First values are ID (Append adds element to a list)
              record.append(len(sequence)) #Second values are sequences lengths
              sequence = &quot; &quot;.join(sequence) #It converts into one line
              record.append(sequence)#Third values are sequences
                  
              #Fourth column will contain the species
              if &quot;Human&quot; in title_splits:
                      record.append(&quot;Human&quot;)    
              else:
                      record.append(&quot;Dog&quot;)
                  
             
                
              records.append(record)
      return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe
  
  #Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)        
  #And in the second one the names of columns
  data = read_fasta(&quot;Proof.txt&quot;, columns=[&quot;id&quot;,&quot;sequence_length&quot;, &quot;sequence&quot;, &quot;animal&quot;])
  data
  </code></pre>
  <p>The FASTA format file is this:</p>
  <pre><code>&gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Dog|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  &gt;Numer|Human|Hearth
  HSSFIEIVNIEHVIEHIVK
  </code></pre>
  <p>My code prints a dataframe like:</p>
  <pre><code>       id  sequence_length                               sequence animal
  0   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  1   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  2   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  3   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  4   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  5   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  6   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  7   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  8   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  9   Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  10  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  11  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  </code></pre>
  <p>But I would like that the number of rows for Human is the same for Dog (because, in other words, I would like the same number of data for each class).</p>
  <p>Hoping to have been clear, I thank you in advance.</p>
  <p>P.S.: My program is an adapted version of <a href="https://github.com/aws-samples/amazon-sagemaker-protein-classification/blob/main/sm-protein-classification.ipynb" rel="nofollow noreferrer">this</a> notebook .</p>
   3848: <p>An example of contructing a pandas dataframe for fasta on Stack Exchange is <strong><a href="https://stackoverflow.com/questions/69829981/how-to-convert-fasta-file-into-a-pandas-dataframe">here</a></strong> . This is clean code deploying <code>zip</code> outside Biopython. Constructing a pandas dataframe directly via dictionary.</p>
  <p>Are you intending to use <code>split</code> method directly on the sequence, such that each residue is a separate column in a dataframe?</p>
  <p>Anyway,</p>
  <pre><code>title_splits=re.findall(r&quot;[\w']+&quot;, title) # Data cleaning is needed
  </code></pre>
  <p>This is a bad idea because you will remove the Genbank unique identifier (its an integer) and this should not leave the sequence.</p>
  <p>You will need to switch out of the Fasta object. Your approacha and @user324810's use of</p>
  <pre><code>sequence = &quot; &quot;.join(sequence)
  </code></pre>
  <p>is okayish. However, I think it should be '', or &quot;&quot; (no space character). I spotted manually removing fasta, well the method should do this automatically. I am not familiar with Fastaparser, but AlignIO and SeqIO will both automatically process fasta without the need deal with &quot;&gt;&quot;. Biopython will have a length method and whilst you have the Biopython object it is preferable to use Biopython (OOP) methods.</p>
  <pre><code>from Bio.SeqIO.FastaIO import SimpleFastaParser 
  </code></pre>
  <p>This should be at the top of the code.</p>
  <p>Overall, you manually open the data, pass it through the Bioparser then immediately dump the object. I do agree to write a pandas dataframe from a Biopython object these conversion to string, but Biopython will provide a method to print the object (pass it to a string) without all the parsing. Some of the parsing is removing key data (Genbank ID).</p>
  <p>Personally I would look at the Stackoverflow example and base your code around this.</p>
  <p>You clearly have an understanding of pandas, but I get the impression Biopython is something your not familiar with. My suspicion is that Biopython would infact answer the entirety of your question (whatever it might be) without passing to pandas.</p>
  <p>pandas is cool and powerful for non-molecular data, but unless your using machine/deep learning (where a dataframe is necessary input) I think Biopython will provide most of the answers.</p>
  <p><strong>Important Terminology</strong>
  The class is FastaIO and the method is SimpleFastaParser. Humans and dogs are instances of this method and are therefore object(s).</p>
  <p>Please don't write 'class' of dogs and humans in a formal report because it is incorrect OOP and will cause alarm if your reviewer is a coder. Your code is functional programming rather than OOP. You could say the human/dog fasta object, or just refer to it as human/dog dataframes and avoid any OOP terminology.</p>
   4413: <p>Assuming what I said in the comments is true. Here's my attempt, there may be an easier solution but this should do it:</p>
  <pre><code>import pandas as pd
  import re
  
  # a function to return the count number of animals in the fasta in a dictionary
  def count_animals(file_path, separator, animal_index):
    animal_dictionary={}
    with open(&quot;Proof.txt&quot;) as fasta_file:
      fasta_contents = fasta_file.readlines()
      for line in fasta_contents:
        if line.startswith(&quot;&gt;&quot;):
          animal = line.split(separator)[animal_index]
          if animal not in animal_dictionary:
            animal_dictionary[animal] = 1
          else:
            animal_dictionary[animal] = animal_dictionary[animal] + 1
    return animal_dictionary
  
  def read_fasta(file_path, columns) :
    counter = 0  # counter used to print no more than number of times 'Dog'
    animal_d = count_animals(file_path, '|', 1) # execute function
    max_animal =  max(animal_d.values()) # get max number of animals
    min_animal =  min(animal_d.values()) # get min number of animals
  
    from Bio.SeqIO.FastaIO import SimpleFastaParser 
    with open(&quot;Proof.txt&quot;) as fasta_file :
      records = [] # create empty list
      for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘&gt;’ character), and the sequence (with any whitespace removed). 
        record = []
        title_splits=re.findall(r&quot;[\w']+&quot;, title) # Data cleaning is needed
  
        record.append(title_splits[0])  #First values are ID (Append adds element to a list)
        record.append(len(sequence)) #Second values are sequences lengths
        sequence = &quot; &quot;.join(sequence) #It converts into one line
        record.append(sequence)#Third values are sequences
        # if 'Human' exists in title and the counter is less than min_animal('Dog'), add the record
        if 'Human' in title and counter &lt; min_animal:
          counter += 1 # add +1 to counter
  
          #Fourth column will contain the species
          record.append(&quot;Human&quot;)  
            
          records.append(record)
        elif 'Dog' in title:
          record.append(&quot;Dog&quot;)  
            
          records.append(record)
  
      # print(records)
    return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe
  
  #Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)    
  #And in the second one the names of columns
  data = read_fasta(&quot;Proof.txt&quot;, columns=[&quot;id&quot;,&quot;sequence_length&quot;, &quot;sequence&quot;, &quot;animal&quot;])
  
  print(data)
  </code></pre>
  <p>Should give:</p>
  <pre><code>      id  sequence_length                               sequence animal
  0  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  1  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  2  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  3  Numer               19  H S S F I E I V N I E H V I E H I V K  Human
  4  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  5  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  6  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  7  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog
  </code></pre>
  