text: user4845: Hi I am trying to use this python script (running with bash script) to do my analysis. I am getting the following error:
  File "my_python_script.py", line 33
  logging.info("Reading input2 file")
  ^
  SyntaxError: invalid syntax
  My Python script is as folllows and albeit I omitted the indentation and my full apologies therein, please keep in mind I'm not the coder in this instance.
  import itertools
  import argparse
  import logging
  import sys
  import numpy as np
  import scipy
  import pandas as pd
  
  def parse_commandline():
  """Parse command line parameters.
  Return type: Object
  """
  parser = argparse.ArgumentParser(description="Association of variants")
  parser.add_argument("--input1", metavar="FILE1.tsv", help="gene data file", required=True)
  parser.add_argument("--input2", metavar="FILE2.tsv", help="mutation data file", required=True)
  parser.add_argument("--output", metavar="FILE.tsv", help="Output file name", required=True)
  if len(sys.argv) == 1:
      parser.print_help()
      sys.exit(0)
  return parser.parse_args()
  
  def main():
  params = parse_commandline()
  
  #load genedata
  logging.info("Reading input1 file")
  try:
  gene_data = pd.read_csv(params.input1, sep="\t", index_col ="Model")
  sys.exit(0)
      
  #load mutation data
  logging.info("Reading input2 file")
  try:
  mutation_data = pd.read_csv(params.input2, sep="\t", 
  index_col="Mutation")
  sys.exit(0)
  
  ##All mutations and gene in data files
  all_mutations = list(mutation_data.index)
  all_genes = list(gene_data.columns)
  cell line Models in mutation_data file
  models_mutations = list(mutation_data.columns)
  
  ## models in gene_data file
  models_genes = list(gene_data.index)
  intersection_set = set.intersection(set(models_genes), 
  set(models_mutations))
  models_common = list(intersection_set)
  unique_series_gene = gene_data.index.isin(models_common)
  end_gene_table = gene_data[unique_series_gene]
  transposed_mutation_data = mutation_data.transpose()
  unique_series_mutation = 
  transposed_mutation_data.index.isin(models_common)
  end_mutation_table = transposed_mutation_data[unique_series_gene]
  
  ##calcuate p-values
  all_p_values = []
  genes = []
  mutations = []
  for gene, mutation in itertools.product(all_genes, all_mutations):
  gene_table_to_test = np.array(end_gene_table[gene])
  mutation_table_to_test = np.array(end_mutation_table[mutation])    
  p_value = scipy.stats.wilcoxon(gene_table_to_test, 
  mutation_table_to_test)[1]
  genes.append(gene)
  mutations.append(mutation)
  all_p_values.append(p_value)
  
  ## calculating adjusted_pvalues
  adjusted_pvalues = p_adjust_bh(all_p_values)
  d = {'Gene': genes, 'Mutation': mutations, 'pvalue': 
  all_p_values,'p_adj':adjusted_pvalues}
  df = pd.DataFrame(data=d)
  
  # Print output file
  logging.info("Writing into file")
  df.to_csv(params.output, sep="\t")
  
  if __name__ == "__main__":
  main()
  
  My Bash script
  #!/bin/sh
  
  ### This script will run python script
  
  python3 my_python_script.py --input1 Gene.tsv --input2 
  Mutation.tsv --output tmp
  diff tmp adjusted_pvalues.tsv
  rm tmp
  
  
  user3848: It is certainly possible the 'error' is the input file 2.
  However, both try: routines are weird and if this is the real code it would fall over. Something isn't quite right however, which I'll touch on at the end.
  What the code is supposed to say is:
  try:
      load stuff
  except:
      raise Exception ('Input file 2 is has deviation from csv format and will not load')
  
  raise Exception isn't needed, the following would be a more harmonised approach:
  try:
      load stuff
  except:
      print ("Input file 2 is has deviation from csv format and will not load')
  finally:
      sys.exit('Abandon ship!')
  
  The idea is that if the CSV file didn't load, it would stop the code and tell you approximately where to start looking. You might then ask, can it actually sees the file and then peer into the loaded CSV. If it loads the code continues.
  You could just stay:
  try:
      load stuff
  except:
      sys.exit('Oh the file did not to load')
  
  This isn't cool, but who am I (who sticks Exceptions in except statements), but it will work and it could be that is the actual code.
  What the code actually says is
  try:
      load stuff
      sys.exit()
  
  or
  try: 
      load stuff
  sys.exit()
  
  Same outcome the code stop because sys.exit stops everything ... nothing is left. What the code should really say is not just perform a try: except: but perform a 'getter' and 'setter' on the input file to check the CSV format, this is because your the user of the code not the developer and the coder has to put safe guards in because stuff happens.
  sys.exit is really used in development, for example in debugging.
  However, somehow you got past the first try:, sys.exit and I don't know how you did it, because that should not have been possible.  I would carefully check the original code.
  Just to point out as @ChrisR mentioned, if this was your actual python code it would fall over because you've omitted all the identation, which in python is essential. Its not like shell where indentitation matters not a jot.
  Finally, as @supertech has mentioned, there are numerous line breaks in the code and whether you are in shell or python this can cause problems. I think the line break(s) are within an extra layer of identation, which Python can read through (I never understood how).
  Personally, I'd go back to the coder and point out the try: sys.exit() and just point out any beginners webpage on try: except:, this is not advanced coding. If anything sys.exit() risks weird behaviour for a user particularly as no distinguishing information is provided.
  
original_text: 4845: <p>Hi I am trying to use this python script (running with bash script) to do my analysis. I am getting the following error:</p>
  <p>File &quot;my_python_script.py&quot;, line 33
  logging.info(&quot;Reading input2 file&quot;)
  ^
  SyntaxError: invalid syntax</p>
  <p>My Python script is as folllows and albeit I omitted the indentation and my full apologies therein, please keep in mind I'm not the coder in this instance.</p>
  <pre><code>import itertools
  import argparse
  import logging
  import sys
  import numpy as np
  import scipy
  import pandas as pd
  
  def parse_commandline():
  &quot;&quot;&quot;Parse command line parameters.
  Return type: Object
  &quot;&quot;&quot;
  parser = argparse.ArgumentParser(description=&quot;Association of variants&quot;)
  parser.add_argument(&quot;--input1&quot;, metavar=&quot;FILE1.tsv&quot;, help=&quot;gene data file&quot;, required=True)
  parser.add_argument(&quot;--input2&quot;, metavar=&quot;FILE2.tsv&quot;, help=&quot;mutation data file&quot;, required=True)
  parser.add_argument(&quot;--output&quot;, metavar=&quot;FILE.tsv&quot;, help=&quot;Output file name&quot;, required=True)
  if len(sys.argv) == 1:
      parser.print_help()
      sys.exit(0)
  return parser.parse_args()
  
  def main():
  params = parse_commandline()
  
  #load genedata
  logging.info(&quot;Reading input1 file&quot;)
  try:
  gene_data = pd.read_csv(params.input1, sep=&quot;\t&quot;, index_col =&quot;Model&quot;)
  sys.exit(0)
      
  #load mutation data
  logging.info(&quot;Reading input2 file&quot;)
  try:
  mutation_data = pd.read_csv(params.input2, sep=&quot;\t&quot;, 
  index_col=&quot;Mutation&quot;)
  sys.exit(0)
  
  ##All mutations and gene in data files
  all_mutations = list(mutation_data.index)
  all_genes = list(gene_data.columns)
  cell line Models in mutation_data file
  models_mutations = list(mutation_data.columns)
  
  ## models in gene_data file
  models_genes = list(gene_data.index)
  intersection_set = set.intersection(set(models_genes), 
  set(models_mutations))
  models_common = list(intersection_set)
  unique_series_gene = gene_data.index.isin(models_common)
  end_gene_table = gene_data[unique_series_gene]
  transposed_mutation_data = mutation_data.transpose()
  unique_series_mutation = 
  transposed_mutation_data.index.isin(models_common)
  end_mutation_table = transposed_mutation_data[unique_series_gene]
  
  ##calcuate p-values
  all_p_values = []
  genes = []
  mutations = []
  for gene, mutation in itertools.product(all_genes, all_mutations):
  gene_table_to_test = np.array(end_gene_table[gene])
  mutation_table_to_test = np.array(end_mutation_table[mutation])    
  p_value = scipy.stats.wilcoxon(gene_table_to_test, 
  mutation_table_to_test)[1]
  genes.append(gene)
  mutations.append(mutation)
  all_p_values.append(p_value)
  
  ## calculating adjusted_pvalues
  adjusted_pvalues = p_adjust_bh(all_p_values)
  d = {'Gene': genes, 'Mutation': mutations, 'pvalue': 
  all_p_values,'p_adj':adjusted_pvalues}
  df = pd.DataFrame(data=d)
  
  # Print output file
  logging.info(&quot;Writing into file&quot;)
  df.to_csv(params.output, sep=&quot;\t&quot;)
  
  if __name__ == &quot;__main__&quot;:
  main()
  </code></pre>
  <p>My Bash script</p>
  <pre><code>#!/bin/sh
  
  ### This script will run python script
  
  python3 my_python_script.py --input1 Gene.tsv --input2 
  Mutation.tsv --output tmp
  diff tmp adjusted_pvalues.tsv
  rm tmp
  </code></pre>
   3848: <p>It is certainly possible the 'error' is the input file 2.</p>
  <p>However, both <code>try:</code> routines are weird and if this is the real code it would fall over. Something isn't quite right however, which I'll touch on at the end.</p>
  <p>What the code is supposed to say is:</p>
  <pre><code>try:
      load stuff
  except:
      raise Exception ('Input file 2 is has deviation from csv format and will not load')
  </code></pre>
  <p><code>raise Exception</code> isn't needed, the following would be a more harmonised approach:</p>
  <pre><code>try:
      load stuff
  except:
      print (&quot;Input file 2 is has deviation from csv format and will not load')
  finally:
      sys.exit('Abandon ship!')
  </code></pre>
  <p>The idea is that if the CSV file didn't load, it would stop the code and tell you approximately where to start looking. You might then ask, can it actually sees the file and then peer into the loaded CSV. If it loads the code continues.</p>
  <p>You could just stay:</p>
  <pre><code>try:
      load stuff
  except:
      sys.exit('Oh the file did not to load')
  </code></pre>
  <p>This isn't cool, but who am I (who sticks Exceptions in except statements), but it will work and it could be that is the actual code.</p>
  <p>What the code actually says is</p>
  <pre><code>try:
      load stuff
      sys.exit()
  </code></pre>
  <p>or</p>
  <pre><code>try: 
      load stuff
  sys.exit()
  </code></pre>
  <p>Same outcome the code stop because sys.exit stops everything ... nothing is left. What the code should really say is not just perform a try: except: but perform a 'getter' and 'setter' on the input file to check the CSV format, this is because your the user of the code not the developer and the coder has to put safe guards in because stuff happens.</p>
  <p><code>sys.exit</code> is really used in development, for example in debugging.</p>
  <p>However, somehow you got past the first <code>try:, sys.exit</code> and I don't know how you did it, because that should not have been possible.  I would carefully check the original code.</p>
  <p>Just to point out as @ChrisR mentioned, if this was your actual python code it would fall over because you've omitted all the identation, which in python is essential. Its not like shell where indentitation matters not a jot.</p>
  <p>Finally, as @supertech has mentioned, there are numerous line breaks in the code and whether you are in shell or python this can cause problems. I think the line break(s) are within an extra layer of identation, which Python can read through (I never understood how).</p>
  <p>Personally, I'd go back to the coder and point out the try: sys.exit() and just point out any beginners webpage on <code>try: except:</code>, this is not advanced coding. If anything sys.exit() risks weird behaviour for a user particularly as no distinguishing information is provided.</p>
  