original_docstring: Calculate ANIb for files in input directory.
  
      - infiles - paths to each input file
      - org_lengths - dictionary of input sequence lengths, keyed by sequence
  
      Calculates ANI by the ANIb method, as described in Goris et al. (2007)
      Int J Syst Evol Micr 57: 81-91. doi:10.1099/ijs.0.64483-0. There are
      some minor differences depending on whether BLAST+ or legacy BLAST
      (BLASTALL) methods are used.
  
      All FASTA format files (selected by suffix) in the input directory are
      used to construct BLAST databases, placed in the output directory.
      Each file's contents are also split into sequence fragments of length
      options.fragsize, and the multiple FASTA file that results written to
      the output directory. These are BLASTNed, pairwise, against the
      databases.
  
      The BLAST output is interrogated for all fragment matches that cover
      at least 70% of the query sequence, with at least 30% nucleotide
      identity over the full length of the query sequence. This is an odd
      choice and doesn't correspond to the twilight zone limit as implied by
      Goris et al. We persist with their definition, however.  Only these
      qualifying matches contribute to the total aligned length, and total
      aligned sequence identity used to calculate ANI.
  
      The results are processed to give matrices of aligned sequence length
      (aln_lengths.tab), similarity error counts (sim_errors.tab), ANIs
      (perc_ids.tab), and minimum aligned percentage (perc_aln.tab) of
      each genome, for each pairwise comparison. These are written to the
      output directory in plain text tab-separated format.
      
docstring: Calculate ANIb for files in input directory.
  infiles - paths to each input file
  org_lengths - dictionary of input sequence lengths, keyed by sequence
  
  Calculates ANI by the ANIb method, as described in Goris et al.
  
  All FASTA format files (selected by suffix) in the input directory are
  used to construct BLAST databases, placed in the output directory.
  Each file's contents are also split into sequence fragments of length
  options.fragsize, and the multiple FASTA file that results written to
  the output directory. These are BLASTNed, pairwise, against the
  databases.
  
  The BLAST output is interrogated for all fragment matches that cover
  at least 70% of the query sequence, with at least 30% nucleotide
  identity over the full length of the query sequence. This is an odd
  choice and doesn't correspond to the twilight zone limit as implied by
  Goris et al. We persist with their definition, however.  Only these
  qualifying matches contribute to the total aligned length, and total
  aligned sequence identity used to calculate ANI.
  
  The results are processed to give matrices of aligned sequence length
  (aln_lengths.tab), similarity error counts (sim_errors.tab), ANIs
  (perc_ids.tab), and minimum aligned percentage (perc_aln.tab) of
  each genome, for each pairwise comparison. These are written to the
  output directory in plain text tab-separated format.
docstring_tokens: Calculate ANIb for files in input directory . infiles - paths to each input file org_lengths - dictionary of input sequence lengths keyed by sequence Calculates ANI by the ANIb method as described in Goris et al . All FASTA format files ( selected by suffix ) in the input directory are used to construct BLAST databases placed in the output directory . Each file ' s contents are also split into sequence fragments of length options . fragsize and the multiple FASTA file that results written to the output directory . These are BLASTNed pairwise against the databases . The BLAST output is interrogated for all fragment matches that cover at least 70% of the query sequence with at least 30% nucleotide identity over the full length of the query sequence . This is an odd choice and doesn ' t correspond to the twilight zone limit as implied by Goris et al . We persist with their definition however . Only these qualifying matches contribute to the total aligned length and total aligned sequence identity used to calculate ANI . The results are processed to give matrices of aligned sequence length ( aln_lengths . tab ) similarity error counts ( sim_errors . tab ) ANIs ( perc_ids . tab ) and minimum aligned percentage ( perc_aln . tab ) of each genome for each pairwise comparison . These are written to the output directory in plain text tab - separated format .
code: def unified_anib(infiles, org_lengths):
      """Calculate ANIb for files in input directory.
  
      - infiles - paths to each input file
      - org_lengths - dictionary of input sequence lengths, keyed by sequence
  
      Calculates ANI by the ANIb method, as described in Goris et al. (2007)
      Int J Syst Evol Micr 57: 81-91. doi:10.1099/ijs.0.64483-0. There are
      some minor differences depending on whether BLAST+ or legacy BLAST
      (BLASTALL) methods are used.
  
      All FASTA format files (selected by suffix) in the input directory are
      used to construct BLAST databases, placed in the output directory.
      Each file's contents are also split into sequence fragments of length
      options.fragsize, and the multiple FASTA file that results written to
      the output directory. These are BLASTNed, pairwise, against the
      databases.
  
      The BLAST output is interrogated for all fragment matches that cover
      at least 70% of the query sequence, with at least 30% nucleotide
      identity over the full length of the query sequence. This is an odd
      choice and doesn't correspond to the twilight zone limit as implied by
      Goris et al. We persist with their definition, however.  Only these
      qualifying matches contribute to the total aligned length, and total
      aligned sequence identity used to calculate ANI.
  
      The results are processed to give matrices of aligned sequence length
      (aln_lengths.tab), similarity error counts (sim_errors.tab), ANIs
      (perc_ids.tab), and minimum aligned percentage (perc_aln.tab) of
      each genome, for each pairwise comparison. These are written to the
      output directory in plain text tab-separated format.
      """
      logger.info("Running %s" % args.method)
      # Build BLAST databases and run pairwise BLASTN
      if not args.skip_blastn:
          # Make sequence fragments
          logger.info("Fragmenting input files, and writing to %s" %
                      args.outdirname)
          # Fraglengths does not get reused with BLASTN
          fragfiles, fraglengths = anib.fragment_FASTA_files(infiles,
                                                             args.outdirname,
                                                             args.fragsize)
          # Export fragment lengths as JSON, in case we re-run BLASTALL with
          # --skip_blastn
          if args.method == "ANIblastall":
              with open(os.path.join(args.outdirname,
                                     'fraglengths.json'), 'w') as outfile:
                  json.dump(fraglengths, outfile)
  
          # Which executables are we using?
          if args.method == "ANIblastall":
              format_exe = args.formatdb_exe
              blast_exe = args.blastall_exe
          else:
              format_exe = args.makeblastdb_exe
              blast_exe = args.blastn_exe
  
          # Run BLAST database-building and executables from a jobgraph
          logger.info("Creating job dependency graph")
          jobgraph = anib.make_job_graph(infiles, fragfiles, args.outdirname,
                                         format_exe, blast_exe, args.method)
          if args.scheduler == 'multiprocessing':
              logger.info("Running jobs with multiprocessing")
              logger.info("Running job dependency graph")
              cumval = run_mp.run_dependency_graph(jobgraph, verbose=args.verbose,
                                                   logger=logger)
              if 0 < cumval:
                  logger.warning("At least one BLAST run failed. " +
                                 "%s may fail." % args.method)
              else:
                  logger.info("All multiprocessing jobs complete.")
          else:
              run_sge.run_dependency_graph(jobgraph, verbose=args.verbose,
                                           logger=logger)
              logger.info("Running jobs with SGE")
      else:
          # Import fragment lengths from JSON
          if args.method == "ANIblastall":
              with open(os.path.join(args.outdirname, 'fraglengths.json'),
                        'rU') as infile:
                  fraglengths = json.load(infile)
          else:
              fraglengths = None
          logger.warning("Skipping BLASTN runs (as instructed)!")
  
      # Process pairwise BLASTN output
      logger.info("Processing pairwise %s BLAST output." % args.method)
      try:
          data = anib.process_blast(args.outdirname, org_lengths,
                                    fraglengths=fraglengths, mode=args.method)
      except ZeroDivisionError:
          logger.error("One or more BLAST output files has a problem.")
          if not args.skip_blastn:
              if 0 < cumval:
                  logger.error("This is possibly due to BLASTN run failure, " +
                               "please investigate")
              else:
                  logger.error("This is possibly due to a BLASTN comparison " +
                               "being too distant for use.")
          logger.error(last_exception())
      return data
code_tokens: def unified_anib ( infiles , org_lengths ) : logger . info ( "Running %s" % args . method ) if not args . skip_blastn : logger . info ( "Fragmenting input files, and writing to %s" % args . outdirname ) fragfiles , fraglengths = anib . fragment_FASTA_files ( infiles , args . outdirname , args . fragsize ) if args . method == "ANIblastall" : with open ( os . path . join ( args . outdirname , 'fraglengths.json' ) , 'w' ) as outfile : json . dump ( fraglengths , outfile ) if args . method == "ANIblastall" : format_exe = args . formatdb_exe blast_exe = args . blastall_exe else : format_exe = args . makeblastdb_exe blast_exe = args . blastn_exe logger . info ( "Creating job dependency graph" ) jobgraph = anib . make_job_graph ( infiles , fragfiles , args . outdirname , format_exe , blast_exe , args . method ) if args . scheduler == 'multiprocessing' : logger . info ( "Running jobs with multiprocessing" ) logger . info ( "Running job dependency graph" ) cumval = run_mp . run_dependency_graph ( jobgraph , verbose = args . verbose , logger = logger ) if 0 < cumval : logger . warning ( "At least one BLAST run failed. " + "%s may fail." % args . method ) else : logger . info ( "All multiprocessing jobs complete." ) else : run_sge . run_dependency_graph ( jobgraph , verbose = args . verbose , logger = logger ) logger . info ( "Running jobs with SGE" ) else : if args . method == "ANIblastall" : with open ( os . path . join ( args . outdirname , 'fraglengths.json' ) , 'rU' ) as infile : fraglengths = json . load ( infile ) else : fraglengths = None logger . warning ( "Skipping BLASTN runs (as instructed)!" ) logger . info ( "Processing pairwise %s BLAST output." % args . method ) try : data = anib . process_blast ( args . outdirname , org_lengths , fraglengths = fraglengths , mode = args . method ) except ZeroDivisionError : logger . error ( "One or more BLAST output files has a problem." ) if not args . skip_blastn : if 0 < cumval : logger . error ( "This is possibly due to BLASTN run failure, " + "please investigate" ) else : logger . error ( "This is possibly due to a BLASTN comparison " + "being too distant for use." ) logger . error ( last_exception ( ) ) return data
short_docstring: Calculate ANIb for files in input directory.
short_docstring_tokens: Calculate ANIb for files in input directory .
comment: """Calculate ANIb for files in input directory.
  
      - infiles - paths to each input file
      - org_lengths - dictionary of input sequence lengths, keyed by sequence
  
      Calculates ANI by the ANIb method, as described in Goris et al. (2007)
      Int J Syst Evol Micr 57: 81-91. doi:10.1099/ijs.0.64483-0. There are
      some minor differences depending on whether BLAST+ or legacy BLAST
      (BLASTALL) methods are used.
  
      All FASTA format files (selected by suffix) in the input directory are
      used to construct BLAST databases, placed in the output directory.
      Each file's contents are also split into sequence fragments of length
      options.fragsize, and the multiple FASTA file that results written to
      the output directory. These are BLASTNed, pairwise, against the
      databases.
  
      The BLAST output is interrogated for all fragment matches that cover
      at least 70% of the query sequence, with at least 30% nucleotide
      identity over the full length of the query sequence. This is an odd
      choice and doesn't correspond to the twilight zone limit as implied by
      Goris et al. We persist with their definition, however.  Only these
      qualifying matches contribute to the total aligned length, and total
      aligned sequence identity used to calculate ANI.
  
      The results are processed to give matrices of aligned sequence length
      (aln_lengths.tab), similarity error counts (sim_errors.tab), ANIs
      (perc_ids.tab), and minimum aligned percentage (perc_aln.tab) of
      each genome, for each pairwise comparison. These are written to the
      output directory in plain text tab-separated format.
      """$SEP$# Build BLAST databases and run pairwise BLASTN$SEP$# Make sequence fragments$SEP$# Fraglengths does not get reused with BLASTN$SEP$# Export fragment lengths as JSON, in case we re-run BLASTALL with$SEP$# --skip_blastn$SEP$# Which executables are we using?$SEP$# Run BLAST database-building and executables from a jobgraph$SEP$# Import fragment lengths from JSON$SEP$# Process pairwise BLASTN output