text: user180: What's the most straightforward way to filter a BAM file for read pairs where one or both of the reads starts with a given sequence pattern? Would a tool that deals with UMIs work for this? Which one would be best?
  
  user71: First option
  My solution uses the following steps:
  
  use picard sortsam to sort the records on query-name (not samtools sort because the order is not the same between java and C ) 
  use jjs (java scripting engine) and the htsjdk library to build a bufferof reads having the same name. If any read starts with a pattern, print the whole buffer. The htsjdk should be already contained in the picard.jar.
  sort back to coordinate order using samtools.
  
  All in one:
  java -jar /path/to/picard.jar SortSam I=in.bam O=/dev/stdout  SO=queryname |\
  jjs -cp /path/to/picard.jar  script.js |\
  samtools sort -T TMP -o out.bam - 
  
  with script.js
  var out=java.lang.System.out;
  var File = Java.type("java.io.File");
  var SamReaderFactory = Java.type("htsjdk.samtools.SamReaderFactory");
  var SamInputResource = Java.type("htsjdk.samtools.SamInputResource");
  var SAMFileWriterFactory = Java.type("htsjdk.samtools.SAMFileWriterFactory");
  var samReader =  SamReaderFactory.makeDefault().open(SamInputResource.of(java.lang.System.in));
  var samWriter =  new SAMFileWriterFactory().makeBAMWriter(samReader.getFileHeader(),true,java.lang.System.out);
  var iter=samReader.iterator();
  var buffer=[];
  for(;;)
          {
          var rec=null;
          if(iter.hasNext()) rec=iter.next();
          if(rec==null || (buffer.length>0 && !rec.getReadName().equals(buffer[0].getReadName())))
              {
              var i=0;
              for(i=0;i< buffer.length;++i)
                  {
                  if(buffer[i].getReadString().match(/^AAAA/)) break;
                  }
              if(i!=buffer.length)
                  {
                  for(i=0;i< buffer.length ;++i)
                      {
                      samWriter.addAlignment(buffer[i]);
                      }
                  }
              if(rec==null) break;
              buffer=[];
              }
          buffer.push(rec);
          }
  
  samReader.close();
  samWriter.close();
  
  A second option is to use samtools view and awk to export the read names:
   samtools view  input.bam | awk -F '\t' '{if($10 ~ /^AAAA/) print $1;}' | sort | uniq > names.txt
  
  and to use Picard FilterSamReads and READ_LIST_FILE to filter the reads.
  UPDATE: 3rd solution:
  I've updated my tool samjdk: there is now a new option --pair to work with query-name-sorted BAM files.
  The command would be:
  java -jar picard.jar SortSam I=S1.bam O=/dev/stdout  SO=queryname |\
  java -jar dist/samjdk.jar --samoutputformat BAM --pair \
       -e 'return records.stream().anyMatch(R->R.getReadString().startsWith("AAAAAAG"));' |\
  samtools sort -T TMP -o out.bam - 
  
  
  user57: For at least one in a pair matching the sequence, if the read names of pairs are identical, some grep magic could do the job:
  
  Print the sequence at the beginning of the line and read the name in the second column
  Find all lines starting with the sequence
  Extract only read names
  
  And then just grep all found reads and save into the BAM file:
  SEQ=<my_sequence>
  samtools view unfiltered.bam | \
     awk '{print $10 "\t" $1}' | \
     grep "^$SEQ" | \
     cut -f 2 > list_of_matching_names
  samtools view unfiltered.bam | \
     grep -f list_of_matching_names | \
     samtools view -b > filtered.bam
  
  Two small notes:
  
  You might want to consider also reverse complementary sequences.
  You also might want to keep header of BAM file (you can extract it at the beginning and add it in the end).
  
  
original_text: 180: <p>What's the most straightforward way to filter a BAM file for read pairs where one or both of the reads starts with a given sequence pattern? Would a tool that deals with UMIs work for this? Which one would be best?</p>
   71: <p><strong>First option</strong></p>
  
  <p>My solution uses the following steps:</p>
  
  <ul>
  <li>use picard sortsam to sort the records on query-name (not samtools sort because the order <a href="https://github.com/samtools/hts-specs/issues/5" rel="nofollow noreferrer">is not the same</a> between java and C ) </li>
  <li>use <a href="https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jjs.html" rel="nofollow noreferrer">jjs</a> (java scripting engine) and the <a href="https://github.com/samtools/htsjdk" rel="nofollow noreferrer">htsjdk library</a> to build a <code>buffer</code>of reads having the same name. If any read starts with a pattern, print the whole <code>buffer</code>. The htsjdk should be already contained in the picard.jar.</li>
  <li>sort back to coordinate order using samtools.</li>
  </ul>
  
  <p>All in one:</p>
  
  <pre><code>java -jar /path/to/picard.jar SortSam I=in.bam O=/dev/stdout  SO=queryname |\
  jjs -cp /path/to/picard.jar  script.js |\
  samtools sort -T TMP -o out.bam - 
  </code></pre>
  
  <p>with script.js</p>
  
  <pre><code>var out=java.lang.System.out;
  var File = Java.type("java.io.File");
  var SamReaderFactory = Java.type("htsjdk.samtools.SamReaderFactory");
  var SamInputResource = Java.type("htsjdk.samtools.SamInputResource");
  var SAMFileWriterFactory = Java.type("htsjdk.samtools.SAMFileWriterFactory");
  var samReader =  SamReaderFactory.makeDefault().open(SamInputResource.of(java.lang.System.in));
  var samWriter =  new SAMFileWriterFactory().makeBAMWriter(samReader.getFileHeader(),true,java.lang.System.out);
  var iter=samReader.iterator();
  var buffer=[];
  for(;;)
          {
          var rec=null;
          if(iter.hasNext()) rec=iter.next();
          if(rec==null || (buffer.length&gt;0 &amp;&amp; !rec.getReadName().equals(buffer[0].getReadName())))
              {
              var i=0;
              for(i=0;i&lt; buffer.length;++i)
                  {
                  if(buffer[i].getReadString().match(/^AAAA/)) break;
                  }
              if(i!=buffer.length)
                  {
                  for(i=0;i&lt; buffer.length ;++i)
                      {
                      samWriter.addAlignment(buffer[i]);
                      }
                  }
              if(rec==null) break;
              buffer=[];
              }
          buffer.push(rec);
          }
  
  samReader.close();
  samWriter.close();
  </code></pre>
  
  <p><strong>A second option</strong> is to use samtools view and awk to export the read names:</p>
  
  <pre><code> samtools view  input.bam | awk -F '\t' '{if($10 ~ /^AAAA/) print $1;}' | sort | uniq &gt; names.txt
  </code></pre>
  
  <p>and to use <a href="http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads" rel="nofollow noreferrer">Picard FilterSamReads</a> and READ_LIST_FILE to filter the reads.</p>
  
  <p><strong>UPDATE: 3rd solution:</strong></p>
  
  <p>I've updated my tool <a href="http://lindenb.github.io/jvarkit/SamJdk.html" rel="nofollow noreferrer">samjdk</a>: there is now a new option <code>--pair</code> to work with query-name-sorted BAM files.</p>
  
  <p>The command would be:</p>
  
  <pre><code>java -jar picard.jar SortSam I=S1.bam O=/dev/stdout  SO=queryname |\
  java -jar dist/samjdk.jar --samoutputformat BAM --pair \
       -e 'return records.stream().anyMatch(R-&gt;R.getReadString().startsWith("AAAAAAG"));' |\
  samtools sort -T TMP -o out.bam - 
  </code></pre>
   57: <p>For at least <strong>one in a pair</strong> matching the sequence, if the read names of pairs are identical, some grep magic could do the job:</p>
  
  <ol>
  <li>Print the sequence at the beginning of the line and read the name in the second column</li>
  <li>Find all lines starting with the sequence</li>
  <li>Extract only read names</li>
  </ol>
  
  <p>And then just grep all found reads and save into the BAM file:</p>
  
  <pre><code>SEQ=&lt;my_sequence&gt;
  samtools view unfiltered.bam | \
     awk '{print $10 "\t" $1}' | \
     grep "^$SEQ" | \
     cut -f 2 &gt; list_of_matching_names
  samtools view unfiltered.bam | \
     grep -f list_of_matching_names | \
     samtools view -b &gt; filtered.bam
  </code></pre>
  
  <p>Two small notes:</p>
  
  <ol>
  <li>You might want to consider also reverse complementary sequences.</li>
  <li>You also might want to keep header of BAM file (you can extract it at the beginning and add it in the end).</li>
  </ol>
  