title: Frame by frame decode using Media Source Extension
body: <p>I've been digging through the Media Source Extension examples on the internet and haven't quite figured out a way to adapt them to my needs. </p>
  
  <p>I'm looking to take a locally cached MP4/WebM video (w/ 100% keyframes and 1:1 ratio of clusters/atoms to keyframes) and decode/display them non-sequentially (ie. frame 10, 400, 2, 100, etc.) and to be able to render these non-sequential frames on demand at rates from 0-60fps. The simple non-MSE approach using the currentTime property fails due to the latency in setting this property and getting a frame displayed.</p>
  
  <p>I realize this is totally outside normal expectations for video playback, but my application requires this type of non-sequential high speed playback. Ideally I can do this with h264 for GPU acceleration but I realize there could be some platform specific GPU buffers to contend with, though it seems that a zero frame buffer should be possible (see <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=465324" rel="noreferrer">here</a>). I am hoping that MSE can accomplish this non-sequential high framerate low latency playback, but I know I'm asking for a lot.</p>
  
  <p>Questions:</p>
  
  <ol>
  <li><p>Will appendBuffer accept a single WebM cluster / MP4 Atom made up of a single keyframe, and also be able to decode at a high frequency (60fps)? </p></li>
  <li><p>Do you think what I'm trying to do is possible in the browser?</p></li>
  </ol>
  
  <p>Any help, insight, or code suggestions/examples would be much appreciated. </p>
  
  <p>Thanks!</p>
  
  <h2>Update 4/5/16</h2>
  
  <p>I was able to get MSE mostly working with single frame MP4 fragments in Firefox, Edge, and Chrome. However, Chrome seems to be running into the frame buffer issue linked above and I haven't found a way to pre-process a MP4 to invoke this "low delay" mode. Anyone have any clues if it's possible to create such a file with an existing tool like MP4Box?</p>
  
  <p>Firefox and Edge decode/display the individual frames immediately with very little latency, but of course something breaks once I load this video into a Three.js WebGL project (no video output, no errors). I'm ignoring this for now as I'd much rather have things working on Chrome as I'll be targeting Android as well.</p>