title: Implementing a multiple input filter graph with the Libavfilter library in Android NDK
body: <p>I am trying to use the <code>overlay</code> filter with multiple input sources, for an Android app. Basically, I want to overlay multiple video sources on top of a static image.
  I have looked at the sample that comes with ffmpeg and implemented my code based on that, but things don't seem to be working as expected.</p>
  
  <p>In the ffmpeg filtering sample there seems to be a single video input. I have to handle multiple video inputs and I am not sure that my solution is the correct one. I have tried to find other examples, but looks like this is the only one.</p>
  
  <p>Here is my code:</p>
  
  <pre class="lang-c prettyprint-override"><code>AVFilterContext **inputContexts;
  AVFilterContext *outputContext;
  AVFilterGraph *graph;
  
  int initFilters(AVFrame *bgFrame, int inputCount, AVCodecContext **codecContexts, char *filters)
  {
      int i;
      int returnCode;
      char args[512];
      char name[9];
      AVFilterInOut **graphInputs = NULL;
      AVFilterInOut *graphOutput = NULL;
  
      AVFilter *bufferSrc  = avfilter_get_by_name("buffer");
      AVFilter *bufferSink = avfilter_get_by_name("buffersink");
  
      graph = avfilter_graph_alloc();
      if(graph == NULL)
          return -1;
  
      //allocate inputs
      graphInputs = av_calloc(inputCount + 1, sizeof(AVFilterInOut *));
      for(i = 0; i &lt;= inputCount; i++)
      {
          graphInputs[i] = avfilter_inout_alloc();
          if(graphInputs[i] == NULL)
              return -1;
      }
  
      //allocate input contexts
      inputContexts = av_calloc(inputCount + 1, sizeof(AVFilterContext *));
      //first is the background
      snprintf(args, sizeof(args), "video_size=%dx%d:pix_fmt=%d:time_base=1/1:pixel_aspect=0", bgFrame-&gt;width, bgFrame-&gt;height, bgFrame-&gt;format);
      returnCode = avfilter_graph_create_filter(&amp;inputContexts[0], bufferSrc, "background", args, NULL, graph);
      if(returnCode &lt; 0)
          return returnCode;
      graphInputs[0]-&gt;filter_ctx = inputContexts[0];
      graphInputs[0]-&gt;name = av_strdup("background");
      graphInputs[0]-&gt;next = graphInputs[1];
  
      //allocate the rest
      for(i = 1; i &lt;= inputCount; i++)
      {
          AVCodecContext *codecCtx = codecContexts[i - 1];
          snprintf(args, sizeof(args), "video_size=%dx%d:pix_fmt=%d:time_base=%d/%d:pixel_aspect=%d/%d",
                      codecCtx-&gt;width, codecCtx-&gt;height, codecCtx-&gt;pix_fmt,
                      codecCtx-&gt;time_base.num, codecCtx-&gt;time_base.den,
                      codecCtx-&gt;sample_aspect_ratio.num, codecCtx-&gt;sample_aspect_ratio.den);
          snprintf(name, sizeof(name), "video_%d", i);
  
          returnCode = avfilter_graph_create_filter(&amp;inputContexts[i], bufferSrc, name, args, NULL, graph);
          if(returnCode &lt; 0)
              return returnCode;
  
          graphInputs[i]-&gt;filter_ctx = inputContexts[i];
          graphInputs[i]-&gt;name = av_strdup(name);
          graphInputs[i]-&gt;pad_idx = 0;
          if(i &lt; inputCount)
          {
              graphInputs[i]-&gt;next = graphInputs[i + 1];
          }
          else
          {
              graphInputs[i]-&gt;next = NULL;
          }
      }
  
      //allocate outputs
      graphOutput = avfilter_inout_alloc();   
      returnCode = avfilter_graph_create_filter(&amp;outputContext, bufferSink, "out", NULL, NULL, graph);
      if(returnCode &lt; 0)
          return returnCode;
      graphOutput-&gt;filter_ctx = outputContext;
      graphOutput-&gt;name = av_strdup("out");
      graphOutput-&gt;next = NULL;
      graphOutput-&gt;pad_idx = 0;
  
      returnCode = avfilter_graph_parse_ptr(graph, filters, graphInputs, &amp;graphOutput, NULL);
      if(returnCode &lt; 0)
          return returnCode;
  
      returnCode = avfilter_graph_config(graph, NULL);
          return returnCode;
  
      return 0;
  }
  </code></pre>
  
  <p>The <code>filters</code> argument of the function is passed on to <code>avfilter_graph_parse_ptr</code> and it can looks like this: <code>[background] scale=512x512 [base]; [video_1] scale=256x256 [tmp_1]; [base][tmp_1] overlay=0:0 [out]</code></p>
  
  <p>The call breaks after the call to <code>avfilter_graph_config</code> with the warning:
  <code>Output pad "default" with type video of the filter instance "background" of buffer not connected to any destination</code> and the error <code>Invalid argument</code>.</p>
  
  <p>What is it that I am not doing correctly? </p>
  
  <p>EDIT: The are two issues that I have discovered:</p>
  
  <ol>
  <li><p>Looks like the description of <code>avfilter_graph_parse_ptr</code> is a bit vague. The <code>ouputs</code> parameter represents a list of the current outputs of the graph, in my case that being the <code>graphInputs</code> variable, because these are the outputs from the <code>buffer</code> filter. The <code>inputs</code> parameter represents a list of the current inputs of the graph, in this case this is the <code>graphOutput</code> variable, because it represents the input to the <code>buffersink</code> filter.</p></li>
  <li><p>I did some testing with a <code>scale</code> filter and a single input. It seems that the name of the <code>AVFilterInOut</code> structure required by <code>avfilter_graph_parse_ptr</code> needs to be <code>in</code>. I have tried with different versions: <code>in_1</code>, <code>in_link_1</code>. None of them work and I have not been able to find any documentation related to this.</p></li>
  </ol>
  
  <p>So the issue still remains. How do I implement a filter graph with multiple inputs?</p>