title: Texture rendering on iOS using OpenGL ES in Unity project
body: <p>I'm working on a project, part of which is to stream video to my iPhone. I use my laptop to create the video stream to my iPhone with ffmpeg.</p>
  
  <p>The stream code in shell is below:</p>
  
  <pre><code>ffmpeg \
      -f avfoundation -i "1" -s 1280*720 -r 29.97 \
      -c:v mpeg2video -q:v 20 -pix_fmt yuv420p -g 1 -threads 4\
      -f mpegts udp://192.168.1.102:6666
  </code></pre>
  
  <p>with this, I successfully create my video stream.</p>
  
  <p>In Unity, I want to decode the video stream to create a texture. After I have gone through some ffmpeg tutorial and Unity tutorial, I created my link library. Some of these codes are below (ask me if more is needed):</p>
  
  <p><strong>In my library:</strong></p>
  
  <p>buffer alloc:</p>
  
  <pre><code>uint8_t *buffer;
  int buffer_size;
  buffer_size = avpicture_get_size(AV_PIX_FMT_RGBA, VIEW_WIDTH, VIEW_HEIGHT);
  
  buffer = (uint8_t *) av_malloc(buffer_size*sizeof(uint8_t));
  
  avpicture_fill((AVPicture *) pFrameRGB, buffer, AV_PIX_FMT_RGBA,
                 VIEW_WIDTH, VIEW_HEIGHT);
  </code></pre>
  
  <p>getContext:</p>
  
  <pre><code>    is-&gt;sws_ctx = sws_getContext
      (
       is-&gt;video_st-&gt;codec-&gt;width,
       is-&gt;video_st-&gt;codec-&gt;height,
       is-&gt;video_st-&gt;codec-&gt;pix_fmt,
       VIEW_WIDTH,
       VIEW_HEIGHT,
       AV_PIX_FMT_RGBA,
       SWS_BILINEAR,
       NULL,
       NULL,
       NULL
       );
  </code></pre>
  
  <p>sws_scale:</p>
  
  <pre><code>sws_scale(
            is-&gt;sws_ctx,
            (uint8_t const * const *)pFrame-&gt;data,
            pFrame-&gt;linesize,
            0,
            is-&gt;video_st-&gt;codec-&gt;height,
            pFrameRGB-&gt;data,
            pFrameRGB-&gt;linesize
            );
  </code></pre>
  
  <p>texture render:</p>
  
  <pre><code>static void UNITY_INTERFACE_API OnRenderEvent(int texID)
  {
      GLuint gltex = (GLuint)(size_t)(texID);
  
      glBindTexture(GL_TEXTURE_2D, gltex);
  
      glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, VIEW_WIDTH, VIEW_HEIGHT,
                      GL_RGBA, GL_UNSIGNED_BYTE, pFrameRGB-&gt;data[0]);
  
      glGetError();
      return;
  }
  
  extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API GetRenderEventFunc()
  {
      return OnRenderEvent;
  }
  </code></pre>
  
  <p><strong>In Unity:</strong></p>
  
  <p>texture created:</p>
  
  <pre><code>    private Texture2D texture;
      private int texID;
      texture = new Texture2D (width, height, TextureFormat.RGBA32, false);
      texture.filterMode = FilterMode.Point;
      texture.Apply ();
      GetComponent&lt;Renderer&gt; ().material.mainTexture = texture;
      texID = texture.GetNativeTexturePtr ().ToInt32();
  </code></pre>
  
  <p>update func:</p>
  
  <pre><code>    void Update ()
      {
          GL.IssuePluginEvent(GetRenderEventFunc(), texID);
      }
  </code></pre>
  
  <p><strong>Video stream info:</strong></p>
  
  <pre><code>Input #0, mpegts, from 'udp://0.0.0.0:6666':
    Duration: N/A, start: 2.534467, bitrate: N/A
    Program 1 
      Metadata:
        service_name    : Service01
        service_provider: FFmpeg
      Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
  </code></pre>
  
  <p>Leave other details, my library works fine on the Unity simulator, but when I compiled all my libraries for arm64 and used the xcode project that Unity created to build my app and ran it, I couldn't get any texture rendered in my iPhone, I checked my network and I'm sure that data had been sent to my iPhone and the Debug log showed me that frame has been successfully decoded also the <code>OnRenderEvent</code> function had been called.</p>
  
  <p><strong>FYI:</strong></p>
  
  <p>Unity 5.3.2f1 Personal</p>
  
  <p>Xcode 7.2.1</p>
  
  <p>iOS   9.2.1</p>
  
  <p>ffmpeg 3.0</p>