title: Embree: stream mode - how does gather and scatter work and what are pid and tid?
body: <p>I'm trying to upgrade my application from single ray intersection to stream intersection.</p>
  <p>What I don't quite understand is how it's possible that the <code>gather</code> and <code>scatter</code> <a href="https://github.com/embree/embree/blob/ae029e2ff83bebbbe8742c88aba5b0521aba1a23/tutorials/intersection_filter/intersection_filter_device.cpp#L158" rel="nofollow noreferrer">functions</a> shown in the tutorials are even working</p>
  <p>The example defines a custom extended ray struct <code>Ray2</code></p>
  <pre><code>struct Ray2
  {
    Ray ray;
  
    // ray extensions
    float transparency; //!&lt; accumulated transparency value
  
    // we remember up to 16 hits to ignore duplicate hits
    unsigned int firstHit, lastHit;
    unsigned int hit_geomIDs[HIT_LIST_LENGTH];
    unsigned int hit_primIDs[HIT_LIST_LENGTH];
  };
  </code></pre>
  <p>then it defines an array of these <code>Ray2</code>structs:</p>
  <pre><code>Ray2 primary_stream[TILE_SIZE_X*TILE_SIZE_Y];
  </code></pre>
  <p>this array is set as the userRayExt before calling the intersection method:</p>
  <pre><code>primary_context.userRayExt = &amp;primary_stream;
  rtcIntersect1M(data.g_scene,&amp;primary_context.context,(RTCRayHit*)&amp;primary_stream,N,sizeof(Ray2));
  </code></pre>
  <p>now, for each ray bundle that embree intersects with geometry, the filter callback is invoked:</p>
  <pre><code>/* intersection filter function for streams of general packets */
  void intersectionFilterN(const RTCFilterFunctionNArguments* args)
  {
    int* valid = args-&gt;valid;
    const IntersectContext* context = (const IntersectContext*) args-&gt;context;
    struct RTCRayHitN* rayN = (struct RTCRayHitN*)args-&gt;ray;
    //struct RTCHitN* hitN = args-&gt;hit;
    const unsigned int N = args-&gt;N;
                                    
    /* avoid crashing when debug visualizations are used */
    if (context == nullptr) return;
  
    /* iterate over all rays in ray packet */
    for (unsigned int ui=0; ui&lt;N; ui+=1)
    {
      /* calculate loop and execution mask */
      unsigned int vi = ui+0;
      if (vi&gt;=N) continue;
  
      /* ignore inactive rays */
      if (valid[vi] != -1) continue;
  
      /* read ray/hit from ray structure */
      RTCRayHit rtc_ray = rtcGetRayHitFromRayHitN(rayN,N,ui);
      Ray* ray = (Ray*)&amp;rtc_ray;
  
      /* calculate transparency */
      Vec3fa h = ray-&gt;org + ray-&gt;dir  * ray-&gt;tfar;
      float T = transparencyFunction(h);
  
      /* ignore hit if completely transparent */
      if (T &gt;= 1.0f) 
        valid[vi] = 0;
      /* otherwise accept hit and remember transparency */
      else
      {
        /* decode ray IDs */
        const unsigned int pid = ray-&gt;id / 1;
        const unsigned int rid = ray-&gt;id % 1;
        Ray2* ray2 = (Ray2*) context-&gt;userRayExt;
        assert(ray2);
        scatter(ray2-&gt;transparency,sizeof(Ray2),pid,rid,T);
      }
    }
  }
  </code></pre>
  <p>the last line of this method is what I don't understand</p>
  <pre><code>scatter(ray2-&gt;transparency,sizeof(Ray2),pid,rid,T);
  </code></pre>
  <p>I understand what it is SUPPOSED to do. It should update the transparency property of the <code>Ray2</code> that corresponds to the traced ray with T. But I don't get why/how this works, since the implementation of <code>scatter</code> looks like this:</p>
  <pre><code>inline void scatter(float&amp; ptr, const unsigned int stride, const unsigned int pid, const unsigned int rid, float v) {
    ((float*)(((char*)&amp;ptr) + pid*stride))[rid] = v;
  }
  </code></pre>
  <p>I will reformulate this function a bit to better ask my question (but it should be completely equivalent if I'm not mistaken):</p>
  <pre><code>inline void scatter(float&amp; ptr, const unsigned int stride, const unsigned int pid, const unsigned int rid, float v) {
    float* uptr = ((float*)(((char*)&amp;ptr) + pid*stride));
    uptr[rid] = v;
  }
  </code></pre>
  <p>So, the first line still makes sense for me. A pointer to the transparency field of the first Ray2 struct is constructed and then incremented by <code>tid * sizeof(Ray2)</code> - this makes sense as it will land on another <code>transparency</code> field, since it is incremented by a multiple of <code>sizeof(Ray2)</code></p>
  <p>but then the next line</p>
  <pre><code>uptr[rid] = v;
  </code></pre>
  <p>I don't get at all. <code>uptr</code> is a float pointer, pointing to a transparency field. So unless <code>rid</code> itself is a multiple of <code>sizeof(Ray2)</code>, this won't point to a transparency field of one of the rays at all.</p>
  <p><code>pid</code> and <code>rid</code> are calculated as</p>
  <pre><code>  const unsigned int pid = ray-&gt;id / 1;
    const unsigned int rid = ray-&gt;id % 1;
  </code></pre>
  <p>which I find weird. Isn't that always the same as</p>
  <pre><code>  const unsigned int pid = ray-&gt;id;
    const unsigned int rid = 0;
  </code></pre>
  <p>?</p>
  <p>what are <code>pid</code> and <code>rid</code> and why are they computed like this?</p>