Tracing custom rays


REDsdk uses ray-tracing to render some lighting effects. But ray-tracing is also a powerful tool to compute additional information like occlusion or collision. In this tutorial we'll use the REDsdk ray-tracing interface to fire custom rays in order to estimate the ambient occlusion during shading.


Ambient occlusion is an approximation in each point of space of how surrounding sky is occluded by scene geometries. It gives a measure of how much global light reaches a point and is useful to gives a feeling of global illumination only at a fraction of the computational cost of the full global illumination solution.

Basically, ambient occlusion is estimated by firing many rays in the hemisphere centred on the point of interest and oriented along the normal vector to that point.

Black rays are ambient occlusion rays; two of them hit scene objects (in pink) and are considered as occluded.

The ratio of the un-occluded rays over the total number of rays gives the ambient occlusion measure. The higher the number of rays, the better the ambient occlusion estimate (the less noise you get).

Ambient occlusion (AO for short), is implemented in a REDsdk custom software shader using the RED::IGeometryDatabase service to fire custom rays. You can access to the service using RED::IViewpointRenderList::GetViewpointGeometryDatabase :

RED::IWindow* iwindow = app::GetWindow()->As< RED::IWindow >();
RED::Object* vrl;
RC_TEST( iwindow->GetDefaultVRL( vrl ) );
RED::IViewpointRenderList* ivrl = vrl->As< RED::IViewpointRenderList >();

RED::Object* geodb;
RC_TEST( ivrl->GetViewpointGeometryDatabase( geodb, app::GetViewpoint() ) );
RED::IGeometryDatabase* igeo = geodb->As< RED::IGeometryDatabase >();

The shader is called for every shaded points in the scene. A user-defined number of rays is traced from the shaded point towards random position inside the hemisphere around that point:

// The ray origin is the hit point slightly moved along the normal to avoid
// self-intersections (not robust but sufficient here).
ray_start[0] = hit[0] + 1e-2f * nrm[0];
ray_start[1] = hit[1] + 1e-2f * nrm[1];
ray_start[2] = hit[2] + 1e-2f * nrm[2];

// Build an orthogonal basis around the hit point normal. The ray directions
// are computed in that basis.
RED::Vector3 n( nrm[0], nrm[1], nrm[2] ), u, v;
n.OrthoBasis( u, v );

// Trace the requested number of samples to estimate the ambient occlusion
// at the hit point.
double theta_sin, theta_cos, phi;
double dx, dy, dz;

for( int s = 0; s < samples_count; ++s )
  // Get a pair of pseudo-random numbers.
  Get2dRandom( vec2 );

  // Pick a sampling direction around the hit point.
 theta_sin = sqrt( vec2[0] );
 theta_cos = sqrt( REDMax( 0.0, 1.0 - vec2[0] ) );
 phi = RED_2PI * vec2[1];

  // Compute the direction of sampling from pseudo-random angle values (in spherical coordinates system). 
  dx = max_dist * cos( phi ) * theta_sin;
  dy = max_dist * sin( phi ) * theta_sin;
  dz = max_dist * theta_cos;

  ray_end[0] = ray_start[0];
  ray_end[1] = ray_start[1];
  ray_end[2] = ray_start[2];

  ray_end[0] += dx * u._x; ray_end[1] += dx * u._y; ray_end[2] += dx * u._z; 
  ray_end[0] += dy * v._x; ray_end[1] += dy * v._y; ray_end[2] += dy * v._z; 
  ray_end[0] += dz * n._x; ray_end[1] += dz * n._y; ray_end[2] += dz * n._z; 

  // Trace a ray along the sampling direction and checks for an intersection.
  if( igeo->GetIntersection( tid, thit, ray_start, ray_end, 0, 0, 0.f, false, false, iRayContext.GetThreadID() ) == false )


The rays are chosen randomly using the Halton pseudo-random sequence with a cosine-weighted sampling distribution. The maximum length of each ray is determined by the user to reduce the noise by ignoring objects which are too distant.

The resulting AO estimate is then:

// The output color reflects the amount of unoccluded rays.
ioFrameBufferSample._out_color[0] = 0.85 * (double)unoccluded_rays / rays_count;
ioFrameBufferSample._out_color[1] = 0.85 * (double)unoccluded_rays / rays_count;
ioFrameBufferSample._out_color[2] = 0.85 * (double)unoccluded_rays / rays_count;
ioFrameBufferSample._out_color[3] = 1.0;

The AO computed using the described shader.