I am trying to get the screen space position of each pixel in a texture on a mesh and write it into a texture. I was hoping to use a compute shader to do this:

#pragma kernel CSMain

struct VertexData
    float4 v_point;
    float2 uv;
    float4x4 MVP; 

RWTexture2D<float4> Result;
RWStructuredBuffer<VertexData> vBuf;

void CSMain (uint3 id : SV_DispatchThreadID)
    // one thread per pixel in texture

    //Get pixels position in relation to vertices in UV space, using models UV layout

    //multiply pixels object space position against MVP matrix to bring it into clip space and do some stuff to it

    //finally, output this pixels colour back into the texture, storing screen space related data.

In the above code (lump of comments) I can tell that I am going to run into some problems. The VertexData struct contains mesh data per vertex, but I want to run a thread for each pixel in a texture and then figure out what my pixels position is on a triangle. In a regular shader, this would be interpolated from the vertex shader when using data in the fragment shader, but from my little knowledge of compute shaders, this would require something extra. In the back of my mind I feel like its going to end up being something to do with Barycentric coords, but I just can't put my finger on it!

A picture is worth a thousand words: enter image description here

The desired end result is a texture mapped to the UV coords of a mesh, storing the screen space positions of each pixel on the mesh. If there is a better way of tackling this, please let me know!

  • 1
    $\begingroup$ Vertex attribute interpolation is quite easy, you can simply use each component as the weight for the sum of attributes. I.e. for vertex colours C1, C2 and C3 and barycentric coordinate B, fragment color is F = C1 * B.x + C2 * B.y + C3 * B.z. That is, as long as you either compute the barycentrics in world space, or you perspective-correct them: scratchapixel.com/lessons/3d-basic-rendering/… $\endgroup$ – IneQuation Dec 1 '16 at 13:11
  • $\begingroup$ Why use a compute shader? You could simply render to a render target as usual, grab the data off the GPU and save it to an image file of your choice. While possible to do this on the compute shader, you're essentially forcing yourself to perform much of the work that the GPU already knows how to do. Unless you really want to work with compute shaders, I'd advise using the prior technique I mentioned. $\endgroup$ – cehnehdeh Dec 2 '16 at 2:10

You seem to do things more complicatedly than you need to, as adressed in the comments. What you actually want to do is simple the opposite of the normal render flow. Rather than drawing your mesh in world space and mapping the texture into it, you draw it in texture space and map its vertices onto it and let your graphics hardware worry about interpolation. I'll give the general outline here, as I honestly don't know the actual DirectX terms for the stuff here, but it should be easy to map onto real HLSL if you know your way around DirectX (which I'm sure you do when messing with Compute Shaders).

You basically render into your output texture using an offscreen render target. You then render your mesh using its texture coordinates as positions rather than its model space vertices, which you just pass through to the pixel shader, where you finally transform them into screen space and write out as the pixel's colour:

[Vertex Shader]

PER VERTEX float2 uv;
PER VERTEX float4 v_point;

TO PS float4 modelPos;

void VSMain() {
    modelPos = v_point;

[Pixel Shader]

CONSTANT float4x4 MVP;
CONSTANT float2 viewportSize;

FROM VS float4 modelPos;

void PSMain() {
    float4 clipPos = MVP * modelPos;
    float3 ndcPos = clipPos.xyz / clipPos.w;
    float3 screenPos = float3(viewportSize * (screePos.xy * 0.5 + float2(0.5)), ndcPos.z);
    OUT_COLOR = screenPos;

Feel free to adjust this to the realities of proper DirectX screen-space transformations and matrix multiplication orders, but the general idea should be clear.


Your Answer

