title: How to use Vulkan Timestamp Queries?
body: <p>This is the simplified pseudocode where I'm trying to measure a GPU workload:</p>
  <pre><code>for(N) vkCmdDrawIndexed();
  
  vkCmdWriteTimestamp(VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT);
  vkCmdWriteTimestamp(VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT);
  
  submit();
  vkDeviceWaitIdle();
  vkGetQueryPoolResults();
  </code></pre>
  <p>Things to note:</p>
  <ul>
  <li>N is 224 in my case</li>
  <li>I have to wait for an idle device - without it, I continue to receive a validation error saying me that the data is not ready though I have multiple query pools in flight</li>
  <li>putting the first timestamp I expect that the query value will be written as soon as all previous commands reached a preprocessing step. I was pretty sure that all 224 commands are preprocessed almost at the same time but the reality shows that this is not true.</li>
  <li>putting the second timestamp I expect that the query value will be written after all previous commands are finished. I.e. the time difference between these two query values should give me the time it takes for the GPU to do all the work for a single frame.</li>
  <li>I'm taking into account <code>VkPhysicalDeviceLimits::timestampPeriod</code> (1 on my machine) and <code>VkQueueFamilyProperties::timestampValidBits</code> (64 on my machine)</li>
  </ul>
  <p>I created a big dataset that visually takes approx 2 seconds (~2000ms) to render a single frame. But the calculated time has only 2 (two) different values - either 0.001024ms or 0.002048ms so the frame by frame output can look like this:</p>
  <pre><code>0.001024ms
  0.001024ms
  0.002048ms
  0.001024ms
  0.002048ms
  0.002048ms
  ...
  </code></pre>
  <p>Don't know how about you, but I find these values <strong>VERY</strong> suspicious. I have no answer for that. Maybe at the time, the last draw command reaches the command processor all the work is already done, but why the hell 1024 and 2048??</p>
  <p>I tried to modify the code and move the first timestamp above, i.e.:</p>
  <pre><code>vkCmdWriteTimestamp(VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT);
  
  for(N) vkCmdDrawIndexed();
      
  vkCmdWriteTimestamp(VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT);
  </code></pre>
  <p>Now at the time the preprocessor hits the timestamp command, it writes the query value immediately, because there was no previous work and nothing to wait (remember idle device). This time I have another, closer to the truth values:</p>
  <pre><code>20.9336ms
  20.9736ms
  21.036ms
  21.0196ms
  20.9572ms
  21.3586ms
  ...
  </code></pre>
  <p>which is better but still far beyond expected ~2000ms.</p>
  <p>What's going on, what's happening inside the device when I set timestamps, how to get correct values?</p>