<p>I see istio is adding <code>x-b3-traceid</code>, <code>x-b3-spanid</code> and other headers to the incoming request when tracing is enabled. But none of them are returned to the caller.</p>
<p>I am able to capture the <code>x-b3-traceid</code> in the log and can find it out in Tempo/Grafana.
I can see the <code>traceid</code> at the istio envoy proxy (sidecar), I am able to access the header using <code>EnvoyFilter</code>.</p>
<p>Can someone let me know where it is filtered?</p>

<p><strong>TL;DR</strong>
these are the headers so that <code>jaeger</code> or <code>zipkin</code> can track individual requests. This application is responsible for their proper propagation. Additionally, they are <strong>request headers</strong> and <strong>not response header</strong>, so everything works fine.</p>
<hr />
<p>Explanation:</p>
<p>It looks okay. First, let's start with <a href="https://docs.oracle.com/cd/F17999_01/docs.10/SCP/GUID-3DD925E4-A4C9-49F1-A6D8-323DB637056F.htm" rel="nofollow noreferrer">what these requests are</a>:</p>
<pre><code>
Field Name      Request/ Response Type          Description
x-request-id    request The x-request-idheader is used by Envoy to uniquely identify a request as well as perform stable access logging and tracing
x-b3-traceid    request The x-b3-traceidHTTP header is used by the Zipkin tracer in Envoy. The TraceId is 64-bit in length and indicates the overall ID of the trace. Every span in a trace shares this ID
x-b3-spanid     request The x-b3-spanidHTTP header is used by the Zipkin tracer in Envoy. The SpanId is 64-bit in length and indicates the position of the current operation in the trace tree
x-b3-sampled    request The x-b3-sampledHTTP header is used by the Zipkin tracer in Envoy. When the Sampled flag is either not specified or set to 1, the span will be reported to the tracing system
</code></pre>
<p>On the github you can find the question: <a href="https://github.com/Nike-Inc/riposte/issues/54" rel="nofollow noreferrer">What is the usage of X-B3-TraceId, traceId, parentSpanId and spanId ?</a>:</p>
<blockquote>
<p>http headers contain only a small amount of data: IDs and sampling flags these go synchronously with your application requests, and allow the other side to continue the trace. <a href="https://github.com/openzipkin/b3-propagation#overall-process" rel="nofollow noreferrer">https://github.com/openzipkin/b3-propagation#overall-process</a> If zipkin is enabled, details including these IDs and other data like duration send asynchronously to zipkin after a request completes. you can see a diagram about that here under &quot;Example Flow&quot; <a href="http://zipkin.io/pages/architecture.html" rel="nofollow noreferrer">http://zipkin.io/pages/architecture.html</a></p>
</blockquote>
<blockquote>
<blockquote>
<p>X-B3-TraceId is same or different from every call of the same client? different per overall request. each top-level request into your system has</p>
</blockquote>
<p>a different trace id. Each step in that request has a different span id</p>
</blockquote>
<blockquote>
<blockquote>
<p>X-B3-SpanId is not send back to the caller, then how could i set the parent(which show be the X-B3-SpanId of the the present call) of the next call? Here is a response shows the absent of X-B3-SpanId in the header： I don't quite understand. The parent is only used when creating a span.</p>
</blockquote>
<p><strong>The span is created before a request is sent. So, there's no relevance to response headers in span creation.</strong></p>
</blockquote>
<p>In <a href="https://istio.io/latest/docs/tasks/observability/distributed-tracing/overview/" rel="nofollow noreferrer">this doc</a> you can find information about headers from istio site:</p>
<blockquote>
<p>Distributed tracing enables users to track a request through mesh that is distributed across multiple services. This allows a deeper understanding about request latency, serialization and parallelism via visualization.</p>
<p>Istio leverages Envoy’s distributed tracing feature to provide tracing integration out of the box. Specifically, Istio provides options to install various tracing backend and configure proxies to send trace spans to them automatically. See Zipkin, Jaeger and Lightstep task docs about how Istio works with those tracing systems.</p>
</blockquote>
<p>If you want to full understand how it works you should read <a href="https://www.envoyproxy.io/docs/envoy/v1.12.0/intro/arch_overview/observability/tracing" rel="nofollow noreferrer">this envoyproxy documentation</a>:</p>
<blockquote>
<p>Distributed tracing allows developers to obtain visualizations of call flows in large service oriented architectures. It can be invaluable in understanding serialization, parallelism, and sources of latency. Envoy supports three features related to system wide tracing:</p>
<ul>
<li><p><strong>Request ID generation</strong>: Envoy will generate UUIDs when needed and populate the  <a href="https://www.envoyproxy.io/docs/envoy/v1.12.0/configuration/http/http_conn_man/headers#config-http-conn-man-headers-x-request-id" rel="nofollow noreferrer">x-request-id</a>  HTTP header. Applications can forward the x-request-id header for unified logging as well as tracing.</p>
</li>
<li><p><strong>Client trace ID joining</strong>: The  <a href="https://www.envoyproxy.io/docs/envoy/v1.12.0/configuration/http/http_conn_man/headers#config-http-conn-man-headers-x-client-trace-id" rel="nofollow noreferrer">x-client-trace-id</a>  header can be used to join untrusted request IDs to the trusted internal  <a href="https://www.envoyproxy.io/docs/envoy/v1.12.0/configuration/http/http_conn_man/headers#config-http-conn-man-headers-x-request-id" rel="nofollow noreferrer">x-request-id</a>.</p>
</li>
<li><p><strong>External trace service integration</strong>: Envoy supports pluggable external trace visualization providers, that are divided into two subgroups:</p>
</li>
<li><p>External tracers which are part of the Envoy code base, like  <a href="https://lightstep.com/" rel="nofollow noreferrer">LightStep</a>,  <a href="https://zipkin.io/" rel="nofollow noreferrer">Zipkin</a>  or any Zipkin compatible backends (e.g.  <a href="https://github.com/jaegertracing/" rel="nofollow noreferrer">Jaeger</a>), and  <a href="https://datadoghq.com/" rel="nofollow noreferrer">Datadog</a>.</p>
</li>
<li><p>External tracers which come as a third party plugin, like  <a href="https://www.instana.com/blog/monitoring-envoy-proxy-microservices/" rel="nofollow noreferrer">Instana</a>.</p>
</li>
</ul>
</blockquote>
<p>Answering your question:</p>
<blockquote>
<p>Can someone let me know where it is filtered?</p>
</blockquote>
<p>They do this by default. It is the application (zipkin, jaeger) that is responsible for acting on these headers. Additionally, they are <strong>request headers</strong> and <strong>not response header</strong>, so everything works fine.</p>
