Tracing and httpclient

[noelmcgrath]

We are just getting started with opentelemetry tracing.
Based on this Traceparent Header Propagation if we have an asp.net web api service where we have added opentelemetry tracing and this calls to other asp.net services internally via httpclient.

What happens if our external customers uses asp.net and httpclient to call to our services?

Will tracing be broke now(we wont have traces) due to issue described with httpclient in link?

[vishweshbankwar]

@noelmcgrath - Are you instrumenting your app for both HttpClient and ASP.NET Core?. If you are running in to a specific issue, share the details here.

[noelmcgrath]

The issue is I have 2 ASP.NET applications(say ProjectB and ProjectA).
ProjectB calls to ProjectA via httpclient.
Both services have otel tracing configured e.g.

builder.Services.AddOpenTelemetry()
	.WithTracing(builder =>
	{
		builder.AddSource("ProjectB.WebApi")
		  .SetResourceBuilder(
				ResourceBuilder.CreateDefault()
				  .AddService(serviceName: "ProjectB.WebApi"))
		.AddAspNetCoreInstrumentation()
		.AddHttpClientInstrumentation()
		.AddConsoleExporter();
	});

When this is the case tracing works:

Now when I don't setup tracing in ProjectB, there is no trace in ProjectA(still has tracing setup).
When I set a breakpoint in ProjectAand view Activity.Current it is not null and the TraceId value is the same as ProjectB,

But there is no trace showing up in ConsoleExporter.

So my issue is if we have tracing setup in our service(ProjectA) and an external customer calls to us using an ASP.NET application using httpclient, and the DONT have otel tracing setup then we will not get traces. Surely this is not right?

If I call the service(ProjectA) via curl I get a trace so it seems to be an ASP.NET\httpclient issue

[vishweshbankwar]

@noelmcgrath - Do you always want the trace on your service to be exported? If yes, you can set Sampler to AlwayOn on your side.

opentelemetry-dotnet/examples/AspNetCore/Program.cs

Line 48 in 0820101

.SetSampler(new AlwaysOnSampler())

[noelmcgrath]

thanks @vishweshbankwar
Yes we do always want the traces to be exported. I have added this and can now see we are getting a trace.

I have looked at documentation on samplers and its not clear to me how doing this makes tracing work.

So it is using AlwaysOnSampler which using SamplingDecision.RecordAndSample which is the same as AlwaysOn.
The default sampler is ParentBased(root=AlwaysOn).

Could you please explain for clarity how this differs from default.
Thanks

[vishweshbankwar]

In case of parent-based sampler the sampling decision is based on the sampling decision of parent span. In your case, ProjectB was sending the traceparent ending with -00 i.e. it did not sample in the trace. Following that, ProjectA was following the parent's decision and was not sampling the span.

With AlwaysOn sampler, it does not take the parent's sampling decision in to account and always samples in the activity.

Now, with the case when the traceparent is not received by ProjectA(when tried with curl) and it is using ParentBasedSampler. This is a special case where the sampler follows AlwaysOn sampling behavior. You can read more about it in spec here https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/sdk.md#built-in-samplers

[noelmcgrath]

@vishweshbankwar thanks for explanation.