Header Propagation in ConnectSoft Microservice Template

Purpose & Overview

Header Propagation is a mechanism for automatically forwarding HTTP headers from incoming requests to outgoing HTTP client calls. In the ConnectSoft Microservice Template, header propagation ensures that important context information such as trace IDs, correlation IDs, and distributed tracing headers are automatically carried through service-to-service communication, enabling end-to-end request tracking and observability.

Why Header Propagation?

Header propagation offers several critical benefits for microservices:

• Request Correlation: Automatically propagate correlation IDs across service boundaries
• Distributed Tracing: Maintain trace context across multiple services
• Observability: Enable end-to-end request tracking and logging
• Consistency: Ensure headers are consistently propagated without manual intervention
• Transparency: Automatic propagation without requiring changes to business logic
• Security Context: Propagate authentication and authorization context
• Context Preservation: Maintain request context across async boundaries

Header Propagation Philosophy

Header propagation ensures that request context is automatically maintained across service boundaries. This enables distributed tracing, correlation tracking, and observability without requiring developers to manually pass headers through every service call.

Architecture Overview

Header Propagation Flow

Incoming HTTP Request
    ↓
Header Propagation Middleware
    ├── Captures headers from incoming request
    ├── X-TraceId
    ├── X-Correlation-Id
    └── traceparent (W3C Trace Context)
    ↓
Stored in HttpContext.Items
    ↓
Outgoing HttpClient Request
    ├── Header Propagation Message Handler
    ├── Automatically adds propagated headers
    └── Headers forwarded to downstream service
    ↓
Downstream Service
    ├── Receives headers
    ├── Continues trace context
    └── Can propagate further

Components

Component	Purpose	Location
HeaderPropagationExtensions	Configuration and registration	ApplicationModel
HeaderPropagationMiddleware	Captures incoming headers	ASP.NET Core
HeaderPropagationMessageHandler	Adds headers to outgoing requests	ASP.NET Core
HttpClientExtensions	Configures HttpClient with header propagation	ApplicationModel

Configuration

Service Registration

AddMicroserviceHeaderPropagation:

// HeaderPropagationExtensions.cs
internal static IServiceCollection AddMicroserviceHeaderPropagation(
    this IServiceCollection services)
{
    ArgumentNullException.ThrowIfNull(services);

    services.AddHeaderPropagation(options =>
    {
        options.Headers.Add("X-TraceId");
        options.Headers.Add("X-Correlation-Id");
        options.Headers.Add("traceparent");
    });

    return services;
}

Registered Headers: - X-TraceId: Custom trace identifier for request tracking - X-Correlation-Id: Business-level correlation identifier - traceparent: W3C Trace Context header for distributed tracing

Middleware Configuration

UseMicroserviceHeaderPropagation:

// HeaderPropagationExtensions.cs
internal static IApplicationBuilder UseMicroserviceHeaderPropagation(
    this IApplicationBuilder application)
{
    ArgumentNullException.ThrowIfNull(application);

    application.UseHeaderPropagation();

    return application;
}

HttpClient Integration

AddHeaderPropagation to HttpClient:

// HttpClientExtensions.cs
internal static IServiceCollection AddMicroserviceHttpClient(
    this IServiceCollection services)
{
    ArgumentNullException.ThrowIfNull(services);

    // Configure HttpClient defaults
    services.ConfigureHttpClientDefaults(http =>
    {
        if (OptionsExtensions.ServiceDiscoveryOptions.Enabled)
        {
            http.AddServiceDiscovery();
        }
    });

    // Add HttpClient with header propagation
    services.AddHttpClient()
        .AddHeaderPropagation();

    return services;
}

Integration in Program.cs

Service Registration:

// MicroserviceRegistrationExtensions.cs
services.AddMicroserviceHeaderPropagation();
services.AddMicroserviceHttpClient(); // Includes header propagation

Middleware Configuration:

// MicroserviceRegistrationExtensions.cs
// After routing, before endpoints
application.UseRouting();
application.UseMicroserviceRateLimiter();
application.UseMicroserviceLocalization();
application.UseMicroserviceHeaderPropagation(); // Must be after routing

Middleware Order: 1. Forwarded Headers 2. Exception Handling 3. Logging 4. Routing ← Header propagation must be after routing 5. Header Propagation ← Captures headers from current request 6. Endpoints

Propagated Headers

X-TraceId

Purpose: Custom trace identifier for request tracking.

Usage: - Set by correlation/tracing middleware - Automatically propagated to outgoing requests - Used for request correlation in logs

Example:

X-TraceId: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

X-Correlation-Id

Purpose: Business-level correlation identifier.

Usage: - Set by conversation ID middleware - Links related operations across services - Used for business-level request tracking

Example:

X-Correlation-Id: 550e8400-e29b-41d4-a716-446655440000

traceparent

Purpose: W3C Trace Context header for distributed tracing.

Usage: - Set by OpenTelemetry instrumentation - Standard format: 00-{trace-id}-{parent-id}-{trace-flags} - Enables distributed tracing across services

Format:

traceparent: 00-{trace-id}-{parent-id}-{trace-flags}

Example:

traceparent: 00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

Components: - 00: Version (00 = current version) - 4bf92f3577b34da6a3ce929d0e0e4736: Trace ID (32 hex characters) - 00f067aa0ba902b7: Parent ID (16 hex characters) - 01: Trace flags (01 = sampled)

Usage

Automatic Propagation

Standard HttpClient Usage:

public class OrderService
{
    private readonly HttpClient httpClient;

    public OrderService(IHttpClientFactory httpClientFactory)
    {
        this.httpClient = httpClientFactory.CreateClient();
    }

    public async Task<PaymentResult> ProcessPaymentAsync(Order order)
    {
        // Headers are automatically propagated
        var response = await this.httpClient.PostAsJsonAsync(
            "https://payment-service/api/payments",
            new PaymentRequest { OrderId = order.Id });

        return await response.Content.ReadFromJsonAsync<PaymentResult>();
    }
}

Headers are automatically added: - No manual header handling required - Headers from incoming request are automatically included - Works with all HttpClient instances created via IHttpClientFactory

Named HttpClient

Configure Named Client:

// In service registration
services.AddHttpClient("PaymentService", client =>
{
    client.BaseAddress = new Uri("https://payment-service");
    client.DefaultRequestHeaders.Add("X-Api-Version", "v1");
})
.AddHeaderPropagation(); // Ensure header propagation is enabled

Use Named Client:

public class OrderService
{
    private readonly IHttpClientFactory httpClientFactory;

    public OrderService(IHttpClientFactory httpClientFactory)
    {
        this.httpClientFactory = httpClientFactory;
    }

    public async Task<PaymentResult> ProcessPaymentAsync(Order order)
    {
        var client = this.httpClientFactory.CreateClient("PaymentService");

        // Headers are automatically propagated
        var response = await client.PostAsJsonAsync(
            "/api/payments",
            new PaymentRequest { OrderId = order.Id });

        return await response.Content.ReadFromJsonAsync<PaymentResult>();
    }
}

Typed HttpClient

Define Typed Client:

public class PaymentServiceClient
{
    private readonly HttpClient httpClient;

    public PaymentServiceClient(HttpClient httpClient)
    {
        this.httpClient = httpClient;
    }

    public async Task<PaymentResult> ProcessPaymentAsync(PaymentRequest request)
    {
        // Headers are automatically propagated
        var response = await this.httpClient.PostAsJsonAsync(
            "/api/payments",
            request);

        return await response.Content.ReadFromJsonAsync<PaymentResult>();
    }
}

Register Typed Client:

// In service registration
services.AddHttpClient<PaymentServiceClient>(client =>
{
    client.BaseAddress = new Uri("https://payment-service");
})
.AddHeaderPropagation(); // Ensure header propagation is enabled

Manual Header Override

Override Propagated Headers (if needed):

public async Task<PaymentResult> ProcessPaymentAsync(Order order)
{
    var request = new HttpRequestMessage(HttpMethod.Post, "/api/payments")
    {
        Content = JsonContent.Create(new PaymentRequest { OrderId = order.Id })
    };

    // Override correlation ID if needed
    request.Headers.Add("X-Correlation-Id", customCorrelationId);

    // Other propagated headers still included
    var response = await this.httpClient.SendAsync(request);

    return await response.Content.ReadFromJsonAsync<PaymentResult>();
}

Integration with Other Features

Distributed Tracing

OpenTelemetry Integration:

Header propagation works seamlessly with OpenTelemetry:

// OpenTelemetry automatically sets traceparent header
// Header propagation automatically forwards it
services.AddOpenTelemetry()
    .WithTracing(tracingBuilder =>
    {
        tracingBuilder
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation();
    });

// Header propagation forwards traceparent automatically
services.AddHeaderPropagation(options =>
{
    options.Headers.Add("traceparent"); // W3C Trace Context
});

Result: - Trace context automatically propagated - Distributed traces span multiple services - End-to-end request visibility

Correlation IDs

Conversation ID Middleware:

// Conversation ID middleware sets X-Correlation-Id
application.UseConversationId();

// Header propagation forwards it automatically
application.UseMicroserviceHeaderPropagation();

Flow: 1. Request arrives 2. Conversation ID middleware generates/reads correlation ID 3. Sets X-Correlation-Id header 4. Header propagation middleware captures it 5. Outgoing requests automatically include it

Logging

Structured Logging:

public class OrderService
{
    private readonly ILogger<OrderService> logger;
    private readonly HttpClient httpClient;

    public async Task ProcessOrderAsync(Order order)
    {
        // Correlation ID is in headers and logs
        this.logger.LogInformation(
            "Processing order {OrderId}", 
            order.Id);

        // Call downstream service
        var response = await this.httpClient.PostAsJsonAsync(
            "https://payment-service/api/payments",
            new PaymentRequest { OrderId = order.Id });

        // Correlation ID propagated, logs linked
        this.logger.LogInformation(
            "Payment processed for order {OrderId}", 
            order.Id);
    }
}

Log Correlation: - Correlation ID in headers - Correlation ID in logs - Logs linked across services - End-to-end request tracking

Advanced Configuration

Custom Header Configuration

Configure Specific Headers:

services.AddHeaderPropagation(options =>
{
    // Add specific headers
    options.Headers.Add("X-TraceId");
    options.Headers.Add("X-Correlation-Id");
    options.Headers.Add("traceparent");

    // Add custom headers
    options.Headers.Add("X-Request-Id");
    options.Headers.Add("X-User-Id");
    options.Headers.Add("X-Tenant-Id");
});

Header Value Transformation

Transform Header Values:

services.AddHeaderPropagation(options =>
{
    options.Headers.Add("X-Correlation-Id", headerName =>
    {
        // Transform header value if needed
        if (HttpContext.Request.Headers.TryGetValue(headerName, out var value))
        {
            // Add prefix or transform
            return $"req-{value}";
        }
        return null;
    });
});

Conditional Propagation

Conditional Header Propagation:

services.AddHeaderPropagation(options =>
{
    options.Headers.Add("X-Correlation-Id", headerName =>
    {
        // Only propagate if condition is met
        if (HttpContext.Request.Path.StartsWithSegments("/api"))
        {
            return HttpContext.Request.Headers[headerName];
        }
        return null;
    });
});

Multiple Header Propagation Configurations

Different Configurations for Different Clients:

// Default configuration
services.AddHeaderPropagation(options =>
{
    options.Headers.Add("X-Correlation-Id");
    options.Headers.Add("traceparent");
});

// Named client with different headers
services.AddHttpClient("InternalService", client =>
{
    client.BaseAddress = new Uri("https://internal-service");
})
.AddHeaderPropagation(options =>
{
    // Only propagate traceparent for internal services
    options.Headers.Add("traceparent");
});

Best Practices

Do's

1. Use Header Propagation for All HttpClient Calls

   // ✅ GOOD - Header propagation enabled
   services.AddHttpClient()
       .AddHeaderPropagation();
   

2. Configure Header Propagation After Routing

   // ✅ GOOD - Correct middleware order
   application.UseRouting();
   application.UseMicroserviceHeaderPropagation();
   

3. Use Standard Header Names

   // ✅ GOOD - Standard headers
   options.Headers.Add("X-Correlation-Id");
   options.Headers.Add("traceparent");
   

4. Integrate with Distributed Tracing

   // ✅ GOOD - Trace context propagation
   services.AddHeaderPropagation(options =>
   {
       options.Headers.Add("traceparent"); // W3C Trace Context
   });
   

5. Document Custom Headers

   // ✅ GOOD - Document custom headers
   /// <summary>
   /// Propagates X-Custom-Context header for business context.
   /// </summary>
   options.Headers.Add("X-Custom-Context");
   

Don'ts

1. Don't Skip Header Propagation for HttpClient

   // ❌ BAD - Missing header propagation
   services.AddHttpClient();
   // Headers won't be propagated
   
   // ✅ GOOD - Header propagation enabled
   services.AddHttpClient()
       .AddHeaderPropagation();
   

2. Don't Place Middleware Before Routing

   // ❌ BAD - Middleware before routing
   application.UseMicroserviceHeaderPropagation();
   application.UseRouting();
   
   // ✅ GOOD - Middleware after routing
   application.UseRouting();
   application.UseMicroserviceHeaderPropagation();
   

3. Don't Propagate Sensitive Headers

   // ❌ BAD - Propagating sensitive headers
   options.Headers.Add("Authorization");
   options.Headers.Add("Cookie");
   
   // ✅ GOOD - Only propagate safe headers
   options.Headers.Add("X-Correlation-Id");
   options.Headers.Add("traceparent");
   

4. Don't Manually Set Headers That Are Propagated

   // ❌ BAD - Manual header when propagation exists
   request.Headers.Add("X-Correlation-Id", correlationId);
   // Duplicates propagated header
   
   // ✅ GOOD - Let propagation handle it
   // Headers automatically included
   

5. Don't Propagate Headers to External Services

   // ❌ BAD - Propagating internal headers externally
   services.AddHttpClient("ExternalApi", client =>
   {
       client.BaseAddress = new Uri("https://external-api.com");
   })
   .AddHeaderPropagation(); // May expose internal headers
   
   // ✅ GOOD - Configure separately for external services
   services.AddHttpClient("ExternalApi", client =>
   {
       client.BaseAddress = new Uri("https://external-api.com");
   });
   // No header propagation for external services
   

Testing

Unit Testing

Test Header Propagation:

[TestMethod]
public async Task HttpClient_WithHeaderPropagation_IncludesHeaders()
{
    // Arrange
    var services = new ServiceCollection();
    services.AddHeaderPropagation(options =>
    {
        options.Headers.Add("X-Correlation-Id");
    });
    services.AddHttpClient()
        .AddHeaderPropagation();

    var provider = services.BuildServiceProvider();
    var factory = provider.GetRequiredService<IHttpClientFactory>();

    // Create HttpContext with header
    var httpContext = new DefaultHttpContext();
    httpContext.Request.Headers["X-Correlation-Id"] = "test-correlation-id";

    // Act
    var client = factory.CreateClient();
    var request = new HttpRequestMessage(HttpMethod.Get, "https://example.com");

    // Manually invoke header propagation
    var handler = provider.GetRequiredService<HeaderPropagationMessageHandler>();
    // ... test logic

    // Assert
    Assert.IsTrue(request.Headers.Contains("X-Correlation-Id"));
}

Integration Testing

Test with TestServer:

[TestMethod]
public async Task Service_WithHeaderPropagation_PropagatesHeaders()
{
    // Arrange
    var factory = new WebApplicationFactory<Program>()
        .WithWebHostBuilder(builder =>
        {
            builder.ConfigureServices(services =>
            {
                // Services configured with header propagation
            });
        });

    var client = factory.CreateClient();

    // Add header to incoming request
    client.DefaultRequestHeaders.Add("X-Correlation-Id", "test-id");

    // Act
    var response = await client.GetAsync("/api/orders");

    // Assert
    // Verify downstream service received header
    // (Requires mocking or test downstream service)
}

Troubleshooting

Issue: Headers Not Propagated

Symptoms: Headers from incoming requests not present in outgoing HttpClient calls.

Solutions: 1. Verify AddHeaderPropagation() is called in service registration 2. Verify UseHeaderPropagation() is called in middleware pipeline 3. Check middleware order (must be after UseRouting()) 4. Verify AddHeaderPropagation() is called on HttpClient registration 5. Check header names match exactly (case-sensitive) 6. Verify headers exist in incoming request

Issue: Headers Propagated Twice

Symptoms: Headers appear multiple times in outgoing requests.

Solutions: 1. Check if headers are manually added in addition to propagation 2. Verify header propagation is not configured multiple times 3. Check for duplicate AddHeaderPropagation() calls 4. Review HttpClient configuration for duplicate handlers

Issue: Headers Not Available in Middleware

Symptoms: Headers not captured by header propagation middleware.

Solutions: 1. Verify middleware is placed after routing 2. Check if headers are set before middleware runs 3. Verify headers are in request headers (not response headers) 4. Check for middleware order issues

Issue: Headers Missing in Downstream Service

Symptoms: Downstream service doesn't receive propagated headers.

Solutions: 1. Verify header propagation is enabled on HttpClient 2. Check if headers are being filtered or removed 3. Verify downstream service is configured to accept headers 4. Check network/proxy configuration 5. Review HttpClient message handler pipeline

Security Considerations

Header Security

Safe Headers to Propagate: - X-Correlation-Id: Safe for propagation - traceparent: Safe for propagation (W3C standard) - X-TraceId: Safe for propagation - Custom business context headers

Avoid Propagating: - Authorization: Security risk if propagated externally - Cookie: Security and privacy risk - X-API-Key: Security risk - Sensitive authentication headers

External Service Communication

Configure Separately:

// Internal services - propagate headers
services.AddHttpClient("InternalService", client =>
{
    client.BaseAddress = new Uri("https://internal-service");
})
.AddHeaderPropagation();

// External services - no header propagation
services.AddHttpClient("ExternalService", client =>
{
    client.BaseAddress = new Uri("https://external-service.com");
});
// No AddHeaderPropagation() for external services

Performance Considerations

Overhead

Minimal Performance Impact: - Header propagation is lightweight - Middleware overhead is minimal - Message handler overhead is negligible - No significant performance impact

Best Practices: - Only propagate necessary headers - Avoid propagating large header values - Use efficient header matching - Consider caching header values if needed

Summary

Header propagation in the ConnectSoft Microservice Template provides:

• ✅ Automatic Propagation: Headers automatically forwarded without manual code
• ✅ Request Correlation: Correlation IDs maintained across services
• ✅ Distributed Tracing: Trace context propagated automatically
• ✅ Observability: End-to-end request tracking enabled
• ✅ Consistency: Standardized header propagation across all services
• ✅ Integration: Seamless integration with OpenTelemetry and logging
• ✅ Flexibility: Configurable header propagation with custom rules

By following these patterns, teams can:

• Maintain Context: Request context automatically maintained across services
• Enable Tracing: Distributed tracing works end-to-end
• Track Requests: Correlation IDs link logs across services
• Simplify Code: No manual header passing required
• Ensure Consistency: Standardized header propagation
• Improve Observability: Better request tracking and debugging
• Maintain Security: Control which headers are propagated

Header propagation transforms service-to-service communication from manual header management into an automatic, transparent, and observable operation that ensures request context is consistently maintained across the entire microservice ecosystem.