Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use uriTemplate attribute in metrics #1422

Merged
merged 13 commits into from
Nov 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -506,6 +506,12 @@ set the value of the `spring.cloud.loadbalancer.stats.micrometer.enabled` to `tr

Additional information regarding the service instances, request data, and response data is added to metrics via tags whenever available.

NOTE: For `WebClient` and `RestClient`-backed load-balancing, we use `uriTemplate` for the `uri` tag whenever available.

TIP: It is possible to disable adding `path` to `uri` tag by setting `spring.cloud.loadbalancer.stats.include-path` to `false`.

WARNING: As with `RestTemplate`-backed load-balancing, we don't have access to `uriTemplate`, full path is always used in the `uri` tag. In order to avoid high cardinality issues, if path is a high cardinality value (for example, `/orders/\{id\}`, where `id` takes a big number of values), it is strongly recommended to disable adding path to `uri` tag by setting `spring.cloud.loadbalancer.stats.include-path` to `false`.

NOTE: For some implementations, such as `BlockingLoadBalancerClient`, request and response data might not be available, as we establish generic types from arguments and might not be able to determine the types and read the data.

NOTE: The meters are registered in the registry when at least one record is added for a given meter.
Expand Down
1 change: 1 addition & 0 deletions docs/modules/ROOT/partials/_configprops.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
|spring.cloud.loadbalancer.retry.retryable-exceptions | `+++{}+++` | A `Set` of `Throwable` classes that should trigger a retry.
|spring.cloud.loadbalancer.retry.retryable-status-codes | `+++{}+++` | A `Set` of status codes that should trigger a retry.
|spring.cloud.loadbalancer.service-discovery.timeout | | String representation of Duration of the timeout for calls to service discovery.
|spring.cloud.loadbalancer.stats.include-path | `+++true+++` | Indicates whether the {@code path} should be added to {@code uri} tag in metrics. When {@link RestTemplate} is used to execute load-balanced requests with high cardinality paths, setting it to {@code false} is recommended.
|spring.cloud.loadbalancer.stats.micrometer.enabled | `+++false+++` | Enables Spring Cloud LoadBalancer Micrometer stats.
|spring.cloud.loadbalancer.sticky-session.add-service-instance-cookie | `+++false+++` | Indicates whether a cookie with the newly selected instance should be added by LoadBalancer.
|spring.cloud.loadbalancer.sticky-session.instance-id-cookie-name | `+++sc-lb-instance-id+++` | The name of the cookie holding the preferred instance id.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2012-2023 the original author or authors.
* Copyright 2012-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -33,6 +33,7 @@
import org.springframework.core.env.PropertyResolver;
import org.springframework.http.HttpMethod;
import org.springframework.util.LinkedCaseInsensitiveMap;
import org.springframework.web.client.RestTemplate;

/**
* The base configuration bean for Spring Cloud LoadBalancer.
Expand Down Expand Up @@ -90,10 +91,20 @@ public class LoadBalancerProperties {

/**
* Properties for
* {@link org.springframework.cloud.loadbalancer.core.SubsetServiceInstanceListSupplier}.
* {@code org.springframework.cloud.loadbalancer.core.SubsetServiceInstanceListSupplier}.
*/
private Subset subset = new Subset();

/**
* Enabling X-Forwarded Host and Proto Headers.
*/
private XForwarded xForwarded = new XForwarded();

/**
* Properties for LoadBalancer metrics.
*/
private Stats stats = new Stats();

public HealthCheck getHealthCheck() {
return healthCheck;
}
Expand Down Expand Up @@ -134,11 +145,6 @@ public void setHintHeaderName(String hintHeaderName) {
this.hintHeaderName = hintHeaderName;
}

/**
* Enabling X-Forwarded Host and Proto Headers.
*/
private XForwarded xForwarded = new XForwarded();

// TODO: fix spelling in a major release
public void setxForwarded(XForwarded xForwarded) {
this.xForwarded = xForwarded;
Expand All @@ -164,6 +170,14 @@ public void setCallGetWithRequestOnDelegates(boolean callGetWithRequestOnDelegat
this.callGetWithRequestOnDelegates = callGetWithRequestOnDelegates;
}

public Stats getStats() {
return stats;
}

public void setStats(Stats stats) {
this.stats = stats;
}

public static class StickySession {

/**
Expand Down Expand Up @@ -539,4 +553,23 @@ public void setSize(int size) {

}

public static class Stats {

/**
* Indicates whether the {@code path} should be added to {@code uri} tag in
* metrics. When {@link RestTemplate} is used to execute load-balanced requests
* with high cardinality paths, setting it to {@code false} is recommended.
*/
private boolean includePath = true;

public boolean isIncludePath() {
return includePath;
}

public void setIncludePath(boolean includePath) {
this.includePath = includePath;
}

}

}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2012-2020 the original author or authors.
* Copyright 2012-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -21,6 +21,8 @@
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.boot.autoconfigure.condition.ConditionalOnClass;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.loadbalancer.reactive.ReactiveLoadBalancer;
import org.springframework.cloud.loadbalancer.stats.MicrometerStatsLoadBalancerLifecycle;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
Expand All @@ -38,8 +40,9 @@ public class LoadBalancerStatsAutoConfiguration {

@Bean
@ConditionalOnBean(MeterRegistry.class)
public MicrometerStatsLoadBalancerLifecycle micrometerStatsLifecycle(MeterRegistry meterRegistry) {
return new MicrometerStatsLoadBalancerLifecycle(meterRegistry);
public MicrometerStatsLoadBalancerLifecycle micrometerStatsLifecycle(MeterRegistry meterRegistry,
ReactiveLoadBalancer.Factory<ServiceInstance> loadBalancerFactory) {
return new MicrometerStatsLoadBalancerLifecycle(meterRegistry, loadBalancerFactory);
}

}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2012-2020 the original author or authors.
* Copyright 2012-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand All @@ -16,11 +16,17 @@

package org.springframework.cloud.loadbalancer.stats;

import java.util.Collections;
import java.util.Objects;
import java.util.Optional;
import java.util.Set;

import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.Tags;

import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.loadbalancer.CompletionContext;
import org.springframework.cloud.client.loadbalancer.LoadBalancerProperties;
import org.springframework.cloud.client.loadbalancer.RequestData;
import org.springframework.cloud.client.loadbalancer.RequestDataContext;
import org.springframework.cloud.client.loadbalancer.ResponseData;
Expand All @@ -30,17 +36,25 @@
* Utility class for building metrics tags for load-balanced calls.
*
* @author Olga Maciaszek-Sharma
* @author Jaroslaw Dembek
* @since 3.0.0
*/
final class LoadBalancerTags {
class LoadBalancerTags {

static final String UNKNOWN = "UNKNOWN";

private LoadBalancerTags() {
throw new UnsupportedOperationException("Cannot instantiate utility class");
private final LoadBalancerProperties properties;

// Not using class references in case not in classpath
private static final Set<String> URI_TEMPLATE_ATTRIBUTES = Set.of(
"org.springframework.web.reactive.function.client.WebClient.uriTemplate",
"org.springframework.web.client.RestClient.uriTemplate");

LoadBalancerTags(LoadBalancerProperties properties) {
this.properties = properties;
}

static Iterable<Tag> buildSuccessRequestTags(CompletionContext<Object, ServiceInstance, Object> completionContext) {
Iterable<Tag> buildSuccessRequestTags(CompletionContext<Object, ServiceInstance, Object> completionContext) {
ServiceInstance serviceInstance = completionContext.getLoadBalancerResponse().getServer();
Tags tags = Tags.of(buildServiceInstanceTags(serviceInstance));
Object clientResponse = completionContext.getClientResponse();
Expand Down Expand Up @@ -69,12 +83,23 @@ private static int statusValue(ResponseData responseData) {
return responseData.getHttpStatus() != null ? responseData.getHttpStatus().value() : 200;
}

private static String getPath(RequestData requestData) {
return requestData.getUrl() != null ? requestData.getUrl().getPath() : UNKNOWN;
private String getPath(RequestData requestData) {
if (!properties.getStats().isIncludePath()) {
return UNKNOWN;
}
Optional<Object> uriTemplateValue = Optional.ofNullable(requestData.getAttributes())
.orElse(Collections.emptyMap())
.keySet()
.stream()
.filter(URI_TEMPLATE_ATTRIBUTES::contains)
.map(key -> requestData.getAttributes().get(key))
.filter(Objects::nonNull)
.findAny();
return uriTemplateValue.map(uriTemplate -> (String) uriTemplate)
.orElseGet(() -> (requestData.getUrl() != null) ? requestData.getUrl().getPath() : UNKNOWN);
}

static Iterable<Tag> buildDiscardedRequestTags(
CompletionContext<Object, ServiceInstance, Object> completionContext) {
Iterable<Tag> buildDiscardedRequestTags(CompletionContext<Object, ServiceInstance, Object> completionContext) {
if (completionContext.getLoadBalancerRequest().getContext() instanceof RequestDataContext) {
RequestData requestData = ((RequestDataContext) completionContext.getLoadBalancerRequest().getContext())
.getClientRequest();
Expand All @@ -92,7 +117,7 @@ private static String getHost(RequestData requestData) {
return requestData.getUrl() != null ? requestData.getUrl().getHost() : UNKNOWN;
}

static Iterable<Tag> buildFailedRequestTags(CompletionContext<Object, ServiceInstance, Object> completionContext) {
Iterable<Tag> buildFailedRequestTags(CompletionContext<Object, ServiceInstance, Object> completionContext) {
ServiceInstance serviceInstance = completionContext.getLoadBalancerResponse().getServer();
Tags tags = Tags.of(buildServiceInstanceTags(serviceInstance)).and(exception(completionContext.getThrowable()));
if (completionContext.getLoadBalancerRequest().getContext() instanceof RequestDataContext) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright 2012-2020 the original author or authors.
* Copyright 2012-2024 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -27,31 +27,50 @@

import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.loadbalancer.CompletionContext;
import org.springframework.cloud.client.loadbalancer.LoadBalancerClientsProperties;
import org.springframework.cloud.client.loadbalancer.LoadBalancerLifecycle;
import org.springframework.cloud.client.loadbalancer.LoadBalancerProperties;
import org.springframework.cloud.client.loadbalancer.Request;
import org.springframework.cloud.client.loadbalancer.Response;
import org.springframework.cloud.client.loadbalancer.TimedRequestContext;
import org.springframework.cloud.client.loadbalancer.reactive.ReactiveLoadBalancer;
import org.springframework.cloud.loadbalancer.support.LoadBalancerClientFactory;

import static org.springframework.cloud.loadbalancer.stats.LoadBalancerTags.buildDiscardedRequestTags;
import static org.springframework.cloud.loadbalancer.stats.LoadBalancerTags.buildFailedRequestTags;
import static org.springframework.cloud.loadbalancer.stats.LoadBalancerTags.buildServiceInstanceTags;
import static org.springframework.cloud.loadbalancer.stats.LoadBalancerTags.buildSuccessRequestTags;

/**
* An implementation of {@link LoadBalancerLifecycle} that records metrics for
* load-balanced calls.
*
* @author Olga Maciaszek-Sharma
* @author Jaroslaw Dembek
* @since 3.0.0
*/
public class MicrometerStatsLoadBalancerLifecycle implements LoadBalancerLifecycle<Object, Object, ServiceInstance> {

private final MeterRegistry meterRegistry;

private final ReactiveLoadBalancer.Factory<ServiceInstance> loadBalancerFactory;

private final ConcurrentHashMap<ServiceInstance, AtomicLong> activeRequestsPerInstance = new ConcurrentHashMap<>();

public MicrometerStatsLoadBalancerLifecycle(MeterRegistry meterRegistry) {
public MicrometerStatsLoadBalancerLifecycle(MeterRegistry meterRegistry,
ReactiveLoadBalancer.Factory<ServiceInstance> loadBalancerFactory) {
this.meterRegistry = meterRegistry;
this.loadBalancerFactory = loadBalancerFactory;
}

/**
* Creates a MicrometerStatsLoadBalancerLifecycle instance based on the provided
* {@link MeterRegistry}.
* @param meterRegistry {@link MeterRegistry} to use for Micrometer metrics.
* @deprecated in favour of
* {@link MicrometerStatsLoadBalancerLifecycle#MicrometerStatsLoadBalancerLifecycle(MeterRegistry, ReactiveLoadBalancer.Factory)}
*/
@Deprecated(forRemoval = true)
public MicrometerStatsLoadBalancerLifecycle(MeterRegistry meterRegistry) {
// use default properties when calling deprecated constructor
this(meterRegistry, new LoadBalancerClientFactory(new LoadBalancerClientsProperties()));
}

@Override
Expand Down Expand Up @@ -85,15 +104,19 @@ public void onStartRequest(Request<Object> request, Response<ServiceInstance> lb

@Override
public void onComplete(CompletionContext<Object, ServiceInstance, Object> completionContext) {
ServiceInstance serviceInstance = completionContext.getLoadBalancerResponse().getServer();
LoadBalancerProperties properties = serviceInstance != null
? loadBalancerFactory.getProperties(serviceInstance.getServiceId())
: loadBalancerFactory.getProperties(null);
LoadBalancerTags loadBalancerTags = new LoadBalancerTags(properties);
long requestFinishedTimestamp = System.nanoTime();
if (CompletionContext.Status.DISCARD.equals(completionContext.status())) {
Counter.builder("loadbalancer.requests.discard")
.tags(buildDiscardedRequestTags(completionContext))
.tags(loadBalancerTags.buildDiscardedRequestTags(completionContext))
.register(meterRegistry)
.increment();
return;
}
ServiceInstance serviceInstance = completionContext.getLoadBalancerResponse().getServer();
AtomicLong activeRequestsCounter = activeRequestsPerInstance.get(serviceInstance);
if (activeRequestsCounter != null) {
activeRequestsCounter.decrementAndGet();
Expand All @@ -102,15 +125,15 @@ public void onComplete(CompletionContext<Object, ServiceInstance, Object> comple
if (requestHasBeenTimed(loadBalancerRequestContext)) {
if (CompletionContext.Status.FAILED.equals(completionContext.status())) {
Timer.builder("loadbalancer.requests.failed")
.tags(buildFailedRequestTags(completionContext))
.tags(loadBalancerTags.buildFailedRequestTags(completionContext))
.register(meterRegistry)
.record(requestFinishedTimestamp
- ((TimedRequestContext) loadBalancerRequestContext).getRequestStartTime(),
TimeUnit.NANOSECONDS);
return;
}
Timer.builder("loadbalancer.requests.success")
.tags(buildSuccessRequestTags(completionContext))
.tags(loadBalancerTags.buildSuccessRequestTags(completionContext))
.register(meterRegistry)
.record(requestFinishedTimestamp
- ((TimedRequestContext) loadBalancerRequestContext).getRequestStartTime(),
Expand Down
Loading