Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wire server run options to flags. #1560

Merged
merged 1 commit into from
Sep 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 25 additions & 14 deletions cmd/metrics-server/app/options/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,13 +38,14 @@ import (

type Options struct {
// genericoptions.RecomendedOptions - EtcdOptions
SecureServing *genericoptions.SecureServingOptionsWithLoopback
Authentication *genericoptions.DelegatingAuthenticationOptions
Authorization *genericoptions.DelegatingAuthorizationOptions
Audit *genericoptions.AuditOptions
Features *genericoptions.FeatureOptions
KubeletClient *KubeletClientOptions
Logging *logs.Options
GenericServerRunOptions *genericoptions.ServerRunOptions
SecureServing *genericoptions.SecureServingOptionsWithLoopback
Authentication *genericoptions.DelegatingAuthenticationOptions
Authorization *genericoptions.DelegatingAuthorizationOptions
Audit *genericoptions.AuditOptions
Features *genericoptions.FeatureOptions
KubeletClient *KubeletClientOptions
Logging *logs.Options

MetricResolution time.Duration
ShowVersion bool
Expand All @@ -61,6 +62,9 @@ func (o *Options) Validate() []error {
if err != nil {
errors = append(errors, err)
}
if errs := o.GenericServerRunOptions.Validate(); len(errs) > 0 {
errors = append(errors, errs...)
}
return errors
}

Expand All @@ -81,6 +85,7 @@ func (o *Options) Flags() (fs flag.NamedFlagSets) {
msfs.BoolVar(&o.ShowVersion, "version", false, "Show version")
msfs.StringVar(&o.Kubeconfig, "kubeconfig", o.Kubeconfig, "The path to the kubeconfig used to connect to the Kubernetes API server and the Kubelets (defaults to in-cluster config)")

o.GenericServerRunOptions.AddUniversalFlags(fs.FlagSet("generic"))
o.KubeletClient.AddFlags(fs.FlagSet("kubelet client"))
o.SecureServing.AddFlags(fs.FlagSet("apiserver secure serving"))
o.Authentication.AddFlags(fs.FlagSet("apiserver authentication"))
Expand All @@ -95,13 +100,14 @@ func (o *Options) Flags() (fs flag.NamedFlagSets) {
// NewOptions constructs a new set of default options for metrics-server.
func NewOptions() *Options {
return &Options{
SecureServing: genericoptions.NewSecureServingOptions().WithLoopback(),
Authentication: genericoptions.NewDelegatingAuthenticationOptions(),
Authorization: genericoptions.NewDelegatingAuthorizationOptions(),
Features: genericoptions.NewFeatureOptions(),
Audit: genericoptions.NewAuditOptions(),
KubeletClient: NewKubeletClientOptions(),
Logging: logs.NewOptions(),
GenericServerRunOptions: genericoptions.NewServerRunOptions(),
SecureServing: genericoptions.NewSecureServingOptions().WithLoopback(),
Authentication: genericoptions.NewDelegatingAuthenticationOptions(),
Authorization: genericoptions.NewDelegatingAuthorizationOptions(),
Features: genericoptions.NewFeatureOptions(),
Audit: genericoptions.NewAuditOptions(),
KubeletClient: NewKubeletClientOptions(),
Logging: logs.NewOptions(),

MetricResolution: 60 * time.Second,
}
Expand Down Expand Up @@ -132,6 +138,11 @@ func (o Options) ApiserverConfig() (*genericapiserver.Config, error) {
}

serverConfig := genericapiserver.NewConfig(api.Codecs)

if err := o.GenericServerRunOptions.ApplyTo(serverConfig); err != nil {
return nil, err
}

if err := o.SecureServing.ApplyTo(&serverConfig.SecureServing, &serverConfig.LoopbackClientConfig); err != nil {
return nil, err
}
Expand Down
50 changes: 50 additions & 0 deletions docs/command-line-flags.txt
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,56 @@ Metrics server flags:
--metric-resolution duration The resolution at which metrics-server will retain metrics, must set value at least 10s. (default 1m0s)
--version Show version

Generic flags:

--advertise-address ip The IP address on which to advertise the apiserver to members of the cluster. This address must be reachable by the rest of the cluster. If blank, the --bind-address will be used. If --bind-address is unspecified, the host's default interface will be used.
--cors-allowed-origins strings List of allowed origins for CORS, comma separated. An allowed origin can be a regular expression to support subdomain matching. If this list is empty CORS will not be enabled. Please ensure each expression matches the entire hostname by anchoring to the start with '^' or including the '//' prefix, and by anchoring to the end with '$' or including the ':' port separator suffix. Examples of valid expressions are '//example\.com(:|$)' and '^https://example\.com(:|$)'
--emulated-version strings The versions different components emulate their capabilities (APIs, features, ...) of.
If set, the component will emulate the behavior of this version instead of the underlying binary version.
Version format could only be major.minor, for example: '--emulated-version=wardle=1.2,kube=1.31'. Options are:
kube=1.31..1.31 (default=1.31)If the component is not specified, defaults to "kube"
--external-hostname string The hostname to use when generating externalized URLs for this master (e.g. Swagger API Docs or OpenID Discovery).
--feature-gates colonSeparatedMultimapStringString Comma-separated list of component:key=value pairs that describe feature gates for alpha/experimental features of different components.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't support most of the feature flags listed here.

Copy link
Member

@dgrisonnet dgrisonnet Sep 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you suggesting that we should override the flag to change the description to only include the feature gates that metrics-server's code uses?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From user perspective yes, we should not expose nor document flags that don't work or are not supported in our documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's fair, I'll try to figure out how we can have a curated list that is kept in sync with k/k

If the component is not specified, defaults to "kube". This flag can be repeatedly invoked. For example: --feature-gates 'wardle:featureA=true,wardle:featureB=false' --feature-gates 'kube:featureC=true'Options are:
kube:APIResponseCompression=true|false (BETA - default=true)
kube:APIServerIdentity=true|false (BETA - default=true)
kube:APIServerTracing=true|false (BETA - default=true)
kube:APIServingWithRoutine=true|false (ALPHA - default=false)
kube:AllAlpha=true|false (ALPHA - default=false)
kube:AllBeta=true|false (BETA - default=false)
kube:AnonymousAuthConfigurableEndpoints=true|false (ALPHA - default=false)
kube:AuthorizeWithSelectors=true|false (ALPHA - default=false)
kube:ComponentSLIs=true|false (BETA - default=true)
kube:ConcurrentWatchObjectDecode=true|false (BETA - default=false)
kube:ConsistentListFromCache=true|false (BETA - default=true)
kube:CoordinatedLeaderElection=true|false (ALPHA - default=false)
kube:InPlacePodVerticalScaling=true|false (ALPHA - default=false)
kube:OpenAPIEnums=true|false (BETA - default=true)
kube:ResilientWatchCacheInitialization=true|false (BETA - default=true)
kube:RetryGenerateName=true|false (BETA - default=true)
kube:SeparateCacheWatchRPC=true|false (BETA - default=true)
kube:StorageVersionAPI=true|false (ALPHA - default=false)
kube:StorageVersionHash=true|false (BETA - default=true)
kube:StrictCostEnforcementForVAP=true|false (BETA - default=false)
kube:StrictCostEnforcementForWebhooks=true|false (BETA - default=false)
kube:StructuredAuthenticationConfiguration=true|false (BETA - default=true)
kube:StructuredAuthorizationConfiguration=true|false (BETA - default=true)
kube:UnauthenticatedHTTP2DOSMitigation=true|false (BETA - default=true)
kube:WatchCacheInitializationPostStartHook=true|false (BETA - default=false)
kube:WatchFromStorageWithoutResourceVersion=true|false (BETA - default=false)
kube:WatchList=true|false (ALPHA - default=false)
--goaway-chance float To prevent HTTP/2 clients from getting stuck on a single apiserver, randomly close a connection (GOAWAY). The client's other in-flight requests won't be affected, and the client will reconnect, likely landing on a different apiserver after going through the load balancer again. This argument sets the fraction of requests that will be sent a GOAWAY. Clusters with single apiservers, or which don't use a load balancer, should NOT enable this. Min is 0 (off), Max is .02 (1/50 requests); .001 (1/1000) is a recommended starting point.
--livez-grace-period duration This option represents the maximum amount of time it should take for apiserver to complete its startup sequence and become live. From apiserver's start time to when this amount of time has elapsed, /livez will assume that unfinished post-start hooks will complete successfully and therefore return true.
--max-mutating-requests-inflight int This and --max-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of mutating requests in flight, or a zero value disables the limit completely. (default 200)
--max-requests-inflight int This and --max-mutating-requests-inflight are summed to determine the server's total concurrency limit (which must be positive) if --enable-priority-and-fairness is true. Otherwise, this flag limits the maximum number of non-mutating requests in flight, or a zero value disables the limit completely. (default 400)
--min-request-timeout int An optional field indicating the minimum number of seconds a handler must keep a request open before timing it out. Currently only honored by the watch request handler, which picks a randomized value above this number as the connection timeout, to spread out load. (default 1800)
--request-timeout duration An optional field indicating the duration a handler must keep a request open before timing it out. This is the default request timeout for requests but may be overridden by flags such as --min-request-timeout for specific types of requests. (default 1m0s)
--shutdown-delay-duration duration Time to delay the termination. During that time the server keeps serving requests normally. The endpoints /healthz and /livez will return success, but /readyz immediately returns failure. Graceful termination starts after this delay has elapsed. This can be used to allow load balancer to stop sending traffic to this server.
--shutdown-send-retry-after If true the HTTP Server will continue listening until all non long running request(s) in flight have been drained, during this window all incoming requests will be rejected with a status code 429 and a 'Retry-After' response header, in addition 'Connection: close' response header is set in order to tear down the TCP connection when idle.
--shutdown-watch-termination-grace-period duration This option, if set, represents the maximum amount of grace period the apiserver will wait for active watch request(s) to drain during the graceful server shutdown window.
--storage-initialization-timeout duration Maximum amount of time to wait for storage initialization before declaring apiserver ready. Defaults to 1m. (default 1m0s)
--strict-transport-security-directives strings List of directives for HSTS, comma separated. If this list is empty, then HSTS directives will not be added. Example: 'max-age=31536000,includeSubDomains,preload'

Kubelet client flags:

--deprecated-kubelet-completely-insecure DEPRECATED: Do not use any encryption, authorization, or authentication when communicating with the Kubelet. This is rarely the right option, since it leaves kubelet communication completely insecure. If you encounter auth errors, make sure you've enabled token webhook auth on the Kubelet, and if you're in a test cluster with self-signed Kubelet certificates, consider using kubelet-insecure-tls instead.
Expand Down