Can someone explain why Fleet & Agent needs to talk to 5601?

[philippkahr]

5601 is usually reserved for Kibana. I don't understand why those two ports are listed there as communication parts. Seems werid.

Fleet => Kibana using 5601 makes sense as there is a fleet kibana url in the fleet settings:
https://www.elastic.co/guide/en/fleet/current/agent-environment-variables.html#env-enroll-agent

[mergify]

This pull request does not have a backport label. Could you fix it @philippkahr? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-/d./d is the label to automatically backport to the /d./d branch. /d is the digit
  NOTE: backport-skip has been added to this pull request.

[kilfoyle]

These settings were added recently via elastic/observability-docs#2725
@nimarezainia would you be able to help here?

[nimarezainia]

@philippkahr there are instances where the agent would need to connect to Fleet (kibana). For that we need port 5601 opened. I believe this part of Agent requesting a service-token in one of the commands we have.

adding @cmacknz & @michel-laterman to keep me honest.

[cmacknz]

This is referenced in the docs for running the agent as a container: https://www.elastic.co/guide/en/fleet/current/running-on-kubernetes-managed-by-fleet.html

The agent code has this to say about it:

https://github.com/elastic/elastic-agent/blob/73144c11a608aa425312e67b5a19d1047318fd91/internal/pkg/agent/cmd/container.go#L106-L116

* Preparing Kibana for Fleet
  This prepares the Fleet plugin that exists inside of Kibana. This must either be enabled here or done externally
  before Fleet Server will actually successfully start. All the Kibana variables are not needed in case Elastic Agent
  should not setup Fleet.

  KIBANA_FLEET_HOST - Kibana host accessible from Fleet Server. [$KIBANA_HOST]
  KIBANA_FLEET_USERNAME - Kibana username to service token [$KIBANA_USERNAME]
  KIBANA_FLEET_PASSWORD - Kibana password to service token [$KIBANA_PASSWORD]
  KIBANA_FLEET_CA - path to certificate authority to use with communicate with Kibana [$KIBANA_CA]
  KIBANA_REQUEST_RETRY_SLEEP - sleep duration taken when agent performs a request to Kibana [default 1s]
  KIBANA_REQUEST_RETRY_COUNT - number of retries agent performs when executing a request to Kibana [default 30]

So there is a path in the code where the agent will reach out to Kibana to enable Fleet, and 5601 is only necessary if you have configured the agent specifically to do this.

I remember hearing that at some point the Fleet plugin didn't start automatically, but now this isn't an issue with Fleet starting automatically and this code and configuration might be obsolete.

@juliaElastic is that correct? If so we can probably deprecate all of these environment variables and remove these parts of the docs, but I wasn't around when any of this was implemented so I'm not sure if there's a historical reason to keep this.

[juliaElastic]

Currently Fleet plugin in kibana doesn't need anything from the agent to start up, I don't remember when this worked like this, maybe @nchaulet was here then.
AFAIK these kibana settings are used in agent to create an enrollment token if it's not provided, there are some references in the code for this:

https://github.com/elastic/elastic-agent/blob/7a1937909f142ac5362e4f6e39e707d3cbf21339/dev-tools/kubernetes/base/elastic-agent-managed/.env#L8-L10

https://github.com/elastic/elastic-agent/blob/7a1937909f142ac5362e4f6e39e707d3cbf21339/internal/pkg/agent/cmd/container.go#L77-L79