-
Notifications
You must be signed in to change notification settings - Fork 214
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Updated Automatically Rotating Certs Docs #1757
base: main
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,9 +11,10 @@ shared across clusters, and an *issuer certificate*, which is specific to the | |
cluster. | ||
|
||
While Linkerd automatically rotates the per-proxy TLS certificates, it does not | ||
rotate the issuer certificate. In this doc, we'll describe how to set up | ||
automatic rotation of the issuer certificate and its corresponding private key | ||
using the cert-manager project. | ||
rotate the issuer certificate. Linkerd's out-of-the-box installations generate | ||
static self-signed certificates with a validity of one year but require manual | ||
rotation by the user to prevent expiry. While this setup is convenient for quick | ||
start testing, it's not advisable nor recommended for production environments. | ||
|
||
{{< trylpt >}} | ||
|
||
|
@@ -24,17 +25,23 @@ for making TLS credentials from external sources available to Kubernetes | |
clusters. | ||
|
||
Cert-manager is very flexible. You can configure it to pull certificates from | ||
secrets managemenet solutions such as [Vault](https://www.vaultproject.io). In | ||
secrets managemenet solutions such as [Vault](https://www.vaultproject.io). In | ||
this guide, we'll focus on a self-sufficient setup: we will configure | ||
cert-manager to act as an on-cluster | ||
[CA](https://en.wikipedia.org/wiki/Certificate_authority) and have it re-issue | ||
Linkerd's issuer certificate and private key on a periodic basis, derived from | ||
the trust anchor. | ||
the trust anchor. Additionally, we will use trust-manager create a trust bundle | ||
which will allow Linkerd to verify the authenticity of certificates issued by | ||
cert-manager. | ||
|
||
### Cert manager as an on-cluster CA | ||
|
||
As a first step, [install cert-manager on your | ||
cluster](https://cert-manager.io/docs/installation/). | ||
As a first step, | ||
[install cert-manager on your cluster](https://cert-manager.io/docs/installation/), | ||
then | ||
[install trust-manager](https://cert-manager.io/docs/trust/trust-manager/installation/) | ||
and configure it to use "linkerd" as the | ||
[trust namespace](https://cert-manager.io/docs/trust/trust-manager/installation/#trust-namespace). | ||
|
||
Next, create the namespace that cert-manager will use to store its | ||
Linkerd-related resources. For simplicity, we suggest reusing the default | ||
|
@@ -44,46 +51,111 @@ Linkerd control plane namespace: | |
kubectl create namespace linkerd | ||
``` | ||
|
||
#### Save the signing key pair as a Secret | ||
#### Give cert-manager necessary RBAC permissions | ||
|
||
Next, using the [`step`](https://smallstep.com/cli/) tool, create a signing key | ||
pair and store it in a Kubernetes Secret in the namespace created above: | ||
By default cert-manager will only create certificate secrets in the namespace | ||
where it is installed. Linkerd, however, requires the cert secrets to be created | ||
in the linkerd namespace. To do this we will create a `ServiceAccount` for | ||
cert-manager in the linkerd namespace with the required permissions. | ||
|
||
```bash | ||
step certificate create root.linkerd.cluster.local ca.crt ca.key \ | ||
--profile root-ca --no-password --insecure && | ||
kubectl create secret tls \ | ||
linkerd-trust-anchor \ | ||
--cert=ca.crt \ | ||
--key=ca.key \ | ||
--namespace=linkerd | ||
kubectl apply -f - <<EOF | ||
apiVersion: v1 | ||
kind: ServiceAccount | ||
metadata: | ||
name: cert-manager | ||
namespace: linkerd | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: Role | ||
metadata: | ||
name: cert-manager-secret-creator | ||
namespace: linkerd | ||
rules: | ||
- apiGroups: [""] | ||
resources: ["secrets"] | ||
verbs: ["create", "get", "update", "patch"] | ||
--- | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
kind: RoleBinding | ||
metadata: | ||
name: cert-manager-secret-creator-binding | ||
namespace: linkerd | ||
subjects: | ||
- kind: ServiceAccount | ||
name: cert-manager | ||
namespace: linkerd | ||
roleRef: | ||
kind: Role | ||
name: cert-manager-secret-creator | ||
apiGroup: rbac.authorization.k8s.io | ||
EOF | ||
``` | ||
|
||
For a longer-lived trust anchor certificate, pass the `--not-after` argument | ||
to the step command with the desired value (e.g. `--not-after=87600h`). | ||
#### Create the trust root ClusterIssuer | ||
|
||
To begin, create a self-signing `ClusterIssuer` for the Linkerd trust root | ||
certificate. | ||
|
||
```bash | ||
kubectl apply -f - <<EOF | ||
apiVersion: cert-manager.io/v1 | ||
kind: ClusterIssuer | ||
metadata: | ||
name: linkerd-trust-root-issuer | ||
spec: | ||
selfSigned: {} | ||
EOF | ||
``` | ||
|
||
#### Create an Issuer referencing the secret | ||
#### Create a trust root certificate | ||
|
||
With the Secret in place, we can create a cert-manager "Issuer" resource that | ||
references it: | ||
Now create a cert-manager `Certificate` resource which uses the | ||
previously-created `ClusterIssuer`: | ||
|
||
```bash | ||
kubectl apply -f - <<EOF | ||
apiVersion: cert-manager.io/v1 | ||
kind: Issuer | ||
kind: Certificate | ||
metadata: | ||
name: linkerd-trust-anchor | ||
namespace: linkerd | ||
spec: | ||
commonName: root.linkerd.cluster.local | ||
isCA: true | ||
duration: 87600h0m0s | ||
renewBefore: 87264h0m0s | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Huh. I had thought this was "renew, at latest, when this many hours are remaining"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. you're correct. updating to clarify. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. just double checked their docs: https://cert-manager.io/docs/usage/certificate/#reissuance-triggered-by-expiry-renewal It looks like the original blurb we had on the site originally (where I copied this blurb from) is accurate: "If |
||
issuerRef: | ||
name: linkerd-trust-root-issuer | ||
kind: ClusterIssuer | ||
privateKey: | ||
algorithm: ECDSA | ||
secretName: linkerd-trust-anchor | ||
EOF | ||
``` | ||
|
||
#### Create Linkerd identity issuer | ||
|
||
Using the previously-generated trust root certificate, create a Linkerd identity | ||
`Issuer`: | ||
|
||
```bash | ||
kubectl apply -f - <<EOF | ||
apiVersion: cert-manager.io/v1 | ||
kind: Issuer | ||
metadata: | ||
name: linkerd-identity-issuer | ||
namespace: linkerd | ||
spec: | ||
ca: | ||
secretName: linkerd-trust-anchor | ||
EOF | ||
``` | ||
|
||
#### Create a Certificate resource referencing the Issuer | ||
#### Create a Certificate resource referencing the issuer | ||
|
||
Finally, we can create a cert-manager "Certificate" resource which uses this | ||
Issuer to generate the desired certificate: | ||
Next, create a Linkerd identity issuer certificate which will act as an | ||
intermediary signing CA for all Linkerd mTLS proxy certificates: | ||
|
||
```bash | ||
kubectl apply -f - <<EOF | ||
|
@@ -93,67 +165,131 @@ metadata: | |
name: linkerd-identity-issuer | ||
namespace: linkerd | ||
spec: | ||
secretName: linkerd-identity-issuer | ||
duration: 48h | ||
renewBefore: 25h | ||
issuerRef: | ||
name: linkerd-trust-anchor | ||
kind: Issuer | ||
commonName: identity.linkerd.cluster.local | ||
dnsNames: | ||
- identity.linkerd.cluster.local | ||
isCA: true | ||
privateKey: | ||
algorithm: ECDSA | ||
usages: | ||
- cert sign | ||
- crl sign | ||
- server auth | ||
- client auth | ||
- cert sign | ||
- crl sign | ||
- server auth | ||
- client auth | ||
duration: 28h0m0s | ||
renewBefore: 25h0m0s | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same question as above re |
||
issuerRef: | ||
name: linkerd-identity-issuer | ||
kind: Issuer | ||
privateKey: | ||
algorithm: ECDSA | ||
secretName: linkerd-identity-issuer | ||
EOF | ||
``` | ||
|
||
(In the YAML manifest above, the `duration` key instructs cert-manager to | ||
consider certificates as valid for `48` hours and the `renewBefore` key indicates | ||
that cert-manager will attempt to issue a new certificate `25` hours before | ||
expiration of the current one. These values can be customized to your liking.) | ||
consider certificates as valid for `48` hours and the `renewBefore` key | ||
indicates that cert-manager will attempt to issue a new certificate `25` hours | ||
before expiration of the current one. These values can be customized to your | ||
liking.) | ||
|
||
#### Create Linkerd trust bundle | ||
|
||
Lastly, we will also need to create a trust bundle which will allow Linkerd's | ||
identity controller to verify the authenticity of certificates issued by | ||
cert-manager: | ||
|
||
```bash | ||
kubectl apply -f - <<EOF | ||
apiVersion: trust.cert-manager.io/v1alpha1 | ||
kind: Bundle | ||
metadata: | ||
name: linkerd-identity-trust-roots | ||
namespace: linkerd | ||
spec: | ||
sources: | ||
- secret: | ||
name: "linkerd-trust-anchor" | ||
key: "ca.crt" | ||
target: | ||
configMap: | ||
key: "ca-bundle.crt" | ||
EOF | ||
``` | ||
|
||
## Summary & Validation | ||
|
||
Below are the resources created for managing Linkerd identity certificates with | ||
cert-manager: | ||
|
||
At this point, cert-manager can now use this Certificate resource to obtain TLS | ||
credentials, which will be stored in a secret named `linkerd-identity-issuer`. | ||
To validate your newly-issued certificate, you can run: | ||
- Namespace: `linkerd` (to store certificates and secrets) | ||
- RBAC Permissions: ServiceAccount, Role, and RoleBinding in the `linkerd` | ||
namespace for cert-manager | ||
- ClusterIssuer: `linkerd-trust-root-issuer` (self-signed ClusterIssuer) | ||
- Certificate: `linkerd-trust-anchor` (in the `linkerd` namespace, referencing | ||
`linkerd-trust-root-issuer`) | ||
- Issuer: `linkerd-identity-issuer` (to manage Linkerd identity certificates) | ||
- Certificate: `linkerd-identity-issuer` (in the `linkerd` namespace, acting as | ||
an intermediary signing CA) | ||
- Trust Bundle: `linkerd-identity-trust-roots` (to allow Linkerd's identity | ||
controller to verify certificate authenticity) | ||
|
||
To validate creation and status, run the following commands: | ||
|
||
```bash | ||
kubectl get secret linkerd-identity-issuer -o yaml -n linkerd | ||
# Check namespace creation | ||
kubectl get namespaces linkerd | ||
|
||
# Check RBAC permissions | ||
kubectl get serviceaccount,role,rolebinding -n linkerd | ||
|
||
# Check ClusterIssuer creation | ||
kubectl get clusterissuers linkerd-trust-root-issuer | ||
|
||
# Check Certificate creation | ||
kubectl get certificates -n linkerd | ||
|
||
# Check Issuer creation | ||
kubectl get issuers.cert-manager.io -n linkerd | ||
|
||
# Check Trust Bundle creation | ||
kubectl get bundles -n linkerd | ||
``` | ||
|
||
Now we just need to inform Linkerd to consume these credentials. | ||
## Consuming cert-manager identity certificates | ||
|
||
To have Linkerd consume cert-manager created certificates you will need to add | ||
the following to your values file or pass them in as flags at runtime. | ||
|
||
| Field | Value | | ||
| ------------------------ | ----------------- | | ||
| `identity.externalCA` | true | | ||
| `identity.issuer.scheme` | kubernetes.io/tls | | ||
|
||
## Using these credentials with CLI installation | ||
### Using these credentials with CLI installation | ||
|
||
For CLI installation, the Linkerd control plane should be installed with the | ||
`--identity-external-issuer` flag, which instructs Linkerd to read certificates | ||
from the `linkerd-identity-issuer` secret. Whenever certificate and key stored | ||
in the secret are updated, the `identity` service will automatically detect | ||
this change and reload the new credentials. | ||
in the secret are updated, the `identity` service will automatically detect this | ||
change and reload the new credentials. | ||
|
||
Voila! We have set up automatic rotation of Linkerd's control plane TLS | ||
credentials. | ||
|
||
## Using these credentials with a Helm installation | ||
### Using these credentials with a Helm installation | ||
|
||
For installing with Helm, first install the `linkerd-crds` chart: | ||
|
||
```bash | ||
helm install linkerd-crds -n linkerd --create-namespace linkerd/linkerd-crds | ||
``` | ||
|
||
Then install the `linkerd-control-plane` chart, setting the | ||
`identityTrustAnchorsPEM` to the value of `ca.crt` in the | ||
`linkerd-identity-issuer` Secret: | ||
Then install the `linkerd-control-plane` chart: | ||
|
||
```bash | ||
helm install linkerd-control-plane -n linkerd \ | ||
--set-file identityTrustAnchorsPEM=ca.crt \ | ||
--set identity.externalCA=true \ | ||
--set identity.issuer.scheme=kubernetes.io/tls \ | ||
linkerd/linkerd-control-plane | ||
``` | ||
|
@@ -199,5 +335,5 @@ following instructions to install and configure Linkerd to use it. | |
|
||
## See also | ||
|
||
* [Automatically Rotating Webhook TLS Credentials](../automatically-rotating-webhook-tls-credentials/) | ||
* [Manually rotating Linkerd's trust anchor credentials](../manually-rotating-control-plane-tls-credentials/) | ||
- [Automatically Rotating Webhook TLS Credentials](../automatically-rotating-webhook-tls-credentials/) | ||
- [Manually rotating Linkerd's trust anchor credentials](../manually-rotating-control-plane-tls-credentials/) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"to create"