Add support for Argo Rollouts / Flux Flagger

[brianehlert]

Add support to NGINX Ingress Controller that supports the ability to use Argo's Rollouts and Flux's Flagger.
The Rollouts and Flagger products are pretty commonly used by customers to controller versioning of backend services that are presented through the NGINX Ingress Controller.

Today, the Rollouts and Flagger projects are hard coded to the Ingress resource and a subset of the nginx.ingress.kubernetes.io annotations.

One way to quickly add support for Rollouts / Flagger is to add support for the necessary nginx.ingress.kubernetes.io annotations so those projects could immediately meet the needs of the user base that remains committed to the Ingress resource.

Supporting VirtualServer/VirtualServerRoute would be a much greater lift as Rollouts/Flagger would need to be extended to recognize these custom resources so that it would know how to properly act on them.
The extent of this work is unknown and speculative. We are not aware of any customers that have achieved this through customization of Rollouts/Flagger.

The nginx.ingress.kubernetes.io annotations that are required to support Rollouts/Flagger are:

Rollouts:

• nginx.ingress.kubernetes.io/canary
• nginx.ingress.kubernetes.io/canary-weight

Flagger:

• nginx.ingress.kubernetes.io/canary
• nginx.ingress.kubernetes.io/canary-weight

A copy of the Ingress is created with the canary annotations.
This canary Ingress will need to be allowed to collide with the source Ingress it is a copy of.
Using splits the weights would need to be set appropriately on both upstream groups.
Performing the weight changes without reload should be possible if N+ is used. This should compliment that capability released in 3.5.

[jasonwilliams14]

A copy of the Ingress is created with the canary annotations.
This canary Ingress will need to be allowed to collide with the source Ingress it is a copy of.
Using splits the weights would need to be set appropriately on both upstream groups.

Yes. Correct. When either tool is used for canary use cases, they will point to a existing ingress resource and upon deployment, will create a second ingress copy (with the same host name but different name) and add those specific annotations:

nginx.ingress.kubernetes.io/canary
nginx.ingress.kubernetes.io/canary-weight

Since the community version of NGINX Ingress uses lua under the hood. If we take a look at the backend that are configured with lua using the ingress-nginx krew plugin, we can see the trafficShapePolicy that has been created:

    "trafficShapingPolicy": {
      "weight": 50,
      "weightTotal": 100,
      "header": "",
      "headerValue": "",
      "headerPattern": "",
      "cookie": ""
    }
  },

So we can see a basic weight of 50 (50%) has been defined for this particular ingress resource.

Using splits the weights would need to be set appropriately on both upstream groups.

yes. Using native NGINX functionality, split, map and weights, we can generate the appropriate nginx.conf for canary type scenarios.

https://nginx.org/en/docs/http/ngx_http_split_clients_module.html

https://nginx.org/en/docs/http/ngx_http_map_module.html

https://nginx.org/en/docs/http/ngx_http_upstream_module.html#weight

[brianehlert]

Argo Rollouts Demo: https://youtu.be/hIL0E2gLkf8