Skip to content

Commit

Permalink
Merge pull request #32 from ykulazhenkov/webhook
Browse files Browse the repository at this point in the history
Add Validating webhook
  • Loading branch information
ykulazhenkov authored Oct 2, 2023
2 parents 11f148c + 3a0c4ce commit c7a6c38
Show file tree
Hide file tree
Showing 42 changed files with 1,135 additions and 359 deletions.
75 changes: 52 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,16 @@ NVIDIA IPAM plugin consists of 3 main components:
A Kubernetes(K8s) controller that Watches on IPPools CRs in a predefined Namespace.
It then proceeds by assiging each node via IPPools Status a cluster unique range of IPs of the defined IP Pools.

#### Validation webhook

ipam-controller implements validation webhook for IPPool resource.
The webhook can prevent the creation of IPPool resources with invalid configurations.
Supported X.509 certificate management system should be available in the cluster to enable the webhook.
Currently supported systems are [certmanager](https://cert-manager.io/) and
[Openshift certificate management](https://docs.openshift.com/container-platform/4.13/security/certificates/service-serving-certificate.html)

Activation of the validation webhook is optional. Check the [Deployment](#deployment) section for details.

### ipam-node

The daemon is responsible for:
Expand Down Expand Up @@ -144,48 +154,50 @@ ipam-controller accepts configuration using command line flags and IPPools CRs.
```text
Logging flags:
--log-flush-frequency duration
--log-flush-frequency duration
Maximum number of seconds between log flushes (default 5s)
--log-json-info-buffer-size quantity
[Alpha] In JSON format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The
size can be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature
gate to use this.
--log-json-split-stream
[Alpha] In JSON format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate
to use this.
--logging-format string
--log-json-info-buffer-size quantity
[Alpha] In JSON format with split output streams, the info messages can be buffered for a while to increase performance. The default value of zero bytes disables buffering. The size can
be specified as number of bytes (512), multiples of 1000 (1K), multiples of 1024 (2Ki), or powers of those (3M, 4G, 5Mi, 6Gi). Enable the LoggingAlphaOptions feature gate to use this.
--log-json-split-stream
[Alpha] In JSON format, write error messages to stderr and info messages to stdout. The default is to write a single stream to stdout. Enable the LoggingAlphaOptions feature gate to use
this.
--logging-format string
Sets the log format. Permitted formats: "json" (gated by LoggingBetaOptions), "text". (default "text")
-v, --v Level
-v, --v Level
number for the log level verbosity
--vmodule pattern=N,...
--vmodule pattern=N,...
comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)
Common flags:
--feature-gates mapStringBool
--feature-gates mapStringBool
A set of key=value pairs that describe feature gates for alpha/experimental features. Options are:
AllAlpha=true|false (ALPHA - default=false)
AllBeta=true|false (BETA - default=false)
ContextualLogging=true|false (ALPHA - default=false)
LoggingAlphaOptions=true|false (ALPHA - default=false)
LoggingBetaOptions=true|false (BETA - default=true)
--version
--version
print binary version and exit
Controller flags:
--health-probe-bind-address string
--health-probe-bind-address string
The address the probe endpoint binds to. (default ":8081")
--kubeconfig string
--ippools-namespace string
The name of the namespace to watch for IPPools CRs (default "kube-system")
--kubeconfig string
Paths to a kubeconfig. Only required if out-of-cluster.
--leader-elect
--leader-elect
Enable leader election for controller manager. Enabling this will ensure there is only one active controller manager.
--leader-elect-namespace string
--leader-elect-namespace string
Determines the namespace in which the leader election resource will be created. (default "kube-system")
--metrics-bind-address string
--metrics-bind-address string
The address the metric endpoint binds to. (default ":8080")
--ippools-namespace string
The name of the namespace to watch for IPPools CRs. (default "kube-system")
--webhook
Enable validating webhook server as a part of the controller
```

#### IPPool CR
Expand Down Expand Up @@ -331,11 +343,28 @@ interface should have two IP addresses: one IPv4 and one IPv6. (default: network

### Deploy IPAM plugin

> _NOTE:_ This command will deploy latest dev build with default configuration
> _NOTE:_ These commands will deploy latest dev build with default configuration

The plugin can be deployed with kustomize.

Supported overlays are:

`no-webhook` - deploy without webhook

```shell
kubectl kustomize https://github.com/mellanox/nvidia-k8s-ipam/deploy/overlays/no-webhook?ref=main | kubectl apply -f -
```

`certmanager` - deploy with webhook to the Kubernetes cluster where certmanager is available

```shell
kubectl kustomize https://github.com/mellanox/nvidia-k8s-ipam/deploy/overlays/certmanager?ref=main | kubectl apply -f -
```

`openshift` - deploy with webhook to the Openshift cluster

```shell
kubectl apply -f https://raw.githubusercontent.com/Mellanox/nvidia-k8s-ipam/main/deploy/crds/nv-ipam.nvidia.com_ippools.yaml
kubectl apply -f https://raw.githubusercontent.com/Mellanox/nvidia-k8s-ipam/main/deploy/nv-ipam.yaml
kubectl kustomize https://github.com/mellanox/nvidia-k8s-ipam/deploy/overlays/openshift?ref=main | kubectl apply -f -
```

### Create IPPool CR
Expand Down
128 changes: 128 additions & 0 deletions api/v1alpha1/ippool_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,128 @@
/*
Copyright 2023, NVIDIA CORPORATION & AFFILIATES
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1_test

import (
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"

corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

"github.com/Mellanox/nvidia-k8s-ipam/api/v1alpha1"
)

var _ = Describe("Validate", func() {
It("Valid", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "192.168.0.0/16",
PerNodeBlockSize: 128,
Gateway: "192.168.0.1",
NodeSelector: &corev1.NodeSelector{
NodeSelectorTerms: []corev1.NodeSelectorTerm{{
MatchExpressions: []corev1.NodeSelectorRequirement{{
Key: "foo.bar",
Operator: corev1.NodeSelectorOpExists,
}},
}},
},
},
}
Expect(ipPool.Validate()).To(BeEmpty())
})
It("Valid - ipv6", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "2001:db8:3333:4444::0/64",
PerNodeBlockSize: 1000,
Gateway: "2001:db8:3333:4444::1",
},
}
Expect(ipPool.Validate()).To(BeEmpty())
})
It("Valid - no NodeSelector", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "192.168.0.0/16",
PerNodeBlockSize: 128,
Gateway: "192.168.0.1",
},
}
Expect(ipPool.Validate()).To(BeEmpty())
})
It("Empty object", func() {
ipPool := v1alpha1.IPPool{}
Expect(ipPool.Validate().ToAggregate().Error()).
To(And(
ContainSubstring("metadata.name"),
ContainSubstring("spec.subnet"),
ContainSubstring("spec.perNodeBlockSize"),
ContainSubstring("gateway"),
))
})
It("Invalid - perNodeBlockSize is too large", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "192.168.0.0/24",
PerNodeBlockSize: 300,
Gateway: "192.168.0.1",
},
}
Expect(ipPool.Validate().ToAggregate().Error()).
To(
ContainSubstring("spec.perNodeBlockSize"),
)
})
It("Invalid - gateway outside of the subnet", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "192.168.0.0/16",
PerNodeBlockSize: 128,
Gateway: "10.0.0.1",
},
}
Expect(ipPool.Validate().ToAggregate().Error()).
To(
ContainSubstring("spec.gateway"),
)
})
It("Invalid - invalid NodeSelector", func() {
ipPool := v1alpha1.IPPool{
ObjectMeta: metav1.ObjectMeta{Name: "test"},
Spec: v1alpha1.IPPoolSpec{
Subnet: "192.168.0.0/16",
PerNodeBlockSize: 128,
Gateway: "192.168.0.1",
NodeSelector: &corev1.NodeSelector{
NodeSelectorTerms: []corev1.NodeSelectorTerm{{
MatchExpressions: []corev1.NodeSelectorRequirement{{
Key: "foo.bar",
Operator: "unknown",
}},
}},
},
},
}
Expect(ipPool.Validate().ToAggregate().Error()).
To(
ContainSubstring("spec.nodeSelector"),
)
})
})
72 changes: 72 additions & 0 deletions api/v1alpha1/ippool_validate.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
Copyright 2023, NVIDIA CORPORATION & AFFILIATES
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1

import (
"math"
"net"

cniUtils "github.com/containernetworking/cni/pkg/utils"
"k8s.io/apimachinery/pkg/util/validation/field"
)

// Validate contains validation for the object fields
func (r *IPPool) Validate() field.ErrorList {
errList := field.ErrorList{}
if err := cniUtils.ValidateNetworkName(r.Name); err != nil {
errList = append(errList, field.Invalid(
field.NewPath("metadata", "name"), r.Name,
"invalid IP pool name, should be compatible with CNI network name"))
}
_, network, err := net.ParseCIDR(r.Spec.Subnet)
if err != nil {
errList = append(errList, field.Invalid(
field.NewPath("spec", "subnet"), r.Spec.Subnet, "is invalid subnet"))
}

if r.Spec.PerNodeBlockSize < 2 {
errList = append(errList, field.Invalid(
field.NewPath("spec", "perNodeBlockSize"),
r.Spec.PerNodeBlockSize, "must be at least 2"))
}

if network != nil && r.Spec.PerNodeBlockSize >= 2 {
setBits, bitsTotal := network.Mask.Size()
// possibleIPs = net size - network address - broadcast
possibleIPs := int(math.Pow(2, float64(bitsTotal-setBits))) - 2
if possibleIPs < r.Spec.PerNodeBlockSize {
// config is not valid even if only one node exist in the cluster
errList = append(errList, field.Invalid(
field.NewPath("spec", "perNodeBlockSize"), r.Spec.PerNodeBlockSize,
"is larger then amount of IPs available in the subnet"))
}
}
parsedGW := net.ParseIP(r.Spec.Gateway)
if len(parsedGW) == 0 {
errList = append(errList, field.Invalid(
field.NewPath("spec", "gateway"), r.Spec.Gateway,
"is invalid IP address"))
}

if network != nil && len(parsedGW) != 0 && !network.Contains(parsedGW) {
errList = append(errList, field.Invalid(
field.NewPath("spec", "gateway"), r.Spec.Gateway,
"is not part of the subnet"))
}

if r.Spec.NodeSelector != nil {
errList = append(errList, validateNodeSelector(r.Spec.NodeSelector, field.NewPath("spec"))...)
}
return errList
}
60 changes: 60 additions & 0 deletions api/v1alpha1/ippool_webhook.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
/*
Copyright 2023, NVIDIA CORPORATION & AFFILIATES
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha1

import (
"k8s.io/apimachinery/pkg/runtime"
ctrl "sigs.k8s.io/controller-runtime"
logPkg "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/webhook"
)

var logger = logPkg.Log.WithName("IPPool-validator")

// SetupWebhookWithManager registers webhook handler in the manager
func (r *IPPool) SetupWebhookWithManager(mgr ctrl.Manager) error {
return ctrl.NewWebhookManagedBy(mgr).
For(r).
Complete()
}

var _ webhook.Validator = &IPPool{}

// ValidateCreate implements webhook.Validator so a webhook will be registered for the type
func (r *IPPool) ValidateCreate() error {
logger.V(1).Info("validate create", "name", r.Name)
return r.validate()
}

// ValidateUpdate implements webhook.Validator so a webhook will be registered for the type
func (r *IPPool) ValidateUpdate(_ runtime.Object) error {
logger.V(1).Info("validate update", "name", r.Name)
return r.validate()
}

func (r *IPPool) validate() error {
errList := r.Validate()
if len(errList) == 0 {
logger.V(1).Info("validation succeed")
return nil
}
err := errList.ToAggregate()
logger.V(1).Info("validation failed", "reason", err.Error())
return err
}

// ValidateDelete implements webhook.Validator so a webhook will be registered for the type
func (r *IPPool) ValidateDelete() error {
return nil
}
13 changes: 13 additions & 0 deletions api/v1alpha1/v1alpha1_suite_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package v1alpha1_test

import (
"testing"

. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)

func TestV1alpha1(t *testing.T) {
RegisterFailHandler(Fail)
RunSpecs(t, "V1alpha1 Suite")
}
Loading

0 comments on commit c7a6c38

Please sign in to comment.