Allow setting nodeSelectors and tolerations for pulsar components

[aliabbasjaffri]

For nodes with custom labels and taints, there is currently no way to natively set nodeSelectors and tolerations on all the pods created by pulsar in a kubernetes cluster

[aliabbasjaffri]

This can be done by bumping up pulsar version. The current set version is 2.7.8 which does not support tolerations in the helm chart, however version 3.5.0 supports tolerations.

[aliabbasjaffri]

However i'm not sure what would be the side-effects of bumping the version.

However, pulsar version 2.7.8 was updated 3 years ago, any reason why it has not been updated?

[haorenfsa]

pulsar v3 chart doesn't support pulsar v2 image. So we're still using v2 for backward compatibility.
milvus-io/milvus#30812

As a walk around, You can deploy pulsar with v3 chart and then configure milvus to use that pulsar. Or you can also use milvus-operator.

[aliabbasjaffri]

Is there any timeline to when pulsar v3 would be supported by the helm chart?

[haorenfsa]

@aliabbasjaffri It will probably be in October. We are still designing a solution to introduce it without affecting compatibility.

[colinbjohnson]

@aliabbasjaffri although we would love to utilize pulsar version 3 we are using the existing milvus helm charts with pulsar and nodeSelectors.

proxy:
  nodeSelector:
    kubernetes.io/arch: amd64
  replicaCount: 1
  podMonitor:
    enabled: false
  resources:
    requests:
      memory: 2048Mi
      cpu: 1
  service:
    type: ClusterIP
  ports:
    pulsar: 6650
  configData:
    PULSAR_MEM: >
      -Xms2048m -Xmx2048m
    PULSAR_GC: >
      -XX:MaxDirectMemorySize=2048m

[aliabbasjaffri]

@colinbjohnson that is strange, how do you pass nodeSelector to pulsar_metadata component which then passes those values to bookkeeper-cluster-init and pulsar-cluster-init, given any of those are not supported by pulsar helm chart version 2.7.8.

[haorenfsa]

milvus-operator now supports pulsar-v3 chart, check this sample: https://github.com/zilliztech/milvus-operator/blob/main/config/samples/milvus_pulsar_v3.yaml

milvus-helm chart will support it later.

[haorenfsa]

related to: milvus-io/milvus#30812