Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bitnami/etcd] Stop relying on files for state #75906

Open
wants to merge 16 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -35,14 +35,11 @@ etcd_env_vars=(
ETCD_ON_K8S
ETCD_INIT_SNAPSHOT_FILENAME
ETCDCTL_API
ETCD_DISABLE_STORE_MEMBER_ID
ETCD_DISABLE_PRESTOP
ETCD_NAME
ETCD_LOG_LEVEL
ETCD_LISTEN_CLIENT_URLS
ETCD_ADVERTISE_CLIENT_URLS
ETCD_INITIAL_CLUSTER
ETCD_INITIAL_CLUSTER_STATE
ETCD_LISTEN_PEER_URLS
ETCD_INITIAL_ADVERTISE_PEER_URLS
ETCD_INITIAL_CLUSTER_TOKEN
Expand Down Expand Up @@ -94,16 +91,13 @@ export ETCD_DISASTER_RECOVERY="${ETCD_DISASTER_RECOVERY:-no}"
export ETCD_ON_K8S="${ETCD_ON_K8S:-no}"
export ETCD_INIT_SNAPSHOT_FILENAME="${ETCD_INIT_SNAPSHOT_FILENAME:-}"
export ETCDCTL_API="${ETCDCTL_API:-3}"
export ETCD_DISABLE_STORE_MEMBER_ID="${ETCD_DISABLE_STORE_MEMBER_ID:-no}"
export ETCD_DISABLE_PRESTOP="${ETCD_DISABLE_PRESTOP:-no}"

# etcd native environment variables (see https://etcd.io/docs/current/op-guide/configuration)
export ETCD_NAME="${ETCD_NAME:-}"
export ETCD_LOG_LEVEL="${ETCD_LOG_LEVEL:-info}"
export ETCD_LISTEN_CLIENT_URLS="${ETCD_LISTEN_CLIENT_URLS:-http://0.0.0.0:2379}"
export ETCD_ADVERTISE_CLIENT_URLS="${ETCD_ADVERTISE_CLIENT_URLS:-http://127.0.0.1:2379}"
export ETCD_INITIAL_CLUSTER="${ETCD_INITIAL_CLUSTER:-}"
export ETCD_INITIAL_CLUSTER_STATE="${ETCD_INITIAL_CLUSTER_STATE:-}"
export ETCD_LISTEN_PEER_URLS="${ETCD_LISTEN_PEER_URLS:-}"
export ETCD_INITIAL_ADVERTISE_PEER_URLS="${ETCD_INITIAL_ADVERTISE_PEER_URLS:-}"
export ETCD_INITIAL_CLUSTER_TOKEN="${ETCD_INITIAL_CLUSTER_TOKEN:-}"
Expand Down

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
#!/bin/bash
# Copyright Broadcom, Inc. All Rights Reserved.
# SPDX-License-Identifier: APACHE-2.0

# shellcheck disable=SC1091

set -o errexit
set -o pipefail
set -o nounset

# Load libraries
. /opt/bitnami/scripts/libfs.sh
. /opt/bitnami/scripts/libos.sh
. /opt/bitnami/scripts/libetcd.sh

# Load etcd environment settings
. /opt/bitnami/scripts/etcd-env.sh

########################
# Return a comma separated list of <host>:<port> for each endpoint
# based on "initial-cluster" flag value
# ref: https://etcd.io/docs/latest/op-guide/clustering/#static
# Globals:
# ETCD_INITIAL_CLUSTER
# Arguments:
# None
# Returns:
# String
########################
endpoints_as_host_port() {
echo $ETCD_INITIAL_CLUSTER | tr -s ',' '\n' | awk -F '//' '{print $2}' | tr -s '\n' ',' | sed 's/,$//'
}

# Remove members that are not listed in ETCD_INITIAL_CLUSTER
# from the cluster before running Helm upgrades that potentially scale
# down the etcd cluster

read -r -a extra_flags <<<"$(etcdctl_auth_flags)"
is_boolean_yes "$ETCD_ON_K8S" && extra_flags+=("--endpoints=$(endpoints_as_host_port)")
debug "Listing members"
if ! current="$(etcdctl member list ${extra_flags[@]} --write-out simple | awk -F ", " '{print $3 ":" $1}')"; then
error "Unable to list members, are all members healthy?"
exit 1
fi
info "Current cluster members are: $(echo "$current" | awk -F: '{print $1}' | tr -s '\n' ',' | sed 's/,$//g')"

expected="$(echo $ETCD_INITIAL_CLUSTER | tr -s ',' '\n' | awk -F= '{print $1}')"
info "Expected cluster members are: $(echo "$expected" | tr -s '\n' ',' | sed 's/,$//g')"
juan131 marked this conversation as resolved.
Show resolved Hide resolved
read -r -a obsolete_members <<<"$(comm -23 <(echo "$current" | awk -F: '{print $1}' | sort) <(echo "$expected" | sort))"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small fix:

Suggested change
read -r -a obsolete_members <<<"$(comm -23 <(echo "$current" | awk -F: '{print $1}' | sort) <(echo "$expected" | sort))"
read -r -a obsolete_members <<<"$(comm -23 <(echo "$current" | awk -F: '{print $1}' | sort) <(echo "$expected" | sort) | tr -s '\n' ' ')"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change seems to break the script for me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, something else is breaking. Let me investigate more.

Copy link
Contributor

@juan131 juan131 Jan 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What problems are you experiencing? It works for me

Without this change only one member is removed given the obsolete_members wasn't an array but a string with the 1st obsolete member to remove, so when I did a test scaling down from 5 to 3 replicas it only removed one.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, your suggestion works. I just had to fix the loop as well. Everything should work now.

if [ ${#obsolete_members[@]} -eq 0 ]; then
info "No obsolete members to remove."
else
for member in $obsolete_members; do
info "Removing obsolete member $member"
etcdctl member remove ${extra_flags[@]} "$(echo "$current" | grep "$member" | awk -F: '{print $2}')"
done
fi
info "Pre-upgrade checks completed!"
Loading
Loading