[#181] Implement hotfix for SanchoNet Cardano DB Sync configuration i…

…ssue

Summary:
In response to a critical issue where the SanchoNet Cardano DB Sync
Docker image failed to correctly parse configuration files for the
SanchoNet environment, this commit introduces a hotfix. The resolution
involved modifying the entrypoint script to allow for dynamic
configuration based on external files rather than hardcoded values. This
was achieved by adding additional volumes to the DBSync container for
configuration files and execution scripts, ensuring the DBSync component
can operate with the correct settings sourced from the official
documentation.

Technical Details:
The changes include the creation of new scripts (govtool-dbsync,
govtool-entrypoint, and govtool-sanchonet) within the
scripts/govtool/dbsync directory. These scripts are designed as
one-to-one replacements for the original scripts but with modifications
to the paths, enabling the use of target configurations from a custom
volume. Furthermore, the docker-compose.sanchonet.yml file was updated
to incorporate these new scripts through volume mounts, ensuring the
DBSync component utilizes the correct configurations at runtime.

The prepare-config.sh script was also modified to prepare a scripts
directory specifically for the DBSync entrypoint, including copying the
newly created scripts into the target configuration directory. This
ensures that upon deployment, the DBSync service is equipped with all
necessary custom scripts to correctly interpret the environment-specific
configurations.

This hotfix has been tested in a development environment by triggering
deployment from a local machine, confirming its effectiveness in
resolving the configuration parsing issue without introducing any new
problems or dependencies. This approach not only addresses the immediate
problem but also enhances the flexibility of the deployment process for
the SanchoNet Cardano DB Sync service within a Docker environment.

Note: This solution is a temporary workaround pending a permanent fix
for the configuration parsing issue. It emphasizes the importance of
maintaining adaptability in deployment scripts and configurations to
accommodate changes in the underlying infrastructure or application
requirements.

Related issue: IntersectMBO/cardano-db-sync#1629

scripts/govtool/dbsync/govtool-dbsync

@@ -0,0 +1,72 @@
+#!/nix/store/vqvj60h076bhqj6977caz0pfxs6543nb-bash-5.2-p15/bin/bash
+set -euo pipefail
+
+export CARDANO_NODE_SOCKET_PATH="/node-ipc/node.socket"
+
+
+
+
+
+if [ ! -z "${RESTORE_SNAPSHOT:-}" ]; then
+SNAPSHOT_BASENAME="$(basename "$RESTORE_SNAPSHOT")"
+RESTORED_MARKER="$SNAPSHOT_BASENAME.restored"
+if [ ! -f "$RESTORED_MARKER" ]; then
+  if [[ "$RESTORE_SNAPSHOT" =~ ^https://.* ]]; then
+    echo "Downloading snapshot $RESTORE_SNAPSHOT ..."
+    /nix/store/6g6s5cqbvk1nw2y2c8lpbn95sk0w24f9-curl-8.2.1-bin/bin/curl -LOC - "$RESTORE_SNAPSHOT"
+
+    /nix/store/6g6s5cqbvk1nw2y2c8lpbn95sk0w24f9-curl-8.2.1-bin/bin/curl -LO "$RESTORE_SNAPSHOT.sha256sum"
+    /nix/store/apn3p2b40xvirn7w740wv2gy330ppib5-coreutils-9.3/bin/sha256sum -c "$SNAPSHOT_BASENAME.sha256sum"
+
+
+
+    SNAPSHOT="$SNAPSHOT_BASENAME"
+  else
+    SNAPSHOT="$RESTORE_SNAPSHOT"
+  fi
+  rm -f /var/lib/cexplorer/*.lstate
+  /nix/store/1bys3yjgm97yxy91sfy8f5law41fyif9-postgresql-setup.sh --restore-snapshot "$SNAPSHOT" /var/lib/cexplorer
+  touch $RESTORED_MARKER
+  rm -f $SNAPSHOT{,.sha256sum,.asc}
+fi
+fi
+
+
+if [[ -n "${WAIT_FOR_NODE_SYNC:-}" ]]
+then
+  until [ -S $CARDANO_NODE_SOCKET_PATH ]; do
+    echo Waiting for $CARDANO_NODE_SOCKET_PATH
+    sleep 10
+  done
+  # from scripts/postgresql-setup.sh
+  export PGHOST=$(cut -d ":" -f 1 "${PGPASSFILE}")
+  export PGPORT=$(cut -d ":" -f 2 "${PGPASSFILE}")
+  export PGDATABASE=$(cut -d ":" -f 3 "${PGPASSFILE}")
+  user=$(cut -d ":" -f 4 "${PGPASSFILE}")
+  if [ "$user" != "*" ]; then
+    export PGUSER=$user
+  fi;
+  DB_MAX_BLOCK=$(psql -h $PGHOST $PGDATABASE -U $PGUSER -t -c 'select max (block_no) from block;')
+  NODE_CUR_BLOCK=0
+  while [ $NODE_CUR_BLOCK -lt $DB_MAX_BLOCK ]; do
+    NODE_STATUS="$(cardano-cli query tip --testnet-magic $(jq '.networkMagic' /nix/store/558hxsvn739289w35s5nhy8p89iispb4-shelley-genesis.json) 2>/dev/null || true)"
+    NODE_CUR_BLOCK="$(jq -e -r '.block' <<<"$NODE_STATUS" 2>/dev/null || true)"
+    echo "Waiting... Sync progress at $NODE_CUR_BLOCK /$DB_MAX_BLOCK"
+    sleep 10
+  done
+fi
+
+mkdir -p log-dir
+if [[ "${DISABLE_LEDGER:-N}" == "Y" ]]; then
+  LEDGER_OPTS="--disable-ledger"
+else
+  LEDGER_OPTS="--state-dir /var/lib/cexplorer"
+fi
+
+exec /nix/store/hjlmcj93mxvl1wqfsmac1pj9rfk106hw-cardano-db-sync-exe-cardano-db-sync-13.2.0.0/bin/cardano-db-sync \
+    --config /configuration/db-sync-config.json \
+    --socket-path "$CARDANO_NODE_SOCKET_PATH" \
+    --schema-dir /nix/store/4xzm921ryfrvp73j86alrq8rkk14h7s6-schema \
+    ${LEDGER_OPTS} \
+    ${EXTRA_DB_SYNC_ARGS:-}
+

scripts/govtool/dbsync/govtool-entrypoint

@@ -0,0 +1,81 @@
+#!/nix/store/vqvj60h076bhqj6977caz0pfxs6543nb-bash-5.2-p15/bin/bash
+mkdir -p /configuration
+if [ ! -f /configuration/pgpass ]; then
+  /nix/store/vwp2kcwif9afpcqpm31bxlk9gh3h2r5r-gen-pgpass /run/secrets
+fi
+export PGPASSFILE=/configuration/pgpass
+
+# set up /tmp (override with TMPDIR variable)
+mkdir -p -m 1777 /tmp
+if [[ -z "$NETWORK" ]]; then
+  echo "Connecting to network specified in configuration.yaml"
+  DBSYNC=/nix/store/hjlmcj93mxvl1wqfsmac1pj9rfk106hw-cardano-db-sync-exe-cardano-db-sync-13.2.0.0/bin/cardano-db-sync
+
+  set -euo pipefail
+  if [ ! -z "${RESTORE_SNAPSHOT:-}" ]; then
+SNAPSHOT_BASENAME="$(basename "$RESTORE_SNAPSHOT")"
+RESTORED_MARKER="$SNAPSHOT_BASENAME.restored"
+if [ ! -f "$RESTORED_MARKER" ]; then
+  if [[ "$RESTORE_SNAPSHOT" =~ ^https://.* ]]; then
+    echo "Downloading snapshot $RESTORE_SNAPSHOT ..."
+    /nix/store/6g6s5cqbvk1nw2y2c8lpbn95sk0w24f9-curl-8.2.1-bin/bin/curl -LOC - "$RESTORE_SNAPSHOT"
+
+    /nix/store/6g6s5cqbvk1nw2y2c8lpbn95sk0w24f9-curl-8.2.1-bin/bin/curl -LO "$RESTORE_SNAPSHOT.sha256sum"
+    /nix/store/apn3p2b40xvirn7w740wv2gy330ppib5-coreutils-9.3/bin/sha256sum -c "$SNAPSHOT_BASENAME.sha256sum"
+
+
+
+    SNAPSHOT="$SNAPSHOT_BASENAME"
+  else
+    SNAPSHOT="$RESTORE_SNAPSHOT"
+  fi
+  rm -f /var/lib/cexplorer/*.lstate
+  /nix/store/1bys3yjgm97yxy91sfy8f5law41fyif9-postgresql-setup.sh --restore-snapshot "$SNAPSHOT" /var/lib/cexplorer
+  touch $RESTORED_MARKER
+  rm -f $SNAPSHOT{,.sha256sum,.asc}
+fi
+fi
+
+
+  if [[ "${DISABLE_LEDGER:-N}" == "Y" ]]; then
+    LEDGER_OPTS="--disable-ledger"
+  else
+    LEDGER_OPTS="--state-dir /var/lib/cexplorer"
+  fi
+
+  exec $DBSYNC --schema-dir /nix/store/4xzm921ryfrvp73j86alrq8rkk14h7s6-schema ${LEDGER_OPTS} $@
+
+elif [[ "$NETWORK" == "mainnet" ]]; then
+  echo "Connecting to network: mainnet"
+  exec /nix/store/85v1wxvwsf487ng3pc0cfw999jzgyf00-cardano-db-sync-mainnet/bin/cardano-db-sync-mainnet
+  echo "Cleaning up"
+
+elif [[ "$NETWORK" == "preprod" ]]; then
+  echo "Connecting to network: preprod"
+  exec /nix/store/3qmpvpl9p1azinz2aqp4rgmhfymn7m2y-cardano-db-sync-preprod/bin/cardano-db-sync-preprod
+  echo "Cleaning up"
+
+elif [[ "$NETWORK" == "preview" ]]; then
+  echo "Connecting to network: preview"
+  exec /nix/store/km8zmilk90kb51q1qg4vf8d72dwb3w7l-cardano-db-sync-preview/bin/cardano-db-sync-preview
+  echo "Cleaning up"
+
+elif [[ "$NETWORK" == "private" ]]; then
+  echo "Connecting to network: private"
+  exec /nix/store/674zfi0nxjxbzmi9lbjm6pfdgdxhnvv2-cardano-db-sync-private/bin/cardano-db-sync-private
+  echo "Cleaning up"
+
+elif [[ "$NETWORK" == "sanchonet" ]]; then
+  echo "Connecting to network: sanchonet"
+  exec /scripts/govtool-sanchonet
+  echo "Cleaning up"
+
+elif [[ "$NETWORK" == "shelley_qa" ]]; then
+  echo "Connecting to network: shelley_qa"
+  exec /nix/store/2dv0dff02yx5cixxiacp2ha70z53hhc3-cardano-db-sync-shelley_qa/bin/cardano-db-sync-shelley_qa
+  echo "Cleaning up"
+
+
+else
+  echo "Managed configuration for network "$NETWORK" does not exist"
+fi

scripts/govtool/dbsync/govtool-sanchonet

@@ -0,0 +1,3 @@
+#!/nix/store/vqvj60h076bhqj6977caz0pfxs6543nb-bash-5.2-p15/bin/bash
+set -euo pipefail
+/scripts/govtool-dbsync $@

scripts/govtool/docker-compose.sanchonet.yml

@@ -160,9 +160,13 @@ services:
       - postgres_password
       - postgres_user
       - postgres_db
+    entrypoint:
+      - /scripts/govtool-entrypoint
     volumes:
       - db-sync-data:/var/lib/cexplorer
       - node-ipc:/node-ipc
+      - /home/ubuntu/config/cardano-node:/configuration
+      - /home/ubuntu/config/dbsync-scripts:/scripts
     restart: always
     logging: *logging
 

scripts/govtool/prepare-config.sh

@@ -7,6 +7,13 @@ config_dir="$repo_root_dir/scripts/govtool/config"
 target_config_dir="$repo_root_dir/scripts/govtool/config/target"
 mkdir -p "$target_config_dir"
 
+# prepare a scripts directory for the DBSync entrypoint
+dbsync_scripts_dir="$target_config_dir/dbsync-scripts"
+mkdir -p "$dbsync_scripts_dir"
+cp "$repo_root_dir/scripts/govtool/dbsync/govtool-entrypoint" "$dbsync_scripts_dir/govtool-entrypoint"
+cp "$repo_root_dir/scripts/govtool/dbsync/govtool-sanchonet" "$dbsync_scripts_dir/govtool-sanchonet"
+cp "$repo_root_dir/scripts/govtool/dbsync/govtool-dbsync" "$dbsync_scripts_dir/govtool-dbsync"
+
 # cardano node config
 cardano_node_config_dir="$target_config_dir/cardano-node"
 mkdir -p "$cardano_node_config_dir"