Platform Engineering

Appearance

Install/Upgrade ScopeDB Application With Helm ​

This runbook covers the complete process for installing or upgrading ScopeDB in TrueWatch clusters using Helm.

Prerequisites ​

bash
git clone git@github.com:TrueWatchTech/helmist.git
cd helmist

Overview ​

To deploy or upgrade ScopeDB in the <region-code> cluster, follow these steps:

  1. Pull the latest Helm chart
  2. Configure application settings
  3. Install or upgrade using Helm
  4. Verify deployment health

Step 1: Pull Latest Chart ​

Update the Helm chart to ensure you're deploying the latest version:

bash
make update-chart

This command:

  • Creates a new branch from main
  • Pulls the latest charts
  • Commits changes to the new branch

If you need to modify values.<region-code>.yaml, create a PR after making changes.

Step 2: Configure Application ​

Edit Values File ​

Modify the values.<region-code>.yaml file for your specific region. You can reference other values files for guidance.

Key Configuration Parameters ​

  • --fileQueue.workers: Controls concurrent write workers for scopedb-ingest
  • storageNode: Required when streamaggr is enabled; specifies guance-storage instances.
    • if the storageNode instances are sts pods in k8s, the dns name is in the format of <pod_name>.<service_name>.<namespace>.svc.cluster.local
    • if multiple storageNode instances, please separate them by comma (,) and NO SPACE after the , for example: storageNode: guancedb-cluster-guance-storage-0.guancedb-cluster-guance-storage.middleware:8401,guancedb-cluster-guance-storage-1.guancedb-cluster-guance-storage.middleware:8401

For complete configuration options, see GuanceDB Starting Arguments Reference.

Review Process ​

Submit a PR for team review once configuration changes are complete.

Step 3: Deploy Using Helm ​

Connect to Cluster ​

Configure kubectl to connect to the <region-code> cluster:

Follow the connecting kubectl to EKS documentation.

CAUTION

Verify your KUBECONFIG is connected to the correct cluster before proceeding

New Installation ​

For first-time installation:

bash
helm install guancedb ./downloaded-charts/guancedb -n guancedb \
  -f values.<region-code>.yaml

Upgrade Existing Installation ​

  1. Review changes - Install the helm diff plugin if not already installed, then check what will change:
bash
helm diff upgrade guancedb ./downloaded-charts/guancedb \
   -n guancedb -f values.<region-code>.yaml --context 2

Review the diff output for any unexpected changes.

  1. Apply upgrade:
bash
helm upgrade guancedb ./downloaded-charts/guancedb -n guancedb \
  --dependency-update -f values.<region-code>.yaml --timeout 15m

if for some reason we need to skip meta upgrade, we can do

bash
helm upgrade --no-hooks

Step 4: Verify Deployment / Rollback if Required ​

After installation or upgrade, verify the deployment is healthy:

  • Check with rancher/kubectl to make sure new pods are successfully scheduled and running
  • Check self observe ScopeDB chart on the ingest rate and query metrics

Rollback Procedures ​

If the deployment fails or encounters issues, you can rollback using Helm:

Rollback to Previous Version ​

To rollback to the immediately previous deployment:

bash
helm rollback guancedb -n guancedb

Check Deployment History ​

View previous deployments to understand available rollback options:

bash
helm history guancedb -n guancedb

Rollback to Specific Revision ​

To rollback to a specific revision (replace [revision] with the revision number from history):

bash
helm rollback guancedb [revision] -n guancedb