Every DevOps engineer has a script they're embarrassed about. Usually it's the one that's running in production and only the original author understands.
Start with strict mode
#!/usr/bin/env bash set -euo pipefail IFS=$'\n\t'
-e exits on error. -u treats unset variables as errors. -o pipefail catches failures in pipes.
Log everything, timestamp everything
log() {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >&2
}
log "Starting deployment for ${SERVICE:-unknown}"
Make scripts idempotent
A script that can be run twice and produce the same result is a script you can safely retry. Before creating a resource, check if it exists. This is especially important for scripts that run in CI.
Use functions for anything over 20 lines
A flat 300-line script is much harder to read than ten 30-line functions with clear names. The 2am incident call is not the time to figure out what a script does.