Every DevOps engineer has a script they're embarrassed about. Usually it's the one that's running in production and only the original author understands.

Start with strict mode

#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

-e exits on error. -u treats unset variables as errors. -o pipefail catches failures in pipes.

Log everything, timestamp everything

log() {
  echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >&2
}
log "Starting deployment for ${SERVICE:-unknown}"

Make scripts idempotent

A script that can be run twice and produce the same result is a script you can safely retry. Before creating a resource, check if it exists. This is especially important for scripts that run in CI.

Use functions for anything over 20 lines

A flat 300-line script is much harder to read than ten 30-line functions with clear names. The 2am incident call is not the time to figure out what a script does.