Setting up Loki for log aggregation the right way

Loki is not Elasticsearch. The biggest mistake I see teams make is treating Loki like a full-text search engine. It's designed for label-based querying, and when you fight that model you end up with high cardinality labels, slow queries, and a storage bill that doesn't make sense.

Label design matters more than anything

In Loki, labels are used for indexing. High-cardinality labels — like request IDs, user IDs, or IP addresses — will destroy performance. Keep your label set small and stable:

# Good labels
{app="api", env="prod", namespace="production"}

# Bad labels — never do this
{request_id="abc-123", user_id="9981", ip="10.0.1.55"}

Retention and storage

By default Loki stores everything forever. Set a retention period and configure an S3 or GCS backend for the chunks.

limits_config:
  retention_period: 30d

storage_config:
  aws:
    s3: s3://your-bucket/loki
    region: ap-south-1

Promtail on Kubernetes

When running Promtail as a DaemonSet, make sure you're scraping both the container logs and the Kubernetes metadata. The auto-discovered labels from the Kubernetes API give you everything you need for most queries.