CI/CD CULTURE

The Death of the Staging Environment

2024.01.10 | 6 min read | Bhavesh Kumar Parmar
CI/CD Pipeline

Our staging environment was a graveyard. It was always broken, never matched production, and every team was fighting over it. So we killed it. Here's what we built instead — and why deployment velocity jumped 40% in two months.

The Problem with Shared Staging

If you've worked on a team with more than 5 developers, you know the pain. The shared staging environment becomes a bottleneck:

The Replacement: Ephemeral Preview Environments

Every pull request now gets its own isolated environment — spun up on PR open, destroyed on PR merge/close. The full stack: application, database (seeded with anonymized data), and a unique URL.

# GitHub Actions: Spin up preview on PR
name: Preview Environment
on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  deploy-preview:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Create namespace
        run: |
          NAMESPACE="preview-pr-${{ github.event.number }}"
          kubectl create namespace $NAMESPACE --dry-run=client -o yaml | kubectl apply -f -

      - name: Deploy with Helm
        run: |
          helm upgrade --install preview-${{ github.event.number }} ./chart \
            --namespace preview-pr-${{ github.event.number }} \
            --set image.tag=${{ github.sha }} \
            --set ingress.host=pr-${{ github.event.number }}.preview.bhaveshops.in \
            --set db.seed=true

      - name: Comment PR with URL
        uses: actions/github-script@v7
        with:
          script: |
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '🚀 Preview deployed: https://pr-${{ github.event.number }}.preview.bhaveshops.in'
            })

The Cleanup Pipeline

Equally important — environments must auto-destroy. Otherwise you've just created 50 staging environments instead of one. We use a GitHub Action on PR close:

on:
  pull_request:
    types: [closed]

jobs:
  cleanup:
    runs-on: ubuntu-latest
    steps:
      - name: Destroy preview
        run: |
          NAMESPACE="preview-pr-${{ github.event.number }}"
          helm uninstall preview-${{ github.event.number }} -n $NAMESPACE
          kubectl delete namespace $NAMESPACE

Cost Control: The Surprise Win

Counter-intuitively, ephemeral environments cost less than our always-on staging. Here's why:

Metric Staging Ephemeral
Monthly AWS cost$2,400$890
Avg. environment uptime24/74.2 hrs
Deploy conflicts/week80
Prod parity~70%99%

The Cultural Shift

The biggest impact wasn't technical — it was behavioral. When every PR has a live URL, code reviews get better. Product managers click the link and give feedback before merge. QA tests on the actual branch, not a shared mess. The feedback loop tightened from days to hours.

Staging isn't dead because it was a bad idea. It's dead because isolated, on-demand environments are a better one. If your team is still fighting over a shared staging env, it's time to let go.

End of Transmission