Deployment & DevOps
Observability & Load Testing
Put logs, metrics and request traces in one dashboard, then hit the system with spike tests to find out where it breaks.
DevOps
Jul 2026
In numbers
- concurrent users in 5 seconds (spike test)
- 0 → 300
- concurrent users in 5 seconds (spike test)
- parts: logs, metrics, traces, dashboards
- 4
- parts: logs, metrics, traces, dashboards
- simultaneous requests in a password-reset race test
- 50
- simultaneous requests in a password-reset race test
The problem
When the app slowed down or threw errors, there was no way to see why. Logs were scattered, and nobody knew how much load the system could actually take.
What I did
- 1
One place to look
I set up a Grafana-based stack: Loki collects logs, Prometheus collects metrics (CPU, memory, request counts) and Tempo records the path each request takes through the system. Grafana shows all three on one dashboard, with access locked down.
- 2
Spike test
With k6, I ramped from 0 to 300 concurrent users in 5 seconds and held the peak for 10, watching response times and database behaviour on the dashboards.
- 3
Race condition test
While chasing a loop bug in the password-reset flow, I wrote a test that fires 50 simultaneous requests at the same reset link. Exactly one should succeed; the rest should be told the link has expired.
Result
Slowdowns and errors can now be traced from a single dashboard, and the system's limits are measured rather than guessed.
What I learned
Load tests don't just tell you whether the system holds up. They catch security bugs too, like a reset link that works twice.
Stack
- Grafana
- Loki
- Prometheus
- Tempo
- k6
- Docker