← All posts
#observability
3 posts
-
Switching LLM Accounts at 88% — Manually First, Then Automatic Failover at 80%
I saw the primary account hit 88% on Grafana, worked out when it would run dry, and switched by hand at dawn when nobody was publishing. Then I gave the accounts an order so the sub only takes traffic when the primary is empty or over its limit.
-
[Messenger Load-Testing, Part 3] FCM to a Queue, Swallowed Failures to Sentry — Cleaning Up the Message-Send Handler
Part 3 of load-testing a messenger: pulling FCM push out of the message-send handler into a dedicated worker queue, and surfacing the failures that used to vanish into empty catches through Sentry. Two axes — isolating external I/O and handling error logs.
-
[Messenger Load-Testing, Part 1] Building a Reusable Load-Test Environment — Make-based IaC, Observability, and Auth Seeding
How many TPS can a messenger's send API survive, and where exactly does it break? A 7-node mirror of production, rebuilt on demand from a single Makefile target.
