◆ OPSDECK ENV=prod REGION=us-east-1 CLUSTER=primary
UPTIME.30D99.987%
P95142ms
ERR.BUDGET63%
INCIDENTS01 OPEN
UTC14:32:08
┌─ aws.budget eu-west-1
FETCHING...
┌─ gcp.credits free tier
FETCHING...
┌─ spaceship.balance billing
FETCHING...
┌─ webshare.quota bandwidth
FETCHING...
┌─ requests.per_sec ─ live window=6h res=1m
RPS · NOW
13,402
Δ +4.1% vs prev
BREAKDOWN
/api/checkout4.2k
/api/auth3.1k
/api/search2.8k
/api/billing1.9k
/static/*1.4k
┌─ deploys.recentn=12
SHA       SERVICE          AUTHOR        AGE     STATUS
a91f4c2   checkout-api     maren.vos     12m     OK
7e3b1d8   web-frontend     arjun.r       01h     OK
4c0a92e   auth-service     lin.h         03h     OK
ff21ae0   billing-worker   tomas.k       05h     DEGRADED
28d1c33   search-indexer   priya.s       08h     OK
91ab47e   notifications    arjun.r       11h     OK
6f0e2bd   media-worker     lin.h         14h     ROLLED-BACK
┌─ incident.open ● SEV-3
00:24:11 ELAPSED
Elevated 5xx on
billing-worker
OWNER payments
STATE investigating
IMPACT ~3% of charges
STARTED 14:08 UTC
ON-CALL @maren.vos
LAST UPDATE
"Identified a connection pool leak in stripe client. Patch in review."
┌─ Budgets30d
checkout availability78%
checkout p99 < 500ms63%
auth availability91%
search p95 < 800ms24%
billing availability8%
┌─ logs.tail │ severity≥WARN ▶ STREAMING
14:31:58.412WARNdb.poolpool-exhausted db-replica-3 (87/88)
14:31:42.901ERRstripecharge timeout after 8000ms
14:30:11.220WARNratelimit hit org_8821 (429)
14:28:04.114ERRstripecharge timeout after 8000ms
14:27:33.501WARNcachemiss spike on session-store (+38%)
14:26:02.118WARNk8spod billing-worker-7d restarted (oom)
14:24:55.770INFOdeploycheckout-api a91f4c2 promoted
┌─ services.matrixn=38
LEGEND
■ OK · 35
■ WARN · 02
■ DOWN · 01