summaryrefslogtreecommitdiff
path: root/docs/runbook.md
diff options
context:
space:
mode:
authorChia <Chia@93.nz>2026-08-06 15:58:57 +1200
committerChia <Chia@93.nz>2026-08-06 15:58:57 +1200
commit3f702084d20b3c3a3ea916f3110e99b22bda60b3 (patch)
tree517f76c51025ce1ee085ea4898c60f799e5c37ea /docs/runbook.md
parent41e322c53d7b4b796eb377d0df9c29ecd10ba431 (diff)
feat: complete commercial developer workflowspublish-commercial-control-plane
Add tenant-safe usage observability, prepaid billing controls, API key lifecycle management, Embeddings metering, configurable billing alerts, and resilient provider health propagation. Harden Stripe failure handling, migrations, readiness, and the authenticated control-plane UI with end-to-end verification evidence.
Diffstat (limited to '')
-rw-r--r--docs/runbook.md9
1 files changed, 7 insertions, 2 deletions
diff --git a/docs/runbook.md b/docs/runbook.md
index ddd1a8c..6062e0c 100644
--- a/docs/runbook.md
+++ b/docs/runbook.md
@@ -44,7 +44,9 @@ deduplicated per recipient and UTC day.
## Database migrations
Run `cmd/migrate` before deploying application instances and keep `auto_migrate=false` in
-production. Migrations use a PostgreSQL advisory lock and a recorded checksum. Schema rollback
+production. Migrations use one stable PostgreSQL advisory lock across all versions and a recorded
+checksum. An already-applied matching version exits without replaying DDL, so concurrent process
+starts do not contend with normal control-plane queries. Schema rollback
is always a reviewed forward migration; restore a database backup only for whole-release
rollback after stopping writers. Never edit an already applied migration body.
@@ -55,7 +57,10 @@ database credential. Test `scripts/restore-drill.sh` into a disposable isolated
least monthly. Record row-count evidence and application smoke tests before deleting the drill.
Before each release, run `scripts/load-smoke.sh` against a non-production upstream, then
-`scripts/redis-fault-drill.sh` to prove Redis is optional and PostgreSQL polling keeps readiness.
+`scripts/redis-fault-drill.sh` to prove Redis is optional, PostgreSQL polling keeps readiness,
+and the subscriber reconnects after Redis returns. Set `AIGW_READY_URL`, `AIGW_REDIS_CONTAINER`,
+and, when shared provider health is enabled, `AIGW_METRICS_URL`; the metrics assertion verifies
+`aigw_provider_health_shared_connected` transitions from 1 to 0 and back to 1.
Exercise PostgreSQL failover separately and verify pending settlement jobs resume without duplicate
ledger entries. Archive the command output with the release evidence.