summaryrefslogtreecommitdiff
path: root/docs/commercial-readiness.md
diff options
context:
space:
mode:
authorChia <Chia@93.nz>2026-08-06 09:29:41 +1200
committerChia <Chia@93.nz>2026-08-06 09:32:46 +1200
commit41e322c53d7b4b796eb377d0df9c29ecd10ba431 (patch)
treec730526150e55e39b822d5197e4a20318ecaa449 /docs/commercial-readiness.md
parenteadb2ffe85c43cf6fc741c9823cd28eedb4a844c (diff)
feat: complete commercial control plane, billing, auth, and model catalog
- add PostgreSQL control-plane persistence with Redis-degraded hot reload - implement prepaid balance, usage ledger, Stripe top-up and reconciliation - add registration, email verification, password reset, invitations and RBAC - support TOTP, Passkey MFA, device sessions, quotas and rate limits - add tenant billing profiles, audit logs and operational readiness checks - build authenticated admin console, Quickstart, Playground and usage analytics - add public model catalog with pricing, filtering and cost estimation - support OpenAI Responses providers and provider health failover - validate real upstream usage reporting and balance settlement
Diffstat (limited to '')
-rw-r--r--docs/commercial-readiness.md98
1 files changed, 70 insertions, 28 deletions
diff --git a/docs/commercial-readiness.md b/docs/commercial-readiness.md
index b78ae2d..6dc9064 100644
--- a/docs/commercial-readiness.md
+++ b/docs/commercial-readiness.md
@@ -7,55 +7,95 @@ commercial feature.
## What works now
-- OpenAI Chat Completions and Anthropic Messages proxying, streaming, routing,
+- OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages proxying, streaming, routing,
retry, authentication, persistent usage, prepaid billing, quotas, rate limits,
concurrent request limits, RBAC, audit logs, and a PostgreSQL-backed console.
-- Stripe-hosted Checkout with signed, idempotent Webhook crediting. The gateway
- never accepts card details and never credits a success redirect.
+- Stripe-hosted manual top-up and payment-method setup, off-session automatic
+ top-up, signed/idempotent Webhook crediting, refund/dispute handling, and
+ reconciliation. The gateway never accepts card details and never credits a
+ success redirect.
+- Tenant billing profiles persist invoice name, email, and postal address and
+ synchronize them to Stripe Customer with a stable idempotency key. Stripe
+ failures preserve the local profile for retry; tax IDs stay in Stripe-hosted
+ Checkout or Customer Portal rather than this database.
- PostgreSQL is the source of truth. Redis accelerates invalidation and shared
counters but is not required for startup, control-plane writes, billing, or
balance correctness.
- Verified self-service registration, invitation acceptance, password reset,
persistent login throttles, per-device session revocation, encrypted email
outbox delivery, TOTP with recovery codes, and WebAuthn Passkeys.
+- Tenant-scoped default/fallback model preferences and RBAC-separated low-balance
+ notification thresholds, persisted in PostgreSQL and applied by Quickstart and
+ the notification worker.
+- Per-key model restrictions, monthly spend caps, expiration, tags, and last-use
+ tracking. Restrictions are enforced by the runtime snapshot and billing
+ transaction, not only rendered by the console. The developer console also has
+ a page-memory API Playground and displays current-month settled spend, pending
+ reservations, request count, remaining cap, and last use for each key.
+- The authenticated model catalog includes a customer-safe detail view, current
+ price-version cost estimates for input/output/cache tokens, copyable model IDs,
+ developer filtering, release/price/context sorting, one-click Playground
+ selection, cURL/Python/Node examples, and actionable diagnostics for
+ authentication, balance, model, rate-limit, and provider errors.
+- The unauthenticated `/admin/models` catalog exposes only globally available
+ models and supports search, protocol/input/developer filters, release/price/context
+ sorting, model details, versioned token prices, aggregate route availability,
+ and a pre-registration cost estimate. Tenant/key allowlists, upstream model IDs,
+ provider IDs, URLs, and routing weights are excluded by a dedicated public type.
+- Quickstart colocates balance state, direct starter-key creation, OpenAI and
+ Anthropic SDK base URLs, copyable REST endpoints, environment configuration,
+ code examples, and the live Playground. A newly created key is scoped to the
+ selected project/model and is kept only in page memory after its one-time reveal.
+- Usage events open into a privacy-safe request diagnostic with the complete
+ request ID, route, retry, protocol, latency, throughput, cache-token, and
+ settlement fields; copied JSON excludes prompts, responses, and secrets.
+- Ledger-backed model cost ranking and provider performance views share the
+ Usage filters and report period-over-period charge change, success rate,
+ cache hit, P95 latency, and missing-usage exposure without sampling browser data.
+- Runtime routes use a per-instance 100-attempt availability window, response-header
+ latency EWMA, and a 3-failure/30-second circuit breaker. The customer model detail
+ compares safe provider runtime fields and prevents launching a model while every
+ route is cooling down.
+- Developers can keep automatic failover or pin a request with
+ `model-id:provider-slug`. Provider slugs are stable public identifiers returned by
+ the safe model catalog and selectable in Quickstart and Playground; pinned
+ requests never fail over to another provider, while billing and key allowlists
+ remain keyed by the canonical base model.
## Customer product gaps
### P0 before a public commercial launch
-- Production transactional-email provider selection, domain authentication,
- bounce/complaint handling, low-balance notifications, and monitoring for the
- durable mail outbox. Local development currently uses Mailpit.
-- A public model catalog and detail page containing provider/developer, release
- and retirement dates, input/output modalities, context and maximum output,
- supported parameters and protocols, regional availability, and versioned price
- dimensions.
-- Tenant and API-key model allowlists, budget alerts, low-balance notifications,
- downloadable invoices/receipts, payment history, refunds/disputes operations,
- and explicit tax handling after registrations are confirmed.
+- Production mail provider DNS authentication (SPF/DKIM/DMARC) and provider-side
+ bounce/complaint wiring remain deployment tasks; the signed feedback endpoint,
+ suppression table, low-balance notifications, retries, and dead-letter mail
+ outbox are implemented. Local development uses Mailpit.
+- Explicit tax treatment after registrations are confirmed still needs legal
+ and product sign-off. Invoice details, hosted invoice/PDF/receipt links,
+ payment history, refunds, disputes, reconciliation, and CSV ledger export are
+ implemented.
- Operational separation of the public inference listener from the management
listener, HTTPS-only cookies behind a trusted proxy, backup/restore drills,
migration rollback policy, secret rotation, and alerting for usage settlement
or Webhook backlogs.
-- A durable settlement retry/outbox and reconciliation worker. Persistence and
- balance settlement currently use bounded synchronous database calls after the
- response; a database timeout is logged but is not queued for retry, so a long
- outage can leave reservations pending or usage unbilled.
+- Enterprise identity integrations (OIDC/SAML/SCIM), custom roles, and approval
+ workflows are not included in the current console; password, invite, session,
+ TOTP, Passkey, RBAC, and audit flows are implemented.
### P1 for ZenMux-like breadth
-- OpenAI Responses, Embeddings, Images, Speech and Transcriptions; Gemini native
+- OpenAI Embeddings, Images, Speech and Transcriptions; Gemini native
APIs; rerank and other media endpoints. The existing protocol field does not
make these APIs implemented.
-- Provider health measurements per model and route: availability, first-token
- latency, throughput, error history, health-aware routing, and customer-visible
- status history.
-- Provider comparison and price ranges, cache/search/image/audio pricing units,
- lifecycle aliases and deprecation notices, searchable filters, release sorting,
- SDK examples, and copyable endpoint snippets.
-- Usage exports, cost attribution, budgets, scheduled reports, organization
- invites, custom roles, OIDC/SAML SSO, SCIM, and support impersonation with
- approval and full audit evidence.
+- Active provider probes, first-token latency, throughput-aware selection,
+ cross-instance health aggregation, and customer-visible status history. Runtime
+ circuit breaking and request-derived route health are implemented; historical
+ success, total latency, cache hit, missing usage, and cost come from the Usage Ledger.
+- Provider price ranges, non-token search/image/audio pricing units, and richer
+ deprecation notices. Provider runtime comparison and release sorting are implemented.
+- Usage exports, scheduled reports, organization invites, custom roles,
+ OIDC/SAML SSO, SCIM, and support impersonation with approval and full audit
+ evidence. Model/provider cost attribution is implemented in the console.
## Environment boundary
@@ -70,11 +110,13 @@ with mode `0600`.
| Redis URLs | `AIGW_REDIS_URL`, `AIGW_REDIS_URL_DOCKER` |
| Provider credential encryption | `AIGW_CREDENTIAL_KEY` |
| Bootstrap administrator | `AIGW_ADMIN_TOKEN` |
+| Stripe integration switch | `AIGW_STRIPE_ENABLED` |
| Stripe application key | `AIGW_STRIPE_API_KEY` |
| Stripe CLI development key | `AIGW_STRIPE_CLI_API_KEY` |
| Stripe Webhook signing secret | `AIGW_STRIPE_WEBHOOK_SECRET` |
-| Stripe result URLs | `AIGW_STRIPE_SUCCESS_URL`, `AIGW_STRIPE_CANCEL_URL` |
+| Stripe result URLs | `AIGW_STRIPE_SUCCESS_URL`, `AIGW_STRIPE_CANCEL_URL`, `AIGW_STRIPE_PORTAL_RETURN_URL` |
| Console public URL | `AIGW_PUBLIC_URL` |
+| Public inference/API URL used by customer examples | `AIGW_INFERENCE_PUBLIC_URL` |
| SMTP endpoint/sender | `AIGW_SMTP_ADDRESS`, `AIGW_SMTP_FROM_ADDRESS` |
| SMTP credentials | `AIGW_SMTP_USERNAME`, `AIGW_SMTP_PASSWORD` |
| WebAuthn RP/origins | `AIGW_WEBAUTHN_RP_ID`, `AIGW_WEBAUTHN_ORIGINS` |