summaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorChia <Chia@93.nz>2026-08-06 09:29:41 +1200
committerChia <Chia@93.nz>2026-08-06 09:32:46 +1200
commit41e322c53d7b4b796eb377d0df9c29ecd10ba431 (patch)
treec730526150e55e39b822d5197e4a20318ecaa449 /docs
parenteadb2ffe85c43cf6fc741c9823cd28eedb4a844c (diff)
feat: complete commercial control plane, billing, auth, and model catalog
- add PostgreSQL control-plane persistence with Redis-degraded hot reload - implement prepaid balance, usage ledger, Stripe top-up and reconciliation - add registration, email verification, password reset, invitations and RBAC - support TOTP, Passkey MFA, device sessions, quotas and rate limits - add tenant billing profiles, audit logs and operational readiness checks - build authenticated admin console, Quickstart, Playground and usage analytics - add public model catalog with pricing, filtering and cost estimation - support OpenAI Responses providers and provider health failover - validate real upstream usage reporting and balance settlement
Diffstat (limited to '')
-rw-r--r--docs/architecture.md84
-rw-r--r--docs/commercial-readiness.md98
2 files changed, 139 insertions, 43 deletions
diff --git a/docs/architecture.md b/docs/architecture.md
index c17c65e..f8e617f 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -29,35 +29,35 @@ flowchart LR
## 热路径
1. 入口生成不可预测的请求 ID,并通过 Bearer 或 `x-api-key` 解析客户身份。
-2. 身份包含 `key_id`、`tenant_id`、`project_id` 和 scopes;控制面模式使用 PostgreSQL 摘要快照,代理只依赖 `Authenticator` 接口。
+2. 身份包含 `key_id`、`tenant_id`、`project_id`、scopes、密钥模型白名单、月度上限和过期时间;控制面模式使用 PostgreSQL 摘要快照,代理只依赖 `Authenticator` 接口。
3. 请求体在配置上限内读取一次,以提取公开模型名并支持故障转移时重放。
4. 项目策略从 PG 热更新快照读取。Redis Lua 原子占用 RPM、估算 TPM 和并发额度;Redis 不可用时退回本机窗口。
-5. Router 按协议过滤路由,先按优先级分组,再在同级内按权重选择首选上游。
-6. 计费开启时,请求进入上游前在 PostgreSQL 原子检查月度消费、冻结保守估算额度;余额不足返回 402,月度额度耗尽返回 429。
-7. Provider Adapter 重写上游模型名和凭证,使用进程级共享 Transport 发送请求。上游返回成功头后即锁定路由;SSE 逐块 flush。
+5. Router 按协议过滤路由并跳过仍处于冷却期的 route,再按优先级分组、在同级内按权重选择首选上游。请求使用 `model:provider-slug` 时,基础模型先经过租户/API Key 白名单校验,然后只保留该公开 slug 的兼容 route,绝不跨供应商回退;全部匹配 route 熔断时直接返回可重试的 503。
+6. 计费开启时,请求进入上游前在锁定钱包的 PostgreSQL 事务中同时检查项目月度额度、API Key 月度额度和未结冻结,再冻结保守估算额度;余额不足返回 402,任一月度额度耗尽返回 429。
+7. Provider Adapter 重写上游模型名和凭证,使用进程级共享 Transport 发送请求。每次上游尝试记录响应头延迟与可重试失败;连续 3 次连接失败或 429/502/503/504 后熔断该模型/供应商 route 30 秒。上游返回成功头后即锁定路由;SSE 逐块 flush。
8. 请求结束后释放并发 lease,并用 `request_id` 幂等持久化 UsageEvent、更新月度汇总、按实际 token 结算;结构化日志仅是异步副本。
## 已固定的扩展边界
| 边界 | 当前实现 | 下一阶段替换 |
| --- | --- | --- |
-| 客户身份 | PostgreSQL 快照、内存 SHA-256 索引 | SSO/OIDC、SCIM、模型 allowlist |
+| 客户身份 | PostgreSQL 快照、内存 SHA-256 索引、密钥过期/模型白名单/月度上限 | IP/CIDR 策略、短期服务身份 |
| 控制台权限 | 邮箱验证/邀请/重置、登录限流、设备会话、TOTP/恢复码、Passkey、CSRF、六角色 RBAC、租户 SQL scope、审计日志 | SSO/OIDC、SCIM、组织级 MFA 策略、自定义角色、审批流 |
-| 权限 | `Principal.Scopes` 中的 `inference` + 控制台 RBAC | ABAC、IP 与模型策略 |
-| 模型目录 | PostgreSQL 快照 + 可选 Redis generation 广播 + PG 轮询兜底 | 版本化控制面、热更新、灰度发布 |
-| 路由 | priority + weighted selection + failover | 健康评分、延迟 EWMA、成本/质量策略、熔断 |
+| 权限 | `Principal.Scopes` 中的 `inference`、API Key 模型限制 + 控制台 RBAC | ABAC、IP 与条件策略 |
+| 模型目录 | PostgreSQL 快照 + 可选 Redis generation 广播 + PG 轮询兜底;租户安全目录 API、供应商运行状态与 Quickstart | 公开 SEO 目录、状态历史、灰度发布 |
+| 路由 | priority + weighted selection + failover + `model:provider-slug` 固定供应商 + 真实流量滑动窗口 + 响应头延迟 EWMA + 熔断 | 主动探测、TTFT/吞吐量、成本/质量策略、跨实例健康聚合 |
| 计量 | PostgreSQL UsageEvent + 月度汇总 + 异步日志副本 | 分区表、持久消息流、供应商账单对账 |
-| 计费 | 预付余额、冻结/结算、不可变流水、Stripe Checkout | 价格版本、退款/争议、信用额度、Metronome 企业合同 |
+| 计费 | 版本价格、预付余额、冻结/结算、不可变流水、Stripe 手动/自动充值、退款/争议/对账 | 信用额度、合同价、Metronome 企业合同 |
| 限流 | Redis Lua 全局 RPM/估算 TPM/并发,故障时本机降级;PG 月度消费配额 | 滑动窗口、层级策略、边缘 token bucket |
-| 协议 | 同协议透传 | 规范化 IR + OpenAI/Anthropic/Google 双向转换 |
+| 协议 | Chat Completions、Responses、Anthropic Messages 同 wire API 透传 | 规范化 IR + OpenAI/Anthropic/Google 双向转换 |
## 计费数据原则
真实计费不依赖请求日志。当前以 `request_id` 作为 reservation、usage 和扣费幂等键,价格快照随冻结记录保存,账本流水不可变。余额使用百万分之一币种单位,所有变更在锁定 tenant wallet 的 PostgreSQL 事务中完成。
-Stripe 充值使用 Checkout Session:本地先创建 top-up order,Stripe 请求使用 order ID 作为幂等键;Webhook 验证签名后再次核对 event ID、order ID、session ID、金额和币种。浏览器成功跳转不具有入账权威性。
+Stripe 手动充值使用 Checkout Session:本地先创建 top-up order,Stripe 请求使用 order ID 作为幂等键;Webhook 验证签名后再次核对 event ID、order ID、session ID、金额和币种。自动充值使用 Checkout Setup Session 保存支付方式,低余额 worker 再创建 off-session PaymentIntent;同一订单的同步成功、Webhook 与对账共享幂等账本来源。浏览器成功跳转不具有入账权威性。
-流式请求的 usage 可能只在最后事件出现。当前 observer 会在线解析 OpenAI/Anthropic SSE 中的 usage;如果上游不返回 usage,事件中的 token 为零。接入商业扣费前,应为每种上游建立有测试的 usage normalizer,并使用上游账单进行日对账。
+流式请求的 usage 可能只在最后事件出现。当前 observer 会在线解析 Chat Completions、Responses 和 Anthropic SSE 中的 usage,并把 OpenAI details 中属于总 input 子集的缓存 token 拆成互斥计费桶;Anthropic 独立缓存字段不做减法。如果计费上游不返回 usage,请求会进入 `metering_failed` 并保持余额冻结,而不是按零费用结算。每种上游仍需使用供应商账单做日对账。
## 扩容方式
@@ -67,6 +67,60 @@ Stripe 充值使用 Checkout Session:本地先创建 top-up order,Stripe 请
- 请求体默认最多 16 MiB,响应仅保留最多 64 KiB 用于非流式 usage 提取;正文直接传输。
- 余额冻结和结算当前各需要一次 PostgreSQL 事务,以正确性优先。高吞吐阶段可把计费拆成独立服务并做账户分片,但不能用最终一致缓存替代权威账本事务。
+## 开发者上手路径
+
+租户控制台的 Quickstart 首屏把充值、API key、可用模型和首次成功请求显示为
+可追踪的四步状态。模型目录通过 `GET /admin/api/developer/models` 返回公开元数据和
+当前生效价格,只保留该租户可用的模型与 wire API;它不会返回供应商地址、上游模型名、
+路由权重或 allowlist。控制台可按开发者、协议和输入模态过滤,并按发布时间、价格或上下文
+排序。`GET /admin/api/developer/config` 返回由
+`AIGW_INFERENCE_PUBLIC_URL` 注入的推理地址和协议端点,控制台据此生成 cURL、Python
+和 Node.js 示例。新建客户密钥仍只在创建响应和一次性对话框中出现,代码示例默认使用
+`$AIGW_API_KEY`,不会把明文密钥写入本地存储或 HTML。
+
+同一 Quickstart 页面提供 API Playground。它使用所选客户 Key 直接调用配置中的公开推理
+地址,因此请求仍经过正式鉴权、模型限制、路由、余额冻结、结算与 Usage Ledger。Key 只
+存在于当前页面内存。分离 listener 时,推理服务仅允许从 `AIGW_PUBLIC_URL` 派生出的精确
+Origin,且不接受浏览器 credentials,只向页面暴露 `X-AIGW-Request-ID`。
+
+模型详情由同一份租户安全目录数据渲染,不暴露供应商凭证、内部路由或上游模型名。成本
+估算按当前版本化单价分别计算输入、输出、缓存读取和缓存写入 token;它是请求前预算工具,
+实际扣款仍只认 Usage Ledger。Playground 对 `401/403`、`402`、`404`、`429` 和 `5xx`
+提供不同的恢复入口,同时原样保留结构化错误与 request ID,便于开发者和支持人员定位。
+目录还把当前内存 route 状态按模型投影为 `online`、`degraded` 或 `unavailable`,详情只返回
+供应商公开 slug、显示名、协议、近期可用率、响应头延迟、样本量和熔断恢复时间。Quickstart
+和 Playground 默认使用自动路由,也可以把所选 slug 编入 `model:provider-slug` 来固定供应商。
+`GET /v1/models` 和 Anthropic 模型列表同样只公开 slug 与 wire API,不返回内部 UUID、URL、
+凭证、上游模型名或权重。统计窗口是当前实例
+最近 100 次真实尝试;它不冒充主动健康检查、首 token 延迟、持久状态历史或全局 SLA。
+
+Usage 页不保存 prompt 或响应正文。单请求详情仅把已持久化的身份边界、模型路由、协议、
+重试、延迟、token、缓存和结算字段组成可复制诊断 JSON,因此既能支持工单排障,也不会
+把客户输入扩大为新的控制面敏感数据面。
+
+`GET /admin/api/usage/analytics` 直接聚合 PostgreSQL Usage Ledger,并复用 Usage 页的租户、
+项目、API Key、模型、状态和时间过滤。结果按模型与供应商返回请求量、成功率、token、
+缓存命中、费用、未收金额、缺失 usage 和 P95 延迟,同时用等长前一周期计算费用变化。
+它不在推理热路径执行,也不从浏览器当前加载的有限请求列表推算财务数据。
+
+Quickstart 的 starter key 表单复用正式 `POST /admin/api/keys` 写入链路,为当前租户、所选
+项目和当前模型生成仅含 `inference` scope 的 Key。`api_keys(project_id, tenant_id)` 到
+`projects(id, tenant_id)` 的复合外键保证项目不能跨租户绑定;明文 Key 仍只在创建响应中
+返回一次,随后仅保存在页面内存并填入 Playground。连接面板直接消费
+`GET /admin/api/developer/config`,集中输出两个 SDK Base URL、四个推理/模型端点和不含
+真实凭证的环境变量模板。
+
+密钥表单可设置模型白名单、月度金额上限、到期时间和标签。白名单与密钥摘要在同一个
+PostgreSQL 事务内创建,任何未知模型都会让事务整体回滚。`last_used_at` 在 Usage/结算
+事务内单调更新;过期时间既用于快照过滤,也在每次鉴权时检查,避免长轮询间隔延迟失效。
+密钥列表还通过 key/time 索引从 PostgreSQL 返回本月已结算费用、待结算冻结和请求数。
+
+`tenant_preferences` 保存租户默认模型、不同的 fallback 模型和低余额提醒阈值。
+`GET /admin/api/developer/preferences` 返回当前租户值;两个独立的写接口分别要求
+`developer.preferences.write` 与 `billing.preferences.write`,因此 developer 与 billing 角色
+不能越权修改对方的设置。保存默认/fallback 时服务端会重新验证该模型当前对租户可见、
+未退役且至少存在一条启用路由。邮件扫描直接读取 PostgreSQL 中的租户阈值,不依赖 Redis。
+
## 管理面安全
bootstrap token 只映射为 `platform_admin`,用于首次建号和故障恢复,不是日常用户凭证。租户注册在同一 PostgreSQL 事务内创建租户、默认项目、钱包和待验证的 `tenant_admin` 账号;邮件动作使用仅保存摘要的一次性 token,邮件正文在 outbox 中加密。密码使用 PBKDF2-HMAC-SHA-256 哈希,TOTP secret、Passkey credential 和 WebAuthn challenge 使用 AES-256-GCM 加密。登录创建 HttpOnly、SameSite 会话 Cookie,并为所有写请求校验独立 CSRF Cookie/header;改密、密码重置和撤销成员会立即失效旧会话。平台角色没有 `tenant_id`,租户角色必须绑定一个 tenant。所有管理 API 在 handler 执行前校验 permission,租户过滤在 SQL 查询或资源所有权检查中完成,前端隐藏菜单不承担安全职责。
@@ -76,7 +130,7 @@ bootstrap token 只映射为 `platform_admin`,用于首次建号和故障恢å¤
## 建议的后续顺序
1. 将管理监听端口与公网推理端口分离,并为企业客户接入 OIDC/SAML、SCIM 与组织级强制 MFA 策略。
-2. 增加价格版本、退款/冲正、Stripe dispute 处理和供应商日账单对账。
+2. 增加供应商日账单对账和合同价/信用额度。
3. 为不返回 usage 的上游增加可靠 token 计算器,并监控 `uncollected_micros`。
-4. 把 UsageEvent 做时间分区和归档,增加 CSV 导出与对账作业。
-5. 主动健康检查、熔断、延迟 EWMA 和按成本路由。
+4. 把 UsageEvent 做时间分区和归档,增加定时导出与报告。
+5. 增加主动健康检查、TTFT/吞吐量采样、跨实例状态聚合和按成本/性能路由。
diff --git a/docs/commercial-readiness.md b/docs/commercial-readiness.md
index b78ae2d..6dc9064 100644
--- a/docs/commercial-readiness.md
+++ b/docs/commercial-readiness.md
@@ -7,55 +7,95 @@ commercial feature.
## What works now
-- OpenAI Chat Completions and Anthropic Messages proxying, streaming, routing,
+- OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages proxying, streaming, routing,
retry, authentication, persistent usage, prepaid billing, quotas, rate limits,
concurrent request limits, RBAC, audit logs, and a PostgreSQL-backed console.
-- Stripe-hosted Checkout with signed, idempotent Webhook crediting. The gateway
- never accepts card details and never credits a success redirect.
+- Stripe-hosted manual top-up and payment-method setup, off-session automatic
+ top-up, signed/idempotent Webhook crediting, refund/dispute handling, and
+ reconciliation. The gateway never accepts card details and never credits a
+ success redirect.
+- Tenant billing profiles persist invoice name, email, and postal address and
+ synchronize them to Stripe Customer with a stable idempotency key. Stripe
+ failures preserve the local profile for retry; tax IDs stay in Stripe-hosted
+ Checkout or Customer Portal rather than this database.
- PostgreSQL is the source of truth. Redis accelerates invalidation and shared
counters but is not required for startup, control-plane writes, billing, or
balance correctness.
- Verified self-service registration, invitation acceptance, password reset,
persistent login throttles, per-device session revocation, encrypted email
outbox delivery, TOTP with recovery codes, and WebAuthn Passkeys.
+- Tenant-scoped default/fallback model preferences and RBAC-separated low-balance
+ notification thresholds, persisted in PostgreSQL and applied by Quickstart and
+ the notification worker.
+- Per-key model restrictions, monthly spend caps, expiration, tags, and last-use
+ tracking. Restrictions are enforced by the runtime snapshot and billing
+ transaction, not only rendered by the console. The developer console also has
+ a page-memory API Playground and displays current-month settled spend, pending
+ reservations, request count, remaining cap, and last use for each key.
+- The authenticated model catalog includes a customer-safe detail view, current
+ price-version cost estimates for input/output/cache tokens, copyable model IDs,
+ developer filtering, release/price/context sorting, one-click Playground
+ selection, cURL/Python/Node examples, and actionable diagnostics for
+ authentication, balance, model, rate-limit, and provider errors.
+- The unauthenticated `/admin/models` catalog exposes only globally available
+ models and supports search, protocol/input/developer filters, release/price/context
+ sorting, model details, versioned token prices, aggregate route availability,
+ and a pre-registration cost estimate. Tenant/key allowlists, upstream model IDs,
+ provider IDs, URLs, and routing weights are excluded by a dedicated public type.
+- Quickstart colocates balance state, direct starter-key creation, OpenAI and
+ Anthropic SDK base URLs, copyable REST endpoints, environment configuration,
+ code examples, and the live Playground. A newly created key is scoped to the
+ selected project/model and is kept only in page memory after its one-time reveal.
+- Usage events open into a privacy-safe request diagnostic with the complete
+ request ID, route, retry, protocol, latency, throughput, cache-token, and
+ settlement fields; copied JSON excludes prompts, responses, and secrets.
+- Ledger-backed model cost ranking and provider performance views share the
+ Usage filters and report period-over-period charge change, success rate,
+ cache hit, P95 latency, and missing-usage exposure without sampling browser data.
+- Runtime routes use a per-instance 100-attempt availability window, response-header
+ latency EWMA, and a 3-failure/30-second circuit breaker. The customer model detail
+ compares safe provider runtime fields and prevents launching a model while every
+ route is cooling down.
+- Developers can keep automatic failover or pin a request with
+ `model-id:provider-slug`. Provider slugs are stable public identifiers returned by
+ the safe model catalog and selectable in Quickstart and Playground; pinned
+ requests never fail over to another provider, while billing and key allowlists
+ remain keyed by the canonical base model.
## Customer product gaps
### P0 before a public commercial launch
-- Production transactional-email provider selection, domain authentication,
- bounce/complaint handling, low-balance notifications, and monitoring for the
- durable mail outbox. Local development currently uses Mailpit.
-- A public model catalog and detail page containing provider/developer, release
- and retirement dates, input/output modalities, context and maximum output,
- supported parameters and protocols, regional availability, and versioned price
- dimensions.
-- Tenant and API-key model allowlists, budget alerts, low-balance notifications,
- downloadable invoices/receipts, payment history, refunds/disputes operations,
- and explicit tax handling after registrations are confirmed.
+- Production mail provider DNS authentication (SPF/DKIM/DMARC) and provider-side
+ bounce/complaint wiring remain deployment tasks; the signed feedback endpoint,
+ suppression table, low-balance notifications, retries, and dead-letter mail
+ outbox are implemented. Local development uses Mailpit.
+- Explicit tax treatment after registrations are confirmed still needs legal
+ and product sign-off. Invoice details, hosted invoice/PDF/receipt links,
+ payment history, refunds, disputes, reconciliation, and CSV ledger export are
+ implemented.
- Operational separation of the public inference listener from the management
listener, HTTPS-only cookies behind a trusted proxy, backup/restore drills,
migration rollback policy, secret rotation, and alerting for usage settlement
or Webhook backlogs.
-- A durable settlement retry/outbox and reconciliation worker. Persistence and
- balance settlement currently use bounded synchronous database calls after the
- response; a database timeout is logged but is not queued for retry, so a long
- outage can leave reservations pending or usage unbilled.
+- Enterprise identity integrations (OIDC/SAML/SCIM), custom roles, and approval
+ workflows are not included in the current console; password, invite, session,
+ TOTP, Passkey, RBAC, and audit flows are implemented.
### P1 for ZenMux-like breadth
-- OpenAI Responses, Embeddings, Images, Speech and Transcriptions; Gemini native
+- OpenAI Embeddings, Images, Speech and Transcriptions; Gemini native
APIs; rerank and other media endpoints. The existing protocol field does not
make these APIs implemented.
-- Provider health measurements per model and route: availability, first-token
- latency, throughput, error history, health-aware routing, and customer-visible
- status history.
-- Provider comparison and price ranges, cache/search/image/audio pricing units,
- lifecycle aliases and deprecation notices, searchable filters, release sorting,
- SDK examples, and copyable endpoint snippets.
-- Usage exports, cost attribution, budgets, scheduled reports, organization
- invites, custom roles, OIDC/SAML SSO, SCIM, and support impersonation with
- approval and full audit evidence.
+- Active provider probes, first-token latency, throughput-aware selection,
+ cross-instance health aggregation, and customer-visible status history. Runtime
+ circuit breaking and request-derived route health are implemented; historical
+ success, total latency, cache hit, missing usage, and cost come from the Usage Ledger.
+- Provider price ranges, non-token search/image/audio pricing units, and richer
+ deprecation notices. Provider runtime comparison and release sorting are implemented.
+- Usage exports, scheduled reports, organization invites, custom roles,
+ OIDC/SAML SSO, SCIM, and support impersonation with approval and full audit
+ evidence. Model/provider cost attribution is implemented in the console.
## Environment boundary
@@ -70,11 +110,13 @@ with mode `0600`.
| Redis URLs | `AIGW_REDIS_URL`, `AIGW_REDIS_URL_DOCKER` |
| Provider credential encryption | `AIGW_CREDENTIAL_KEY` |
| Bootstrap administrator | `AIGW_ADMIN_TOKEN` |
+| Stripe integration switch | `AIGW_STRIPE_ENABLED` |
| Stripe application key | `AIGW_STRIPE_API_KEY` |
| Stripe CLI development key | `AIGW_STRIPE_CLI_API_KEY` |
| Stripe Webhook signing secret | `AIGW_STRIPE_WEBHOOK_SECRET` |
-| Stripe result URLs | `AIGW_STRIPE_SUCCESS_URL`, `AIGW_STRIPE_CANCEL_URL` |
+| Stripe result URLs | `AIGW_STRIPE_SUCCESS_URL`, `AIGW_STRIPE_CANCEL_URL`, `AIGW_STRIPE_PORTAL_RETURN_URL` |
| Console public URL | `AIGW_PUBLIC_URL` |
+| Public inference/API URL used by customer examples | `AIGW_INFERENCE_PUBLIC_URL` |
| SMTP endpoint/sender | `AIGW_SMTP_ADDRESS`, `AIGW_SMTP_FROM_ADDRESS` |
| SMTP credentials | `AIGW_SMTP_USERNAME`, `AIGW_SMTP_PASSWORD` |
| WebAuthn RP/origins | `AIGW_WEBAUTHN_RP_ID`, `AIGW_WEBAUTHN_ORIGINS` |