AI usage accounting, guardrails and metering

Every model call in ErmisAI passes through the same three-part path: an admission check before the call, a telemetry and accounting middleware around it, and a durable ai_usage_ledger row after it. Overage, if a tenant has it enabled, is derived from that ledger and pushed to Polar through an outbox.

This page covers the accounting side. For provider modes, stages, model resolution and prompts, see AI runtime.

All paths are relative to the app repository root (the directory is named hermesai; the package is ermisai).

What is billable

Three surfaces charge a tenant's monthly AI capacity. Nothing else does.

The surface list is the pgEnum ai_surface (src/lib/ai/invocation-context.ts:7-14, src/lib/db/schema.ts:120).

billable_cost_cents_usd is set to the resolved cost only when billable && billingScope === 'tenant'; otherwise it is written as 0 (src/lib/ai/usage-accounting.ts:525-528). That column is the sole input to envelope enforcement, overage metering, and the platform spend cap.

The invocation context

Every call site wraps its model call in runWithAiInvocationContext(...) (src/lib/ai/invocation-context.ts:60-78), an AsyncLocalStorage store carrying invocationId, tenantScopeId, userId, actorType, billingScope, billable, surface, stage, storyId, chatId, clusterId, workflowMode and routeKey.

The guardrail and the provider middleware both decide "is this tenant spend?" through one shared predicate, isBillableTenantInvocation (src/lib/ai/invocation-context.ts:49-57), so the reservation taken at admission and the release taken at the terminal event can never disagree.

A model call with no context still runs. The middleware emits an ai_model_invocation_warning carrying the literal warning code unscoped_invocation (src/lib/ai/providers.ts:542-548, :733-739), and the ledger row is recorded with surface: 'unscoped'. The admin panel surfaces the count as the Unscoped warnings metric card.

Admission: assertAiInvocationAllowed

src/lib/ai/usage-guardrails.ts:400-627. Runs before the model call, in exactly this order.

projectedSpendCents = round((summary.billableCostCentsUsd + inFlightReservedCents) * usdToEurRate)

Per-plan limits

Hardcoded per billing plan in src/lib/ai/usage-policy.ts:27-83. These are not configurable per tenant.

The rest of the policy is derived from the subscription (usage-policy.ts:147-200):

• includedCentsMonthly = max(subscription.costEnvelopePerMonthCents, 0) — already standing-aware and override-aware from the billing DTO.
• aiMonthlyHardLimitCents = subscription.aiMonthlyHardLimitCents ?? (aiOverageEnabled ? null : includedCentsMonthly). With overage off — the default for every plan — the envelope is the hard limit.
• envelopeConfigured is false only when the envelope is 0 and there is no explicit hard limit and overage is off. That is the enterprise "operator has not set an override yet" case.
• internal_unlimited short-circuits to aiOverageEnabled: true, aiMonthlyHardLimitCents: null, and an envelope floored at 1 000 000 000 cents (:159-174).

The billing window is recomputed, not read

resolveEffectiveBillingPeriod(currentPeriodEndIso, now) (usage-policy.ts:106-145) does not trust the stored current_period_end. It treats the anchor's day-of-month as the billing anniversary and walks in clamped calendar months to the single window where start <= now < end. An unparseable anchor falls back to a trailing 30 days.

This exists because the stored value is routinely outside the current window: free tenants get a one-shot now + 30 days bootstrap that no Polar webhook ever advances, paid tenants can miss a renewal webhook, and annual plans store an end up to a year ahead. Reading it literally put all new usage outside the window and silently disabled the envelope.

Guardrail errors

All defined in src/lib/ai/usage-errors.ts. Messages are hardcoded English and are not routed through next-intl, so a newsroom on a non-English locale sees the raw string.

The user-spend variant of ai_user_window_limit_reached carries a different message on the same code: "AI usage spend limit reached for the current user window." (usage-guardrails.ts:543-547).

The three guardrail modes

aiGuardrailsMode is persisted admin runtime config, seeded from ERMIS_AI_GUARDRAILS_MODE (default enforce).

A disabled mode resolved inside a Vercel deployment emits exactly one loud stderr warning at config load (src/lib/platform/ai-runtime-config.ts:288-316). It is a warning, not an alert. Monthly limits, overage gating, standing blocks and every rate limit are bypassed.

The ledger

Table ai_usage_ledger, src/lib/db/schema.ts:785-834. One row per AI invocation.

Three indexes (schema.ts:828-832): (tenant_scope_id, finished_at), (user_id, finished_at), and (finished_at) alone — the last one exists because platform-wide aggregates filter on finished_at with an unconstrained leading column, which the composite indexes cannot serve.

Writes are gated on aiUsageLedgerEnabled in the runtime config. recordBlockedAiInvocation additionally returns null when there is no invocation context (usage-accounting.ts:537-556), so a blocked unscoped call produces no row.

Cost resolution

resolveActualOrEstimatedCost (src/lib/ai/usage-accounting.ts:415-459), in order:

1. Gateway-reported actual USD cost from provider metadata → cost_source: 'actual'.
2. Otherwise the estimate from the hand-maintained price cards → cost_source: 'estimated', but only when it is greater than zero.
3. Otherwise zero → cost_source: 'none'.

The estimator peels cache-read and cache-write tokens off the provider-reported input total before charging the base input rate, because the reported total is inclusive of both (provider-pricing.ts:175-177).

Durability: why billable writes block the response

persistAiUsageLedgerEntryWithDurability (src/lib/ai/providers.ts:452-514) splits on isBillableTenantInvocation:

• Non-billable — the ledger write is scheduled post-response via the accounting task scheduler.
• Billable tenant — the row is built once, in request context (capturing a stable id and finishedAt), then inserted inline and awaited within the request or stream lifetime.

The reason is stated in the source: serverless can terminate a post-response task when the response ends, and this is the spend that drives the envelope, the hard limit, and Polar metering. Losing it is worse than adding latency. A failed inline write is logged loudly but never fails the user's generation — the content has already been produced.

On insert failure the already-built row (same id) is pushed to the Redis retry queue by enqueueFailedAiLedgerEntry, and the middleware emits an ai_model_invocation_warning with a durable_accounting_failed:* warning code.

The in-flight reservation is released in the finally of that path, through a synchronous once-claim (claimAiInFlightReservationRelease, invocation-context.ts:92-101) shared with the compose cache-hit path, so two terminal events cannot both decrement and consume a different call's reserved cents.

The ledger retry queue

src/lib/ai/ledger-retry-queue.ts. Upstash Redis lists — deliberately a different plane from the failing dependency, because putting the retry queue in the same Postgres that just rejected the write provides no resilience.

MAX_ATTEMPTS = 10, default redrive batch 100 (:23-24). Claims use LMOVE rather than RPOP so a crash after the claim leaves the entry recoverable. The contract is at-least-once, which is safe because the payload carries the stable client-supplied id: a re-insert is a primary-key no-op, never a second counted row.

purgeQueuedLedgerEntriesForTenantScope (:288) scrubs all three lists during GDPR erasure (src/lib/db/data-erasure-repository.ts:279), so a redrive cannot resurrect deleted usage.

Metering overage to Polar

After a committed billable ledger row, persistBuiltAiUsageLedgerEntry (usage-accounting.ts:621-641) calls enqueueAiMeteringForLedgerRow when aiMeteringEnabled is set. The call is isolated in a try/catch that logs and swallows: the ledger row is already committed, metering is delta-based and reconcilable from the ledger, and a post-commit throw would make the durable wrapper re-enqueue an already-persisted row.

src/lib/billing/ai-metering.ts:35-143. A row is only considered when it is billable, billingScope === 'tenant', status === 'completed', has a tenant scope, and has billableCostCentsUsd > 0.

Inside one transaction holding pg_advisory_xact_lock(hashtext(tenantScopeId)):

1. Sum cumulative billable USD cents for the billing window, with a strict (finishedAt, id) upper bound so two same-millisecond completions cannot both count each other.
2. Convert to EUR with resolveUsdToEurRate() — the same rate the admission guardrail uses, so enforcement and billing agree on where the envelope ends.
3. targetOverage = max(cumulativeSpendEur - includedCentsMonthly, 0).
4. delta = max(targetOverage - alreadyQueuedOverage, 0). Zero or less: stop.
5. If policy.aiOverageEnabled is false: stop and enqueue nothing.
6. Insert one row into ai_usage_metering_outbox with externalEventId = ledgerRowId.

The advisory lock is transaction-scoped (required on a transaction-mode pooler) and auto-releases at commit. The threshold notification runs after the transaction so the lock is never held across notification I/O.

The outbox and its drain

Table ai_usage_metering_outbox (src/lib/db/schema.ts:836-859): ledger_id (FK, cascade), tenant_scope_id, external_event_id (unique index), meter_name, quantity_cents, status, attempt_count, provider_event_id, last_error, processed_at.

flushAiMeteringOutbox (ai-metering.ts:146-198) drains it. The loop is deliberately serial — one Polar request at a time, and the processed/failed mark lands before the next row is claimed, so a crash re-delivers at most the in-flight row. With POLAR_ACCESS_TOKEN absent it no-ops with reason: 'polar_not_configured'.

The meter shape (ai-metering.ts:15-33):

{
  "name": "ai_usage.cents.v1",
  "externalCustomerId": "org:<clerkOrgId>",
  "externalId": "<ledgerRowId>",
  "metadata": { "quantity_cents": 137, "source": "ermisai-ai-ledger" }
}

Polar deduplicates on externalId, which is why the ledger row id is used verbatim.

Crons and manual redrives

Two entries in vercel.json, both every two minutes:

Both routes carry two handlers. GET is the cron path, authorized by isAuthorizedCronRequest (the CRON_SECRET bearer check). POST is the human path, gated on canAccessAdminSurface(appRole, 'ai') — super_admin only.

The manual operator surface for all of this:

Threshold notifications

src/lib/ai/usage-threshold-notifier.ts fires at 50%, 80% and 100% of the envelope, computed from the same EUR-converted cumulative spend the metering step produced. Each (billingPeriodKey, threshold) pair is recorded with an atomic HSETNX (addHashRecordIfAbsent) before the notification is emitted, so two concurrent billable completions cannot double-fire. A new billing period uses new hash fields, so the next period re-fires.

Titles and bodies are hardcoded English (:50-71), not next-intl:

• 50% — 50% of monthly AI capacity used
• 80% — 80% of monthly AI capacity used
• 100% — Monthly AI capacity reached

At 80% and 100% an upsell email is also dispatched (usage-threshold-notifier.ts:118-128), but only to individual_free and individual_plus tenants whose subscription status is free or grants entitlements (src/lib/email/upsell-trigger.ts:12, :36-52). The dispatcher is fully failure-isolated so a send error never affects the metering path.

An envelope of 0 produces no thresholds at all (computeCrossedThresholds returns an empty list when envelopeCents <= 0).

Platform-wide spend cap

src/lib/ai/platform-spend.ts. Independent of any tenant envelope: a total ceiling on sum(billable_cost_cents_usd) across all tenants.

• Windows are UTC start-of-day and UTC start-of-month.
• Caps come from the runtime config (platformDailySpendCapCents / platformMonthlySpendCapCents), seeded from ERMIS_PLATFORM_AI_DAILY_CAP_CENTS / ERMIS_PLATFORM_AI_MONTHLY_CAP_CENTS. null means no cap.
• Only a window whose cap is non-null is queried, so an all-null config costs zero ledger reads.
• Results are cached 60 s per window, so the check adds at most one aggregate query per minute to the dispatch path.
• Exceeded when spend >= cap.

Currency

Plan prices, envelopes, overrides and hard limits are EUR cents. The ledger accrues USD cents (provider cost). There is exactly one conversion function, resolveUsdToEurRate() (src/lib/ai/usage-fx.ts:14-28), read from ERMIS_USD_TO_EUR_RATE, defaulting to 1.0. A non-finite or non-positive value also falls back to 1.

It is deliberately a dependency-free leaf module: both the guardrail and the metering layer need it, and importing it from either created a usage-accounting → ai-metering → usage-guardrails → usage-accounting cycle.

Three call sites use it, and because they all use the same one they agree with each other:

1. The admission guardrail's projectedSpendCents (usage-guardrails.ts:490-492).
2. The overage metering delta (ai-metering.ts:100-102).
3. The tenant usage snapshot shown in the workspace (src/lib/ai/tenant-usage-summary.ts:83-102).

They are consistent, but they are not currency-accurate until an operator sets a real rate.

Failure modes worth knowing

Operator surfaces

/admin/ai → AI usage operations (super_admin only, src/lib/auth/platform-roles.ts:32) has the buttons Refresh usage summary and Flush metering backlog, and the metric cards Tenant billable, Platform cost, Blocked requests, Failed requests, Pending metering, Failed metering, Unscoped warnings, Snapshot generated, plus Ledger anomalies and Provider operations readiness alerts. The Provider stage footprint section below it charts ledger-backed provider and stage costs.

/admin/costs (super_admin and ops) groups ai_usage_ledger by provider_id, stage and billing_scope. Two caveats: no date range is passed, so the figures are all-time totals; and they are USD, while the tenant envelopes on /admin/tenants are EUR.

/admin/tenants/[tenantId] (super_admin only) is where the three per-tenant billing controls live: the Overage enabled (metered billing beyond the included envelope) checkbox, Monthly hard limit (EUR, empty = plan default), and Envelope override (EUR/month, empty = plan envelope; REQUIRED for enterprise). updateTenantBillingAiControlsInDb (src/lib/db/billing-repository.ts:585) is the only write path for those three columns; undefined leaves a field unchanged, explicit null clears it.

Environment variables