Since 2026-10-06 the Codex lanes run
gpt-6.1-sol, notgpt-5.5. GPT-5.5 leaves Codex for ChatGPT sign-in on 2026-10-14, so rig-gitops#923 and #933 moved everygpt-5.5model line in the tables below. That covers the Codex lanes' main model and theproviders.codex-cli.modelsettings. Everything else on this page is the 2026-09-25 snapshot.
Rig agent model and effort inventory¶
Snapshot of 2026-09-25, before tuning. The Mac Mini produced this read-only inventory (job
rig-models, effort high). The orchestrator re-checked its key claims: the OAuth mapping, resumed runs dropping--model, and the image's Claude Code 2.1.281. Citation prefixes:gitops:= dashecorp/rig-gitops 9b65b3f,rar:= rig-agent-runtime,rc:= rig-conductor,c3:= this repo.Changed since, same day: - Every Review-E Claude leg now runs
claude-sonnet-5(rig-gitops#855, verified in-cluster). -llm.effortnow reaches claude-cli, claude-tmux and codex-cli, and resumed claude-cli runs keep--modeland--system-prompt(rig-agent-runtime#749). Effort is still unset everywhere. - codex-cli no longer throws whenpassMcpToCliis off (rig-agent-runtime#750).Tuning plan: one fleet at a time, each change measured against the 30-day baseline (2026-08-26 to 2026-09-25): Dev-E 88 PRs, 82% merged, 64% approved on the first Review-E verdict; Review-E 698 reviews, of which 394 approved, 145 changes requested, 101 commented and 58 dismissed. Next: Dev-E and the planner to
claude-opus-5-5, then effort per role (see Mac Mini delegation for the routing principle).
Read-only inventory, 2026-09-25. Nothing was changed in any repo.
Sources and how to read the citations¶
| Prefix | Path under ~/repos/ |
State read |
|---|---|---|
gitops: |
rig-gitops-fresh/ |
branch main-sync = 9b65b3fd (.git/refs/heads/main-sync:1) = dashecorp/rig-gitops main 9b65b3f. This is what Flux deploys. |
rar: |
dashecorp-rig-agent-runtime/ |
The working tree is on fix/issue-744-review-guard-checks-pr = 15ceb0f (.git/HEAD:1). The deployed image is node-sha-008ed87b… (gitops:apps/dev-e/rig-agent-helmrelease.yaml:22) = local main-sync (.git/refs/heads/main-sync:1). I could not diff them because git is permission-gated. The only #744 code cited here is src/closed-assignment-guard.js; treat that one as possibly undeployed. The chart itself is pulled from main (gitops:apps/dev-e/rig-agent-source.yaml:9-10), which is newer than the image. |
ae: |
automate-e/ |
main = 008ed87b (.git/refs/heads/main:1). This is the rig-agent-runtime checkout that iBuild-E actually runs on the Mac Mini (c3:docs/infrastructure/hosts.md:137). |
rc: |
rig-conductor-rigbuild/ |
branch design-2394 (.git/HEAD:1). It is not certain this equals the deployed conductor. |
c3: |
claude-3-944/ |
a worktree |
Deployment scope: Flux reconciles apps/{dev-e,review-e,rig-planner,rig-assistant,dashecorp,stigjohnny} on GCP (gitops:clusters/gcp/kustomization.yaml:5-20). apps/dev-e and apps/review-e are also reconciled on the legacy Dell cluster (gitops:clusters/dell/kustomization.yaml:6-7, gitops:README.md:33), where the pods have sat Pending since May (gitops:docs/unschedulable-pod-watch.md:44). apps/dev-e-staging is inert: staging-rig is suspend: true (gitops:clusters/gcp/staging-rig.yaml:17).
1. Inventory¶
Row legend:
- Effort: no row can set effort, reasoning or thinking. See the "Effort" note below the table.
- Chain: the provider chain the runtime actually walks. It is [provider, ...fallbackProviders, ...keys(providers)] with duplicates removed (rar:src/agent/provider-chain.js:35-60), so every key under llm.providers is a fallback, not just the entries in fallbackProviders. Each hop uses that provider's providers.<p>.model override (rar:src/agent/provider-chain.js:62-72).
- Auth: every in-cluster claude pod gets CLAUDE_CODE_OAUTH_TOKEN, mapped from secret key anthropic-api-key (rar:charts/rig-agent-runtime/templates/deployment.yaml:80-84). The chart never sets ANTHROPIC_API_KEY (gitops:docs/claude-tmux-pilot.md:31). Codex pods with CODEX_DEVICE_AUTH=true drop OPENAI_API_KEY and use the ChatGPT login in CODEX_HOME/auth.json (rar:src/agent/providers/codex-auth.js:46-57).
- Cost: "flat" means subscription quota: the Anthropic Max plan (gitops:docs/whitepaper/cost-framework.md:263) or the ChatGPT plan. The dollar figures the rig reports for these are counterfactual, not billed (gitops:docs/claude-tmux-pilot.md:52, rar:src/agent/providers/claude-tmux.js:120-133).
Base fleet (GCP, live)¶
| Fleet / variant (AGENT_ID) | Active provider | Model | Chain actually walked | Context | Max turns / timeout | Replicas / KEDA | Auth → cost | Where set |
|---|---|---|---|---|---|---|---|---|
Dev-E node, claude (dev-e-node) |
claude-tmux (pilot) |
claude-opus-4-7[1m] |
claude-tmux → claude-cli (opus-4-7[1m]) → codex-cli (gpt-5.5) | [1m] |
300 is ignored because tmux passes no --max-turns (rar:src/agent/providers/claude-tmux.js:359). Limits: 1,200,000 ms, idle-stall 360,000 ms. |
no replicaCount, so chart default 1 (rar:charts/rig-agent-runtime/values.yaml:4). KEDA 0..1 on signal:dev-e list (gitops:apps/dev-e/scaledobject.yaml:12-19). |
OAuth → flat. The codex hop has no credential: no CODEX_DEVICE_AUTH, and dev-e-secrets has no openai-api-key key (gitops:apps/dev-e/dev-e-secrets.sops.yaml:7-12). |
gitops:apps/dev-e/rig-agent-helmrelease.yaml:196-212 |
Dev-E node, codex (dev-e-node-codex) |
codex-cli |
gpt-5.5 |
codex-cli → claude-cli (claude-sonnet-4-6) | n/a | 150 is ignored: codex passes no max-turns (rar:src/agent/providers/codex-cli.js:598-638). Limit 1,200,000 ms. |
no replicaCount (1). KEDA 0..1. The comment says "pinned to 0 by paused-replicas", but the annotations: block is empty, so it is live (gitops:apps/dev-e/scaledobject-codex.yaml:6,14). |
device-auth, so ChatGPT plan → flat (…-codex.yaml:277-278). The claude hop is OAuth → flat. |
gitops:apps/dev-e/rig-agent-helmrelease-codex.yaml:197-210 |
Dev-E dotnet, claude (dev-e-dotnet) |
claude-cli |
claude-opus-4-7[1m] |
claude-cli only (no providers) |
[1m] |
300 / 1,200,000 ms | default 1. KEDA 0..1 (gitops:apps/dev-e/dev-e-dotnet-scaledobject.yaml:12-13). |
OAuth → flat | gitops:apps/dev-e/dotnet-helmrelease.yaml:137-144 |
Dev-E dotnet, codex (dev-e-dotnet-codex) |
codex-cli |
gpt-5.5 |
codex-cli only | n/a | 150 (ignored) / 1,200,000 ms | replicaCount: 1, KEDA min 1, so always on (gitops:apps/dev-e/dev-e-dotnet-scaledobject-codex.yaml:24-25) |
ChatGPT → flat | gitops:apps/dev-e/dotnet-helmrelease-codex.yaml:143-150 |
Dev-E python, claude (dev-e-python) |
claude-cli |
claude-opus-4-7[1m] |
claude-cli only | [1m] |
300 / 1,200,000 ms | default 1, KEDA 0..1 | OAuth → flat | gitops:apps/dev-e/python-helmrelease.yaml:137-144 |
Dev-E python, codex (dev-e-python-codex) |
codex-cli |
gpt-5.5 |
codex-cli only | n/a | 150 (ignored) / 1,200,000 ms | replicaCount: 0 plus KEDA paused-replicas: "0", so off (…python-helmrelease-codex.yaml:21, …dev-e-python-scaledobject-codex.yaml:7) |
ChatGPT → flat | gitops:apps/dev-e/python-helmrelease-codex.yaml:143-150 |
Review-E, claude (review-e) |
claude-tmux |
claude-sonnet-4-6 (#646). No [1m] since 2026-06-18. |
claude-tmux → claude-cli (sonnet-4-6) → codex-cli (gpt-5.5) | 200K | 150 (ignored under tmux) / 1,200,000 ms, idle 360,000 ms | replicaCount: 1 (…rig-agent-helmrelease.yaml:21). KEDA 0..1 on signal:review-e (gitops:apps/review-e/scaledobject.yaml:18-26). |
OAuth → flat. The codex hop has no credential (same reason as Dev-E node). | gitops:apps/review-e/rig-agent-helmrelease.yaml:202-218 |
Review-E, codex (AGENT_ID also review-e) |
codex-cli |
gpt-5.5 |
codex-cli → claude-cli (sonnet-4-6, #648) | n/a | 75 (ignored) / 1,200,000 ms | replicaCount: 1 (…-codex.yaml:23). KEDA 0..1 on the same signal:review-e list (gitops:apps/review-e/scaledobject-codex.yaml:17-26). |
No CODEX_DEVICE_AUTH (…-codex.yaml:243-265), so the env passes through unchanged and auth depends on whether auth.json exists on the PVC. Comments call the plan "ChatGPT Pro", "Team" and "Business" (…-codex.yaml:21,206, scaledobject.yaml:7). → flat |
gitops:apps/review-e/rig-agent-helmrelease-codex.yaml:205-218 |
Planner, claude (rig-planner) |
claude-cli |
claude-opus-4-7[1m] |
claude-cli → codex-cli (gpt-5.5) | [1m] |
50 / 1,200,000 ms. Temperature 0.1 is inert: CLIs get no temperature (see §2). | replicaCount: 1, KEDA paused-replicas: "1" (manual pin) (gitops:apps/rig-planner/keda-scaledobject.yaml:9) |
OAuth → flat | gitops:apps/rig-planner/rig-agent-helmrelease.yaml:118-132 |
Planner, codex (AGENT_ID rig-planner-codex, but the heartbeat agentId is rig-planner) |
codex-cli |
gpt-5.5 |
codex-cli → claude-cli (sonnet-4-6) | n/a | 30 / 1,200,000 ms | replicaCount: 0, paused 0: off (…-codex.yaml:21, keda-scaledobject-codex.yaml:7) |
ChatGPT → flat | gitops:apps/rig-planner/rig-agent-helmrelease-codex.yaml:115-129 |
Rig-E, tenant chat agent (rig-e) |
openai-api → RunPod vLLM |
Qwen/Qwen2.5-32B-Instruct-AWQ |
openai-api only. No fallback, deliberately (…rig-e-helmrelease.yaml:258-265). |
16,384 tokens, about 13,400 usable (c3:docs/infrastructure/gpu-brain.md:33) |
15 / 300,000 ms | replicaCount: 1, no KEDA (gitops:apps/dashecorp/kustomization.yaml:21-25) |
OPENAI_API_KEY = the pod's VLLM key (gitops:apps/dashecorp/rig-e-secrets.sops.yaml.example:12-13) → per GPU-hour: at most $2.00/h, last seen $1.59/h, about $0.24–0.53 per conversation (c3:docs/infrastructure/gpu-brain.md:31-37) |
gitops:apps/dashecorp/rig-e-helmrelease.yaml:237-267 |
rig-assistant, the "manager". It is not Rig-E (gitops:apps/dashecorp/rig-e-helmrelease.yaml:3). |
claude_code backend (claude -p) |
claude-sonnet-4-6 via ASSISTANT_LLM__MODEL. The app default is claude-opus-4-8. |
none visible | not visible | not visible. The rig-assistant code is not in the sources. | Deployment replicas: 1 (gitops:apps/rig-assistant/deployment.yaml:7) |
CLAUDE_CODE_OAUTH_TOKEN, the operator's token (…rig-assistant-secrets.sops.yaml:9) → flat |
gitops:apps/rig-assistant/deployment.yaml:45-54 |
Tenant fleets (GCP)¶
| Fleet | Provider / model | Chain | Turns | State |
|---|---|---|---|---|
dashecorp Dev-E node (dev-e-node, tenant) |
claude-cli / claude-opus-4-7[1m] |
claude-cli only | 300 | replicaCount: 0 (gitops:apps/dashecorp/dev-e-helmrelease.yaml:30,220-231). KEDA scales 0→1 on signal:…@dashecorp, with no pause annotation. |
| dashecorp Review-E | claude-cli / claude-sonnet-4-6 |
claude-cli only | 150 | replicaCount: 0 (gitops:apps/dashecorp/review-e-helmrelease.yaml:34,198-208), KEDA 0..1 |
| dashecorp Planner | claude-cli / claude-opus-4-7[1m] |
claude-cli → codex-cli | 50 | replicaCount: 0 (gitops:apps/dashecorp/planner-helmrelease.yaml:31,128-141), KEDA 0..1 |
| stigjohnny Dev-E / Planner / Review-E / Review-E-codex | opus-4-7[1m] / opus-4-7[1m] / opus-4-7[1m] / gpt-5.5 (claude hop opus-4-7) | Each includes codex-cli, or claude-cli for the codex variant. | 300 / 50 / 150 / 75 | All gated: replicaCount: 0 plus paused-replicas: "0" (gitops:apps/stigjohnny/{dev-e,planner,review-e,review-e-codex}-{helmrelease,scaledobject}*.yaml). Review-E here never got the #646 trim: it is on claude-opus-4-7[1m] (gitops:apps/stigjohnny/review-e-helmrelease.yaml:216,224), and the codex leg's claude hop is claude-opus-4-7 (…review-e-helmrelease-codex.yaml:224). |
Off-cluster¶
| Agent | Provider / model | Chain | Turns / timeout | Auth → cost | Where set |
|---|---|---|---|---|---|
| iBuild-E (Mac Mini, what actually runs) | rig-agent-runtime claude-cli / claude-opus-4-7[1m] |
claude-cli → codex-cli (gpt-5.4, device-auth) |
200 / 1,200,000 ms | Host CLAUDE_CODE_OAUTH_TOKEN (c3:docs/infrastructure/hosts.md:115) → flat |
ae:character.json:79-98. The process is node src/index.js in automate-e (c3:docs/infrastructure/hosts.md:137). This is a local host file, not gitops. |
| iBuild-E, legacy agent-runner config named in the brief | no model key, so the CLI default |
none | --max-turns 50 hard-coded. --max-budget-usd 2.0, or 10.0 for /opus. |
OAuth → flat | c3:agent-runner/agents/ibuild-e.json:1-18, c3:agent-runner/src/session-manager.js:78-92. The deployed daemon is automate-e, not this. The Mini also carries an uncommitted edit to this file (c3:docs/infrastructure/hosts.md:118). |
| Volt-E / Pi-E / iClaw-E (agent-runner, outside the rig) | "model": "opus" / none / none |
none | 50 | OAuth → flat | c3:agent-runner/agents/{volt-e,pi-e,iclaw-e}.json (per subagent read: volt-e.json:12) |
Effort / reasoning column (all rows): not settable¶
- claude-cli: argv carries
--model,--system-prompt,--max-turnsand permission/MCP flags only (rar:src/agent/providers/claude-cli.js:189-221). There is no--effort, noMAX_THINKING_TOKENSand no--fallback-modelanywhere insrc/. - claude-tmux:
[claude, '--model', m, '--dangerously-skip-permissions'](rar:src/agent/providers/claude-tmux.js:359). - codex-cli:
exec --json … --model m -C cwd --search. Its only-cflags are formcp_servers.*(rar:src/agent/providers/codex-cli.js:598-659). There is nomodel_reasoning_effortand noconfig.tomlis written. - anthropic SDK: no
thinkingfield (rar:src/agent/providers/anthropic-sdk.js:47-54). - openai-api: only
temperature(rar:src/agent/providers/openai-api.js:338). - iBuild-E agent-runner: none (
c3:agent-runner/src/session-manager.js:82-92). - The baked Claude settings file
rar:config/claude-settings.json(copied atrar:Dockerfile.base:107) has no model, effort or thinking key.
[1m]¶
It is passed verbatim as --model (rar:src/agent/providers/claude-cli.js:202, claude-tmux.js:359). The runtime does not parse or strip it and sends no beta header. It appears only as a pricing key (rar:src/usage.js:12). Whether claude-opus-5-5[1m] or claude-sonnet-5[1m] is a valid CLI alias is unverified.
2. Levers¶
A. Gitops value only (HelmRelease character.llm.*, extraEnv, replicaCount, KEDA). Takes effect on the next Flux reconcile, about 10 min.¶
| Lever | Key | Notes |
|---|---|---|
| Model per fleet | llm.model and llm.providers.<p>.model |
The model is required and has no default (rar:src/character.js:92-95). Change both, or a fallback hop silently runs the old model (the reason for #648). |
| Provider | llm.provider |
Values: claude-cli / claude-tmux / codex-cli / openai-api |
| Fallback chain | llm.fallbackProviders and the set of keys under llm.providers |
Removing a codex-cli: key removes codex from the chain. Removing a key is the only way to stop a claude pod from falling to a codex hop that has no credentials. |
| Turns | llm.maxTurns |
Effective only on claude-cli (and openai-api/anthropic). It is not used by claude-tmux or codex (rar:src/agent/providers/claude-tmux.js:359, codex-cli.js:598-638). |
| Timeouts | llm.timeoutMs, llm.idleStallMs |
rar:src/agent/providers/claude-cli.js:277, claude-tmux.js:368,477, codex-cli.js:435 |
| Capacity | replicaCount (0 is now honoured, rar:charts/rig-agent-runtime/templates/deployment.yaml:12-16) and autoscaling.keda.sh/paused-replicas |
Max is 1 per fleet because of the shared PVC (gitops:apps/review-e/scaledobject.yaml:19-21) |
| CLI env passthrough (hypothesis, verify first) | extraEnv |
The claude child inherits the whole pod env (rar:src/agent/providers/claude-cli.js:151-158; claude-tmux uses the same builder, claude-tmux.js:324). Any env-based Claude Code setting could therefore be set with no code change, for example MAX_THINKING_TOKENS, or ANTHROPIC_MODEL to pin the model on the --resume path (§5). Whether the pods' CLI version honours a given variable must be checked against that version first. |
| Temperature | llm.temperature |
Inert for every CLI agent. Only the openai-api and anthropic providers read it (rar:src/agent/providers/openai-api.js:338, anthropic-sdk.js:50). |
| Routing (conductor config, not gitops) | issue labels provider:codex / provider:claude, and routeTierToCodexFirst in the target repo's .rig-agent.yaml |
rc:src/ConductorE.Api/Program.cs:1264-1320, rc:src/ConductorE.Core/UseCases/TierCodexRouter.cs:63-91 |
B. Needs a runtime code change (rig-agent-runtime, then an image-pin bump in gitops)¶
| Change | Where |
|---|---|
Pass --effort (claude) |
Both argv branches at rar:src/agent/providers/claude-cli.js:189-206, and rar:src/agent/providers/claude-tmux.js:359. Config plumbing is free: llm.effort or llm.providers.<p>.effort already flows through resolveCharacterForProvider (rar:src/agent/provider-chain.js:62-72) and the raw-JSON ConfigMap (rar:charts/rig-agent-runtime/templates/configmap.yaml:8-9). |
Pass -c model_reasoning_effort=… (codex) |
Next to the -c loop at rar:src/agent/providers/codex-cli.js:633-635, fed from the caller at :416-424 |
Keep --model on --resume |
rar:src/agent/providers/claude-cli.js:189-197 |
| Honour maxTurns under claude-tmux and codex | rar:src/agent/providers/claude-tmux.js:359, codex-cli.js:598-638 |
| Pricing entries for 5.x models | rar:src/usage.js:4-15 (unknown models fall to $1/$5), rar:src/agent/providers/claude-tmux.js:128-130 (non-sonnet/haiku are priced as Opus 15/75), and the conductor's rc:src/ConductorE.Core/UseCases/AnthropicPricing.cs:24-65 (unknown models priced as sonnet-4-5) |
| Report cost once, with the model that ran | rar:src/stream-consumer.js:1076-1085 duplicates the provider's own reportTokenUsage (claude-cli.js:431-439, claude-tmux.js:556-562) and labels it with the top-level character.llm.model, not the fallback hop's model |
| Pin the Claude Code / codex CLI versions | rar:Dockerfile.base:69 installs unpinned @anthropic-ai/claude-code @openai/codex. Node, dotnet and python images build FROM …:base (rar:Dockerfile.node:3). claude-opus-5-5 needs CLI 2.1.280 or later (c3:docs/infrastructure/hosts.md:114), and the in-pod version is unknown from source. |
| Distinct identity for the Review-E codex pod | A gitops AGENT_ID change, but coupled to the conductor: the router expects review-e-codex (rc:src/ConductorE.Core/UseCases/QuotaAwareReviewRouter.cs:45-48), and today both pods report review-e (gitops:apps/review-e/rig-agent-helmrelease-codex.yaml:63,246). |
C. Impossible, or not possible without a design change¶
- Per-task model or effort by tier. The dispatch payload has no model or effort field (
rc:src/ConductorE.Api/Adapters/ValkeyStreamPublisher.cs:164-185). Tiers only gate dispatch and optional codex-first routing (rc:src/ConductorE.Api/Services/TaskSpecConsumerService.cs:71-95). Per-PR "Sonnet default, Opus on T2/T3" (gitops:docs/2026-06-18-review-e-sonnet-cost-trim.md:38) therefore needs conductor plus runtime work, or separate fleets. - Effort for Rig-E. Qwen2.5-32B-Instruct has no reasoning mode, and its context is capped at 16K by the GPU pod (
c3:docs/infrastructure/gpu-brain.md:33). - Same-provider model fallback (Opus → Sonnet on quota). Only a provider switch changes the model (
rar:src/agent.js:119-185). - iBuild-E via gitops. Its config is a local file on the Mac Mini (
ae:character.json) and has to be changed on that host.
3. Outcome data that exists to judge a change¶
Conductor base: http://rig-conductor-api.rig-conductor.svc.cluster.local:8080. Event wire names are in rc:src/ConductorE.Api/…/SubmitEvent.cs, and records in rc:src/ConductorE.Core/Domain/Events.cs. The conductor citations below come from a delegated read; QuotaAwareReviewRouter.cs, TierCodexRouter.cs and ReviewDispatchRouter.cs were re-read directly.
| Role | "Model or effort was wrong" signal | Events | Where to read it |
|---|---|---|---|
| Dev-E | First-pass merge rate, review rounds per PR, CI failures per PR, re-dispatches, stuck/crashed runs, turns and duration per issue | PR_CREATED, MERGED / MERGE_GATE_MERGED, REVIEW_DISPUTED (= CHANGES_REQUESTED), CI_FAILED / BUILD_FAILED, ISSUE_ASSIGNED.Attempt, ISSUE_UNASSIGNED, SLO_BREACHED, AGENT_STUCK, ISSUE_STUCK, CLI_COMPLETED (Reason = completed/crashed/timeout, plus Turns, DurationMs, CostUsd), ISSUE_DONE.DurationMinutes, TESTS_RUN (runtime hook rar:hooks/pre-push-tests.sh:409) |
Per issue: GET /api/issues/trace?repo&issueNumber. Rounds: IssueStatus.ReviewRoundsCount, via GET /api/issues/item. GET /api/ci-failures, GET /api/stuck-issues. Turn p50/p90/p95 and cost by agent: GET /api/execution-logs/stats?agentId&days. Per run: GET /api/execution-logs (includes Model). Also GitHub PR reviews and checks. |
| Review-E | False blocks (CHANGES_REQUESTED followed by a merge with no code change, or an operator override), misreadings (dispute escalations), lost reviews (a dispatch with no verdict), rounds hitting the cap | REVIEW_PASSED, REVIEW_DISPUTED, REVIEW_FAILED, RE_REVIEW_REQUESTED, DISPUTE_ESCALATED. The max-rounds cap is 3 (MAX_REVIEW_ROUNDS). Operator merge without approval is auto-inferred at rc:Program.cs:2037-2084. |
Verdicts come from the GitHub pull_request_review webhook when the login contains "review-e" (rc:Program.cs:2347-2464,2573-2581). The runtime no longer emits verdicts (rar:src/review-follow-up.js:1-12). The ground truth is GitHub PR reviews by review-e-bot. |
| Planner | Re-plans, T3 escalations, sub-issues that fail downstream | TIER_CLASSIFIED, TASK_SPEC_POSTED, EPIC_DECOMPOSED. There is no dedicated re-plan event: count repeated TASK_SPEC_POSTED per issue. |
GET /api/issues/trace |
| All | Quota pressure, fallback use | AGENT_QUOTA_REPORTED, PROVIDER_EXHAUSTED (defined in the conductor but never emitted by the runtime, rar:docs/usage-tracking.md:126), CLI_STARTED.Model, AgentStatus.LastModel |
GET /api/agents. Fallback hops are only in CLI_PROGRESS text, throttled to one per 55 s, with "Using provider" messages dropped (rar:src/stream-consumer.js:784-797), plus GET /api/agent-logs/{agentId}?count= (gitops:docs/claude-tmux-pilot.md:68-70) and pod stdout, which rotates out within about 30 min. |
| Cost | Spend or quota per model | TOKEN_USAGE (on the agent stream, not the issue stream) |
GET /api/usage?agentId&repo&days, /api/costs/summary?days, /api/costs/daily?days, /api/costs/issue. None of them group by model. |
Gaps that block a clean A/B today:
1. The two Review-E pods share AGENT_ID review-e, so verdicts, heartbeats and costs cannot be split by agent. The only split is ExecutionLog.Model per run.
2. There is no fallback event, so a run that silently fell to another provider's model cannot be told apart in TOKEN_USAGE, which carries the top-level model label.
3. Costs are counterfactual and double-counted (§5).
4. A baseline has to be pulled before changing anything, because the projections are cumulative.
4. Gaps and hypotheses¶
Each item is a hypothesis to test with the data in §3. None is a recommendation.
| # | Hypothesis | Test | Cost / quota side |
|---|---|---|---|
| H1 | Dev-E on claude-opus-4-7[1m] → claude-opus-5-5 raises first-pass merge rate and lowers review rounds and CI fails per PR. |
Before/after on the node fleet: rate of MERGED without REVIEW_DISPUTED, ReviewRoundsCount, CI_FAILED per PR, and CLI_COMPLETED.Reason. First confirm the in-pod CLI is 2.1.280 or later. |
Flat quota (OAuth). Opus-class weekly quota burn is the risk, not dollars. No approval needed for spend unless the credential changes. |
| H2 | Review-E claude-sonnet-4-6 → claude-sonnet-5 (the 2026-09-20 decision) lowers false blocks and misreadings at similar quota use. |
False-block rate and DISPUTE_ESCALATED per review, split by ExecutionLog.Model, because the codex pod shares the agent id. |
Flat quota. Pricing tables have no sonnet-5 entry, but tmux's startsWith('claude-sonnet') gives it 3/15 (rar:src/agent/providers/claude-tmux.js:128), so its reported cost is only as good as that guess. |
| H3 | Planner opus-4-7[1m] → opus-5-5 (or sonnet-5) cuts re-plans and T3 escalations. |
Repeated TASK_SPEC_POSTED per issue; downstream Dev-E success on planned issues |
Flat quota |
| H4 | [1m] is not needed for the planner (50 turns, one TaskSpec per run, rig-agent-helmrelease.yaml:122) or Review-E (already dropped). Dropping it on Dev-E may cost quality on large repos. |
Compare CLI_COMPLETED crashes and context errors before and after |
On a subscription, [1m] may draw quota faster. Untested. |
| H5 | Effort high or xhigh on Dev-E and Review-E (after lever B) reduces rounds and false blocks more cheaply than a model bump. |
Same metrics, same model, effort only | Flat quota, more tokens per task |
| H6 | Codex model_reasoning_effort (after lever B) closes the quality gap between the codex and claude variants. |
Tier-t1 codex-first issues vs claude issues on merge rate | ChatGPT plan quota; bursts already exhausted the 5h window at about 9 PRs (gitops:apps/review-e/rig-agent-helmrelease.yaml:17-20) |
| H7 | Pinning ANTHROPIC_MODEL via extraEnv stops resumed Dev-E iterations from running a CLI-default model. |
CLI_STARTED / ExecutionLog.Model on resumed runs vs fresh runs |
None |
Which changes raise per-token spend (these need the operator's approval):
- Swapping any claude pod's anthropic-api-key secret value to a real API key.
- Adding a LiteLLM proxy or paid key (gitops:docs/whitepaper/cost-framework.md:274).
- Anything that makes Rig-E's GPU pod stay up longer, or land on a pricier card.
- Wider use of OPENAI_API_KEY for memory embeddings (per-token, optional: gitops:apps/dev-e/rig-agent-helmrelease.yaml:288-297).
Every model or effort change on the existing OAuth and device-auth credentials only consumes flat quota.
5. Surprising findings¶
- The decided move to
claude-sonnet-5never shipped. Nosonnet-5,opus-5,fableor2026-09-20string exists anywhere in rig-gitops main 9b65b3f (Grep, case-insensitive). Nothing is in the runtime pricing (rar:src/usage.js:4-15), the conductor pricing (rc:…/AnthropicPricing.cs:24-65) or claude-3 docs either. The rig is still entirely on 4.x:opus-4-7[1m],sonnet-4-6, and gpt-5.5 for codex. The only 5.x use is the orchestrator's own Mac Mini delegation (c3:docs/infrastructure/hosts.md:101,116). (Review-E already has no[1m].) - The fallback chain is wider than
fallbackProviders. Everyllm.providerskey is appended (rar:src/agent/provider-chain.js:37-41). So Dev-E node and Review-E claude fall claude-tmux → claude-cli → codex-cli (gpt-5.5), and the codex hop has no credential in those pods. Any error at all triggers the next hop (fallbackEligibledefaults to true,rar:src/agent/provider-error.js:7,68-71). Rig-E's comment that "claude-cli reads character.llm.model directly; llm.providers only orders the chain" (gitops:apps/dashecorp/rig-e-helmrelease.yaml:258-262) is stale: per-provider models are merged (rar:src/agent/provider-chain.js:62-72). - A resumed claude-cli run does not pass
--model(rar:src/agent/providers/claude-cli.js:189-197). The stream consumer resumes sessions on retries and iterations (rar:src/stream-consumer.js:839-855). Those runs use whatever model the CLI defaults to, not the configured one. This is a silent model change. maxTurnsis dead config on every claude-tmux and codex-cli agent: 7 base-fleet HelmReleases (Dev-E node, Review-E claude, and the five codex variants, two of which are off). Only claude-cli enforces it.temperatureis dead config on every CLI agent.- The dollar figures behind #646 were counterfactual. Every claude pod runs on subscription OAuth. The "$607/7d, Review-E 86%" figure (
gitops:docs/2026-06-18-review-e-sonnet-cost-trim.md:13) is API-equivalent. The Opus→Sonnet trim saved quota, not billed dollars. The cost data is also unreliable: - It is double-counted for claude-cli and claude-tmux (§2B).
- tmux cost counts only the pasted prompt and the final reply at 4 chars/token, not the agentic turns in between (
rar:src/agent/providers/claude-tmux.js:125-133,555). Dev-E node and Review-E are therefore under-reported by a wide margin. - The conductor re-prices unknown models as sonnet-4-5 when cache tokens are present (
rc:…/MartenEventStore.cs:446-459). - The "Daily Cost Report" calls
/api/usagewith nodays, so it posts all-time totals, not daily ones (gitops:apps/rig-conductor/daily-cost-report-cronjob.yaml:31). - Quota-aware Review-E routing is inert. The conductor chooses between
review-eandreview-e-codexby quota (rc:src/ConductorE.Core/UseCases/QuotaAwareReviewRouter.cs:45-101,rc:src/ConductorE.Api/Services/ReviewDispatchRouter.cs:38-39). The codex pod calls itselfreview-e, soreview-e-codexnever heartbeats, and both pods race on one LPOP list. - Contradictory billing comments. The codex Review-E comments say the switch was "instead of metered Anthropic API" (
gitops:apps/review-e/rig-agent-helmrelease-codex.yaml:21), while the claude HR and docs say OAuth subscription (…rig-agent-helmrelease.yaml:203,gitops:docs/claude-tmux-pilot.md:31).rar:src/agent.js:221-223also says tmux uses a meteredANTHROPIC_API_KEY, which contradictsclaude-tmux.js:9-14. - Stale KEDA comments.
dev-e-node-codexsays it is "pinned to 0 by paused-replicas", but it has none, so it is live (gitops:apps/dev-e/scaledobject-codex.yaml:6,14).review-e-codexsays "Idle: codex variant disabled", yet it is the active reviewer atreplicaCount: 1(gitops:apps/review-e/scaledobject-codex.yaml:18-20). The claude Review-E is described as "replicaCount: 0 keeps it idle" (:9-10) but has been 1 since gitops#476 (…rig-agent-helmrelease.yaml:21). - The #646 trim did not reach every copy. stigjohnny Review-E is still
claude-opus-4-7[1m], and its codex leg falls back toclaude-opus-4-7(dormant today). Planner-codex and Dev-E-codex claude hops are sonnet-4-6 while their claude siblings are opus. iBuild-E's codex hop isgpt-5.4, not 5.5 (ae:character.json:94). - iBuild-E is not
ibuild-e.json. The live daemon runs rig-agent-runtime fromautomate-ewithcharacter.json(c3:docs/infrastructure/hosts.md:137,c3:docs/infrastructure/agents.md:31).agent-runner/agents/ibuild-e.jsonhas no model and is not what launchd runs. - Chart
replicaCount: 0was ignored until #732 (rar:charts/rig-agent-runtime/templates/deployment.yaml:12-16). Earlier "dormant" fleets actually rendered 1. - The Dell legacy cluster still reconciles
apps/dev-eandapps/review-e(gitops:clusters/dell/kustomization.yaml:6-7), so any model change also lands there, and those pods are Pending.
Docs¶
No docs changed: this job was read-only by brief, and this report is the only deliverable. Doc updates these findings would need (§5 items 2, 6, 8 and 9) belong to whichever job makes the corresponding change.