Commit Graph

113 Commits

Author SHA1 Message Date
Codex
1bb23a60db fix: park rooms during a transient Claude auth outage instead of failing
ff76d04 told the room about a transient refresh failure but still failed
the execution (completeImmediately 'failed'), which bumps
owner_failure_count and can hand the task to the arbiter after two blips.

- reportClaudeAuthRecovering now parks the turn like the usage-limit
  pause: releaseForClaudeUsagePause (lease released, no failure count, no
  follow-up) plus markRunPausedForClaudeUsage (messages stay unconsumed).
  The room is told once per incident; later blips park silently.
- token-refresh fires setOnClaudeAuthRecovered once a rejected token is
  replaced (deferred so the new token is in memory first); index.ts
  re-queues every parked room so the same turn runs without user input.
- Real-DB test: two parked owner runs leave owner_failure_count/status/
  arbiter_requested_at unchanged (control: a failed run does count).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:02:48 +09:00
Codex
ff76d041d3 fix: tell transient Claude refresh failures apart from a lost login
A 401-rejected token whose refresh failed with 429/5xx/timeout was marked
re-login-required at once, closing tasks and asking for a needless re-login.

- refreshToken throws TokenRefreshError; only 400/401/invalid_grant is
  definitive. A forced refresh of a rejected token retries transient
  failures twice right away; the periodic loop then keeps retrying the
  known-rejected token every check, and only a failure lasting 30 min turns
  into re-login-required.
- While recovering, the room is told once per incident and the paired task
  is kept, so the normal retry path can resume it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 10:55:31 +09:00
Codex
3f2e364c73 fix: stop silent Claude 401 stalls and token-family revocations
- Refreshing the OAuth token revokes the old access token, and agents hold it
  in CLAUDE_CODE_OAUTH_TOKEN (Claude Code keeps env tokens on 401). Refresh
  early only while no run is active; keep the 1h hard deadline.
- forceRefreshToken (after a 401) now probes /api/oauth/profile: a revoked
  token keeps its expiresAt, so it was handed back unchanged forever. A
  rejected token is refreshed; if that fails, re-login is flagged at once and
  held until a different token appears.
- Classify "OAuth access token has been revoked" as auth-expired. When the
  login is gone, close the paired task and tell the room to re-login instead
  of looping owner/arbiter retries with suppressed output.
- Only suppress the CLI's own short 401 banner, not an answer quoting it
  (a sketchup-app TASK_DONE report was swallowed this way).
- Runner denies Bash commands that copy or write .credentials.json: a CLI test
  against a copied login refreshed the real token family on 2026-11-02 and
  logged out every room.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 10:45:21 +09:00
Codex
720223eb10 fix: keep paired rooms moving when owner turn preparation fails
A wargame room stalled after its owner ran npm install in a bun project:
both bun.lock and package-lock.json existed, so every owner turn threw
'Ambiguous package manager' while provisioning the workspace. The queue
had already claimed the turn, nothing released the claim, and the 5s
backoff retry was skipped as 'already claimed elsewhere', so the room
sat idle until a human wrote again (and then failed the same way).

- Owner workspace provisioning and the review handoff no longer fail
  when the dependency install step throws; they log a warning and go on.
- If preparing a paired run throws, mark the claimed turn failed and
  release its execution lease so the retry can run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 15:10:00 +09:00
Codex
0a5436e4ad fix: alignment hold also covers scheduled-task limits and the primer
- A scheduled Claude task that hits the usage limit now extends the
  alignment hold to the next slot when the reset is off the grid, like a
  room turn (it called runAgentProcess directly and skipped that path).
- The Claude primer is skipped while a hold outlasts its slot (e.g. the
  08:00 primer during a realignment hold until 13:00). A hold ending at
  the slot itself does not count: the slot timer fired at 07:59:59.999
  for 08:00 on 2026-10-16.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 13:04:44 +09:00
Codex
a2b9898c0b fix: keep usage resets on the slot grid when Claude hits 100%
Evidence (journal): 10/04 and 10/05 08:00 and 10/12 23:00 the primer
failed (exit 1) because the account was at 100% in an off-slot window
(resets 9am / 3am); the first request after that reset then opened the
next window off the slot too, and the drift carried on.

- Usage limit that resets off the slot grid (not ~04/08/13/18/23:00 KST):
  extend the alignment hold to the next window start (08/13/18/23), so
  nothing opens a window between the reset and the primer. The pause
  notice names both times. Rooms paused by the limit stay paused
  (silently, now alignment-only, @-mention bypass works) until the hold
  ends; an expired real limit no longer counts as a limit.
- A real limit takes precedence over the hold in the pre-run gate.
- Primer: when the current 5h window (cached usage) ends within 10 min
  after the slot, wait until 5 s after that reset before firing, so the
  primer is not counted in, or rejected by, the old window.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 06:03:38 +09:00
Codex
f944f97e54 feat: usage-alignment hold (04-08 KST daily + one-off hold until a set time)
Bring back the dawn gap that keeps Claude's 5h windows on the primer
slots (08-13, 13-18, 18-23, 23-04): it was dropped by the 2026-06-08
upstream merge (1dd948a).

- usage-alignment-hold.ts: hold during 04:00-08:00 KST, or until the ISO
  time in router_state `usage_alignment_hold_until` (ignored once past,
  so only the daily gap remains). A one-off end inside the gap runs on to
  08:00.
- Claude turns during a hold pause through the existing usage-pause path
  (messages kept, room resumed at the hold end) with an alignment notice.
  The stored pause is marked alignmentOnly so it does not act as an
  account-wide usage limit.
- Due Claude scheduled tasks stay due until the hold ends (Codex and
  GitHub CI watchers are not held).
- Urgent bypass like the old gate: a human @-mention lets that room run
  until the hold ends.
- Tests stub the wall-clock hold in test/vitest-env.ts so the suite does
  not pause when run between 04 and 08 KST.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 05:54:39 +09:00
Codex
aed7371cfa feat: pause room work on the Claude usage limit and resume after reset
Before, a Claude run that hit the account usage limit closed the open
paired task as claude_usage_limit, posted the reset time and consumed the
user's messages, so the work was lost and had to be re-requested.

Now the room is paused instead:
- The failed turn is released without touching the paired task (no
  failure count, no status change, no follow-up) and the queue keeps the
  human messages unconsumed, so the same turn can run again.
- The room is recorded in router_state `claude_usage_pause` with the reset
  time (error text, else cached usage API reset, else a 10 min retry). A
  banner time that just passed is read as "now", not tomorrow.
- While the limit lasts, paused rooms run nothing and any other room's
  Claude turn pauses that room without spawning a run (the limit is
  account-wide). A stop request still goes through and drops the room
  from the resume list.
- The scheduler tick clears the pause 1 min after the reset, posts a
  resume notice and re-queues the rooms; the pending turn or the kept
  messages then run again. The state survives restarts.

Safety refusals keep closing the task as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-13 13:14:56 +09:00
Codex
f579b4f406 fix: treat Claude safeguards refusals as failures, reset the session
Claude Code returns "API Error: <model>'s safeguards flagged this message"
as a success-typed result. EJClaw delivered it as the owner's answer,
handed it to reviewer/arbiter, resumed the refused session (refused again)
and carried the banner forward into the next task.

Now it is a `safety-refusal` trigger: the raw banner is suppressed, the
Claude session is cleared, the paired task closes as
`claude_safety_refusal`, the room gets a short Korean notice, and refusal
banners are never carried forward. Global memory gains a rule not to read
thinking blocks from session transcripts (the reasoning_extraction trigger
of the 2026-10-12 incident).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-12 13:17:22 +09:00
Codex
0283abbb56 feat: skip or slim down the owner finalize turn
(a) When the reviewer approves the owner's first TASK_DONE answer on the
first review (one review round, outputs are exactly owner -> reviewer, no
revision or arbiter turns) and the owner workspace still matches the
approved tree with a clean working tree, no owner run happens: the task is
completed as done and the reviewed answer (status line stripped) is posted
as the final message. If the review was requested with a bare "검토해줘"
after a review-skipped answer, a one-line confirmation is posted instead of
repeating the answer.

(b) Every other finalize turn runs in a fresh, non-persisted session (no
resume of the task's long working session, no room memory briefing) with a
self-contained prompt: this task's user messages, the latest six turn
outputs and the finalize instructions/format. All three finalize entry
points (pending turn, inline bot-only follow-up, queued turn) use it.

The owner prompt now states that the first TASK_DONE answer must already
follow the final-message format because it may be posted as the final
message unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:42:12 +09:00
Codex
c858b6e446 feat: scope paired owner/reviewer sessions to one paired task
Each paired room used to resume one owner (and one reviewer) session for
weeks, so every call re-read a huge unrelated history (median ~260k input
tokens). A stored session is now resumed only while it belongs to the
running paired task; the first turn of a new task starts fresh and the new
session is bound to that task (router_state paired_session_task:<folder>,
survives restarts). Later turns of the same task (revisions, follow-up
human messages while the task is open) keep resuming it. Arbiter was
already fresh per call; single rooms are unchanged.

The previous task's final answer still reaches the new session through the
carried-forward turn-0 output, so PAIRED_CARRY_FORWARD_LATEST_OWNER_FINAL
now defaults to true.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:25:35 +09:00
Codex
8c68577a24 refactor: replace the Claude→Codex failover with a usage-limit notice
When Claude hit a limit the run was handed to a Codex service
(codex-main/codex-review) through service_handoffs and a global owner
failover. That service does not exist in this deployment, so handoffs
stayed pending forever (2 rows since May) and the user got no answer.

Removed:
- handoff creation (message-agent-executor-handoff, paired-turn-fallback)
  and the delegated paired-execution path
- the pending-handoff poll/claim worker (message-runtime-handoffs) and the
  service_handoffs CRUD module (db/service-handoffs, db exports)
- global owner failover in service-routing, its recovery timer in index,
  the dashboard failover line, the codex-review failover prompt pack and
  prompts/codex-review-failover-platform.md
- failoverOwner in RoomRoleContext and the `failover=` ROOM_ROLE field
- OWNER_/REVIEWER_/ARBITER_FALLBACK_ENABLED and FAILOVER_MIN_DURATION_MS
- tests for the removed behaviour

Now, when a Claude run fails with a usage/rate limit (trigger reason
429 or usage-exhausted) the executor closes the open paired task as
completed/claude_usage_limit and sends the room one final notice:
"Claude 사용 한도에 도달했습니다. 21:40에 재개할 수 있습니다." The reset time
comes from the error text (CLI banner "resets 9:40pm (Asia/Seoul)", dated
banners, "try again at ...", ISO timestamps), else from the cached usage
API window at 100%, else "잠시 후 다시 시도해 주세요". Nothing retries
silently; the next user message starts a new task. Other Claude failures
(session failure, org access denied) now end as ordinary run errors.

The service_handoffs table, its columns on paired_turn_attempts and the
migrations stay untouched for the later migration squash; the runtime no
longer reads or writes them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:14:48 +09:00
Codex
a9eab5cb94 refactor: run Claude and Codex on one fixed account each
The deployment has a single Claude OAuth credential and a single Codex
ChatGPT account, so the N-account rotation machinery only added state
files, cooldown bookkeeping and code paths that never ran.

Removed:
- Claude token rotation (token-rotation.ts, token-rotation-base.ts):
  rotate-on-limit, per-token cooldowns, token-rotation-state.json.
  CLAUDE_CODE_OAUTH_TOKENS is still read, but only its first entry.
- Codex account rotation (codex-token-rotation.ts): account pool,
  rotate-on-429/auth, dead-auth/cooldown tracking, round-robin advance,
  d7 auto-rotate, codex-rotation-state.json, manual account switch.
- Account index plumbing in token-refresh, claude-credentials-path
  (CLAUDE_ACCOUNTS_DIR / ~/.claude-accounts/<n>), Claude/Codex usage
  collectors, the dashboard rows and the Codex warm-up.
- provider-retry.ts rotation loops and the Codex retry path.

Kept:
- Codex exclusive auth lease (one Codex run at a time), the 15-minute
  waitForCodexAuthLease, and a usage-poll lease that skips while a run
  holds the account (codex-auth.ts).
- Session auth copy and write-back of refreshed tokens
  (syncCodexSessionAuthBack, syncClaudeSessionAuthBack).
- Codex auth path: ~/.codex-accounts/0/auth.json, else ~/.codex/auth.json
  (now resolved per run, so a re-login needs no restart).
- Claude token refresh loop, CLAUDE_CREDENTIALS_PATH and the
  CLAUDE_TOKEN_REFRESH_* / CLAUDE_USAGE_USE_HOME_CREDENTIALS flags; an
  auth-expired run still force-refreshes the token and retries once.
- Error classification (classifyClaudeTrigger, classifyCodexTrigger).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 20:14:38 +09:00
Codex
f127ac7a2b refactor: remove the main room concept
No room has room_settings.is_main = 1, so every main-only path was dead,
and admin work is done from the host shell. Each room's agent may now
only act on its own room:

- IPC: messages/inbound injection only to the source room; schedule,
  update, pause, resume and cancel only for the source room's tasks;
  drop the assign_room and refresh_groups IPC tasks and the deps they
  needed (assignRoom/syncGroups/getAvailableGroups/writeGroupsSnapshot)
- DB/host evidence is always scoped to the source group folder
- runner MCP server: drop the assign_room tool, the target_group_jid
  parameters and the isMain flag; list_tasks shows only the room's tasks
- drop EJCLAW_IS_MAIN, RunnerInput.isMain and AgentInput.isMain; the
  global CLAUDE.md memory is now included for every room
- drop the available_groups.json snapshot (available-groups.ts), the
  inferred restart announcement that only went to the main room, and
  the unused Channel.syncGroups hook
- setup register: drop --is-main; deregister-room: drop the main-room
  refusal and --force, and fix its unknown[] binding type error
- remove the groups/main/ template

The room_settings.is_main column and migrations are untouched; new rows
get the column default (0) and nothing reads it anymore.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 19:37:29 +09:00
Codex
60ff19177a refactor: remove Mixture of Agents and Kimi usage
MOA_* was never configured, so enrichArbiterPromptWithMoa always
returned the prompt unchanged and the Kimi usage fetch only produced an
empty dashboard row. Remove src/moa.ts, src/kimi-usage.ts, the arbiter
prompt enrichment, MoA config/schema, the Kimi row and MoA line in the
Discord status dashboard, and the MOA_* / unused FALLBACK_* provider
examples from the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-01 19:10:19 +09:00
Eyejoker
f1679ac686 [codex] Remove Codex SDK canary runtime (#220)
* Revert "fix: drain sdk follow-up turns (#219)"

This reverts commit 807828e0ce.

* Revert "feat: feature-flag Codex SDK runner (#218)"

This reverts commit 879d16235f.
2026-06-03 19:10:53 +08:00
Eyejoker
807828e0ce fix: drain sdk follow-up turns (#219) 2026-06-03 17:58:24 +08:00
ejclaw
e20cd2f1b0 fix: treat human-interrupted paired runs as preempted 2026-05-26 03:25:38 +09:00
ejclaw
db5332e81a fix: refresh role rules after compaction 2026-05-25 22:57:52 +09:00
ejclaw
fe9d265c66 paired: repair safe owner workspace branch mismatches 2026-04-11 08:45:47 +09:00
ejclaw
c6b86f6dac runtime: split message agent executor lifecycle 2026-04-11 06:06:02 +09:00
ejclaw
d8b34db92c fix: persist direct terminal verdicts for paired reviewers 2026-04-10 23:41:41 +09:00
ejclaw
4e05f60400 refactor: land approved runtime cleanup and room binding cutover 2026-04-10 21:04:23 +09:00
ejclaw
2e62e125ae Audit outbound delivery and fix reviewer turn routing 2026-04-10 15:22:45 +09:00
ejclaw
c67bd1c622 Finalize turn-role invariants and legacy migration cutover 2026-04-10 11:44:34 +09:00
ejclaw
e1837759c0 Format runtime refactor modules 2026-04-07 23:46:36 +09:00
ejclaw
f9d9c2af6d Refactor paired runtime orchestration 2026-04-07 23:45:53 +09:00
ejclaw
7ad4586985 Restore reviewer wake-up after owner delivery 2026-04-07 18:39:00 +09:00
ejclaw
363b1d408b Fix paired owner completion handoff race 2026-04-07 17:55:50 +09:00
ejclaw
75066f345d Deduplicate paired reviewer handoffs 2026-04-07 08:07:25 +09:00
ejclaw
10e10b499c Complete paired-room SSOT cleanup and logger singleton 2026-04-07 05:12:22 +09:00
ejclaw
ae12b157be Persist paired turn output before final delivery 2026-04-07 03:38:16 +09:00
ejclaw
58b20fad6c Format retry transition refactor files 2026-04-07 02:50:08 +09:00
ejclaw
b8e8c9dabf Unify paired status transitions and retry handling 2026-04-07 02:49:02 +09:00
ejclaw
9d8c97a8e2 Fix reviewer-to-owner follow-up race 2026-04-07 01:22:27 +09:00
ejclaw
48592d97ed Format paired execution refactor files 2026-04-07 00:45:55 +09:00
ejclaw
cbb6bc97c7 Refactor paired execution flow boundaries 2026-04-07 00:45:17 +09:00
ejclaw
f7aeac99c8 Format routing simplification files 2026-04-06 22:45:04 +09:00
ejclaw
b62eebc25b Simplify paired turn routing and prompt builders 2026-04-06 22:44:38 +09:00
ejclaw
7612b2c4cc Format retry rule refactor files 2026-04-06 20:38:38 +09:00
ejclaw
83c45c3ce3 Extract executor retry decision rules 2026-04-06 20:38:10 +09:00
ejclaw
a3dac2b983 Extract paired turn fallback planning 2026-04-06 19:53:26 +09:00
ejclaw
5b832042da Format files touched by executor refactor 2026-04-06 19:40:36 +09:00
ejclaw
8daa3d4779 Refactor agent executor retry and follow-up flow 2026-04-06 19:39:56 +09:00
ejclaw
4650abbd10 Rename readonly reviewer runtime remnants 2026-04-06 17:27:16 +09:00
ejclaw
bf55ad65fd Format files touched by reviewer runtime cleanup 2026-04-06 07:56:59 +09:00
ejclaw
d75d26c23d Stabilize paired reviewer recovery and owner follow-up 2026-04-06 07:49:26 +09:00
ejclaw
2983e0501c Serialize merge-ready owner finalize handoff 2026-04-05 20:11:14 +09:00
Eyejoker
9ff8024bf5 fix: clear container SDK sessions on stale session recovery 2026-04-04 22:06:23 +09:00
Eyejoker
54abb5bb4e Apply formatter after local memory refactor 2026-04-04 03:52:28 +09:00