The scheduled primer reused the opportunistic warm-up gates, so the 18:00 slot
(exactly 5h after 13:00, but firing a few hundred ms later) fell just under
minIntervalMs and was silently skipped as no_eligible_accounts — the primer did
not actually fire at every slot. Add a forceAttempt path: the fixed-slot primer
bypasses stagger, post-failure cooldown, min-interval, and per-account
usage/rate-limit filters, attempting a real Codex command on the active account
at every 08/13/18/23 slot and recording success/failure.
Reviewer/arbiter holding stays removed (per user). This does not pin the reset
clock — a trailing window still slides with later usage — but it guarantees the
"simple task at every fixed slot" the user asked for.
Verified: typecheck, build, bundle-smoke; new regression tests (13:00 success
then sub-5h 18:00 still fires; control without forceAttempt is skipped;
forceAttempt bypasses cooldown/stagger) + usage-primer/codex-warmup suites pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The user asked to NOT defer reviewer/arbiter Codex calls, but instead just fire
a simple primer task at each fixed slot independent of the reviewer. That is
already the live behavior (efc8c00: primer fires at 08/13/18/23 KST + the Claude
binary-path fix). Revert the hold/gating work (e324a1b..ef2b26e) so it can't be
deployed; it is preserved on branch backup/primer-hold-work in case the 08:00
overnight-exhaustion gap later warrants a minimal quiet window.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Address review gaps in the primer-alignment hold so the fixed-slot primer
reliably wins the first-request race for the shared Codex window:
- Extract the time-based hold into a dependency-free leaf module
(codex-primer-alignment.ts) so every Codex path can share it without import
cycles.
- Add a pre-slot hold window (2 min) in addition to the post-slot (5 min) and
the 04-08 KST dawn gap, closing the "request a beat before the slot" gap.
- Hold the dashboard's periodic Codex warm-up too: runCodexWarmupCycle now
skips during the hold unless called with isPrimer (the scheduled primer sets
it), so the warm-up can't anchor the window ahead of the primer.
- Remove the urgent @-mention bypass on the reviewer/arbiter and owner holds —
the reset is anchored unconditionally as requested.
- Reviewer/arbiter hold resumes exactly once after the window via the existing
per-revision claim (poll re-detect; no double run).
Note: the unified lease boundary (syncHostCodexSessionFiles) is synchronous and
throws a terminal "Codex unavailable", so it can't host a clean defer+resume;
gating stays at the async turn-scheduling layer that can re-queue. Honest caveat
unchanged: making the primer first is necessary but not sufficient to pin a
trailing reset.
Verified: typecheck, build, bundle-smoke; full suite 1482 passing with only the
9 pre-existing env-config baseline failures (service-routing/paired-context/
migrate-room owner=claude vs codex-main); new alignment + warmup-hold tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The usage primer fires at 08/13/18/23 KST but could not pin the Codex 5h
reset because reviewer/arbiter turns on the same shared account ran off-slot
and anchored the window first. Add a time-based hold
(shouldHoldCodexForPrimerAlignment) that keeps non-primer Codex turns quiet
during the 04-08 KST dawn gap and for a few minutes after each slot, so the
scheduled primer wins the first-request race. Applied at the reviewer/arbiter
dispatch site (the actual off-slot consumer here) and for codex-owner rooms;
urgent @-mention turns bypass it. Time-based by design — the usage/reset
figures are exactly the unreliable data, so a blunt clock rule is predictable
and testable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Live evidence shows the Codex 5h limit is a trailing rolling window whose
reported reset slides forward with continued usage (observed 18:00 → 18:33
over ~33 min). A timed primer therefore cannot pin the reset to a fixed clock
time, and the previous "retry at reported reset" logic was both based on a
wrong fixed-window model and buggy under sliding resets (stale reset target,
single-shot). Revert it.
Keep the verified Claude primer binary-path fix (resolve repo-root
node_modules) and document the real rolling-window behavior so the primer is
correctly framed as a best-effort warm-up + success/failure record.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two evidence-based fixes for the usage-window primer that aligns the 5h
reset to fixed KST slots (08/13/18/23):
1. Claude primer always failed with binary_missing: it only looked under
runners/agent-runner/node_modules, but the SDK platform binary is hoisted
to the repo-root node_modules. Resolve both locations.
2. When a Codex slot lands while the 5h window is still active/exhausted, the
warm-up call fails with a rate-limit and the window only re-anchors at its
natural reset — leaving Codex un-primed until the next 5h slot. Parse the
account's known reset time and schedule a one-shot retry right after it so
the next window anchors close to the slot.
Also corrected a stale comment claiming a 04:00–08:00 gap is enforced by
message-runtime-gating (it is not).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Migration 020 (arbiter_intervention_count) was added without updating the
hardcoded expected list in bootstrap.test.ts, breaking its schema-migration
assertions. Add the entry so the canonical `bun run test` suite passes.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The owner↔reviewer↔arbiter loop could repeat forever: when the arbiter
ruled PROCEED/REVISE/RESET it reset round_trip_count to 0, and nothing
tracked how many times the arbiter had already intervened. A re-deadlock
re-invoked the arbiter without bound.
Add a persistent arbiter_intervention_count (new column + migration 020)
that survives the round-trip reset, and a configurable cap
ARBITER_MAX_INTERVENTIONS (default 1). Once the arbiter has intervened
that many times and the loop still deadlocks, requestArbiterOrEscalate
escalates straight to the user instead of re-invoking the arbiter.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Merging origin/main collided on schema version 15: the deployment DB
recorded v15 as `reviewer_failure_count` (downstream-only), while upstream
defines v15 as `turn_progress_text`. The runner tracks applied migrations
by version number, so it skips upstream's `turn_progress_text` on the live
DB and never creates paired_turns.progress_text / progress_updated_at —
columns the merged runtime reads and writes (db/paired-turns.ts,
web-dashboard-data.ts), which would crash the live paired-room flow.
Add idempotent migration 019 to re-create those columns under a free
version number. No-op on fresh installs where migration 015 already ran.
Caught by a pre-flight boot of the merged code against a copy of the live DB.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
The Codex usage primer spawned a bare `codex` binary, but under bun/systemd
there is no global `codex` on PATH, so every slot failed with ENOENT and the
primer never ran. Resolve the vendored @openai/codex JS launcher and run it
through the current JS runtime (matching the codex-runner), with global-binary
and PATH fallbacks.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
The Codex-only shouldHoldFreshCodexUntilPrimer gate blocked fresh Codex
turns outside the dawn window and replied "Codex 사용량 정렬 대기 중...
다음 프라이머 시각 이후 다시 보내주세요", which Claude rooms never did.
The primer (usage-primer.ts) already fires unconditionally for both
providers at 08/13/18/23 KST, so this hold only delayed user turns
without affecting window alignment.
Remove the gate and its now-orphaned helpers (shouldHoldFreshCodexUntilPrimer,
CODEX_ALIGNMENT_AUTO_REPLY, msUntilNextPrimerSlotKST, formatPrimerHourKST,
and the getAllCodexAccounts import). Keep the 04-08 KST dawn alignment gap
block and the usage-exhaustion gate untouched.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
The previous fix gated the step_done deadlock guard on the same
deadlockThreshold every verdict uses — but handleReviewerCompletion
passes Infinity for that threshold when no arbiter is configured
(isArbiterEnabled()=false, e.g. production with the ARBITER__AGENT_TYPE
env typo). So step_done never tripped the guard and the
owner TASK_DONE ⇄ reviewer STEP_DONE loop still ran forever.
Give step_done its own always-finite threshold
(stepDoneDeadlockThreshold = ARBITER_DEADLOCK_THRESHOLD) independent of
arbiter availability. Once hit, requestArbiterOrEscalate routes to the
arbiter when enabled, or completes the task with completion_reason
'escalated' (user escalation) when not. Productive REVISE/continue
disagreement still uses deadlockThreshold and is intentionally left on
the existing policy (loops to owner when arbiter is off), matching the
existing "returns reviewer change requests to owner ... deadlock
threshold" test.
Adds a shared regression unit test (deadlockThreshold=Infinity +
step_done still escalates) and three integration tests through
completePairedExecutionContext: arbiter-off+threshold -> escalated,
arbiter-on+threshold -> arbiter_requested, below-threshold -> active.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
The reviewer step_done verdict mapped to request_owner_changes
unconditionally, bypassing the deadlock guard that every sibling
verdict respects. When the owner kept replying task_done and the
reviewer kept replying step_done, the room oscillated forever,
spamming duplicate reviewer messages into the channel.
Apply the same `roundTripCount >= deadlockThreshold` guard used by
the done_with_concerns/continue/default branch so step_done
oscillation escalates to the arbiter (or completes via escalation
when no arbiter) instead of looping. Reuses existing round_trip_count
and deadlockThreshold; no migration needed.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Per-turn agent runners do heavy file I/O (git clone, builds, ffmpeg/video
scratch) inside the ejclaw service cgroup. The kernel retains those reads as
reclaimable page cache, so systemd's MemoryCurrent climbs to 4-5GB even though
real anon usage stays ~300MB and the host has 13GB free. Add a soft MemoryHigh
so the cache is reclaimed under pressure before the number balloons; MemoryMax
stays unset (infinity) so a legitimate burst is throttled, never OOM-killed.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
editMessage is called repeatedly by the progress ticker and dashboard
refresh, so appending overflow chunks there duplicated text on every edit.
Revert editMessage to a pure single-message edit and instead expose the
platform per-message cap via Channel.maxMessageLength; deliverOpenWorkItem
now skips the doomed in-place edit and hands the full payload to the
chunking sendMessage path when the final exceeds that cap.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
- editMessage now splits >2000-char text: edits the tracked message with the
first chunk and appends the rest as follow-up messages, instead of throwing
Discord "Invalid Form Body … Must be 2000 or fewer" when promoting a long
owner result over a progress message.
- syncTrackedProgressMessage captures latestProgressText in a local before the
editMessage await, fixing a race where a concurrent resetProgressState()
nulled it and crashed with "null is not an object (this.latestProgressText.length)".
- Add editMessage regression tests (in-place edit vs. chunked overflow).
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
- mark Codex bearer/refresh failures as terminal auth-expired states
- sync refreshed session auth back to rotation slots and revive refreshed dead_auth slots
- stop paired arbiter retry loops when Codex accounts are unavailable
- add regression coverage for rotation leases, follow-up suppression, and arbiter closure
Raise CODEX_PRIMER_MAX_USAGE_PCT from 1 to 100 so the Codex usage-window
primer fires at every fixed KST slot regardless of current 5h usage level,
mirroring the Claude primer (which always fires unless rate-limited). The d7
gate stays at 100 so an account whose weekly quota is exhausted is still
skipped, matching Claude's rate-limit skip.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Codex primer was skipping with no_eligible_accounts because it read a
stale usage cache and required exactly 0% usage. Re-query usage right
before the primer call and treat freshly-reset 0~1% accounts as eligible
so the 5h window can be anchored at the fixed KST slot. Also hold new
Codex-owner turns at fresh usage until the next primer slot.
Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>
Follow-up to e3c8de6: prettier --check failed on the new section
(missing blank line after the "Before accepting any proposal..."
sentence ahead of the numbered list, and the "Do not include:" header
needed a blank line before the bullets). Running prettier --write
fixes those without changing semantics.
bunx prettier --check prompts/owner-common-paired-room.md → clean.
The user only sees the final owner message, not the owner↔reviewer
back-and-forth. Recent turns leaked meta-narrative like "리뷰어 지적
반영해서 정정합니다" / "PROCEED 확정" into user-visible replies and
sometimes omitted explicit action items, leaving the user to guess
whether anything was needed from them.
Add a "Finalize message format (self-contained answer)" section to
owner-common-paired-room.md that mandates:
- direct answer first
- consolidated recap (no transcript, no narrating disagreement)
- explicit "사용자 액션 아이템" section, with "없음" when there is
nothing for the user to do
- no meta phrases referencing the reviewer loop
The runner reads this prompt fresh per agent spawn
(src/agent-runner-environment.ts), so the change takes effect on the
next owner turn without a service restart.
One-off Bun script that logs each configured Discord bot in (owner /
reviewer / arbiter), walks every guild it sees, and prints the bot
member's effective permission bits for the capabilities we plan to
expose (ReadMessageHistory, ManageMessages, ManageChannels,
AddReactions, thread perms, etc.). Used to verify "current permissions
support feature X" empirically instead of guessing from intents.
Follow-up to 6505971: the pre-commit prettier hook rewrapped the new
import and an Array.from() call after the commit landed, leaving the
working tree dirty. Commit the reflow so the tree is clean.
Long bot replies that exceeded Discord's 2000-char limit were split by a
naive byte-slice loop. When the split fell inside a ```fenced code block
the first chunk lost its closing fence and the next chunk lost its
opener, so Discord rendered raw backticks and the remainder as headings
or bold instead of code.
Replace the slice loop with chunkForDiscord(), a fence-aware splitter
that prefers newline boundaries, re-closes any open fence at the end of
a chunk, and reopens it (with the same language tag) at the start of the
next. Single oversize lines still fall back to a surrogate-pair-safe
byte split. Adds 10 unit tests covering the seam, multi-fence
documents, surrogate pairs, and single-line overflow.
