diff --git a/groups/global/CLAUDE.md b/groups/global/CLAUDE.md index 1712487..57124d4 100644 --- a/groups/global/CLAUDE.md +++ b/groups/global/CLAUDE.md @@ -1,287 +1,37 @@ # Global Memory -This file is for mutable memory shared across Claude groups. +Shared, mutable memory for every room. Keep it short: only rules every room needs. +Detailed notes live in `/home/claude/EJClaw/groups/global/ops/` — read the matching file when a task touches that topic: -Use it for durable facts, preferences, and shared context that may change over time. +- `ops/host.md` — GPU (RTX 5050, `/home/claude/gpu-venv`, NVML mismatch fix), tmpfs/RAM rules and watchdogs, Windows CI VM on .9 (nested KVM, resource caps), .5 Docker host and `docker5` context +- `ops/credentials.md` — what is in `secrets.json`, git.tkrmagid.kr credential helper, Claude/Codex per-service credentials and re-login runbooks +- `ops/ejclaw-ops.md` — room mode policy, channel register/deregister, safe restarts, model upgrade checklist, current versions, recent EJClaw fixes +- `ops/projects.md` — javis_bot deployment target, washing_machine_app release process -Do not store platform-wide operating rules here. Those now live in `prompts/claude-platform.md`. +## User preferences -## Host hardware (GPU) +- Answer the user in Korean 존댓말 unless asked otherwise. +- GPU: ask before running GPU workloads, except for the `javis_bot` project (pre-approved). -This host has an NVIDIA GPU available for compute: -- GPU: NVIDIA GeForce RTX 5050 (GB207, Blackwell), 8GB VRAM, compute capability 12.0 (sm_120) -- Driver 595.x (open kernel module), CUDA runtime 13.2; `libcuda.so` present so GPU compute is usable -- Driver packages auto-update via apt. After an update the userspace libs move ahead of the still-loaded kernel module, and `nvidia-smi` fails with `Failed to initialize NVML: Driver/library version mismatch` (CUDA compute through `/home/claude/gpu-venv` keeps working meanwhile). Fix without a reboot when `modinfo -F version nvidia` already equals the new userspace version: stop the GPU holders (`systemctl --user stop wsai-voice vncserver`, `sudo systemctl stop gdm ollama`; check `sudo fuser /dev/nvidia*` is empty), `sudo modprobe -r nvidia_drm nvidia_modeset nvidia_uvm nvidia`, `sudo modprobe nvidia nvidia_uvm nvidia_modeset nvidia_drm`, then start the four services again. Done 2026-10-01 (595.84 → 595.91.07, ~2 min of GPU/desktop downtime; ejclaw/kaclaw unaffected). A newer kernel (7.0.0-34) is also installed and only takes effect on reboot. -- Ready-to-use PyTorch (CUDA 12.8 / Blackwell-capable) venv at `/home/claude/gpu-venv` — run with `/home/claude/gpu-venv/bin/python`. `torch.cuda.is_available()` is True. -- For CUDA C compilation, `nvcc` is not installed; install the CUDA Toolkit only if a task needs it. +## Host safety (always) -Behavioral rule: when a task could meaningfully benefit from the GPU (ML inference/training, heavy parallel compute, large media processing), first ask the user whether they want to use the GPU before doing so. Do not silently run GPU workloads. Once confirmed, use the `/home/claude/gpu-venv` Python (or install what the task needs) and verify with a real GPU run. +- `/tmp` and `/dev/shm` are RAM (tmpfs). Never put venvs, models, datasets or other large files there; use disk paths under `$HOME` (`TMPDIR=$HOME/.tmp`). +- Heavy jobs (GPU, VMs, big builds) must not run inside the bot's cgroup: launch them with `systemd-run --user --scope -p MemoryMax=... ...`. +- Never run a plain `systemctl --user restart ejclaw.service` inside a turn (it kills you). Room binding changes use SIGHUP; code/env changes use the detached `systemd-run` restart. Verify before claiming success. Details: `ops/ejclaw-ops.md`. -- 2026-06-18 exception (javis_bot project): the user pre-approved GPU use for the `javis_bot` project — do NOT ask before using the GPU on javis_bot tasks. Other projects still follow the ask-first rule above. +## Sending files -## javis_bot deployment target +Only files under `/home/claude/EJClaw/data/attachments` can be attached. Copy the file there (prefer `data/attachments/generated/`, auto-deleted after 60 min) and reference it with a `MEDIA:` line. Do not widen the allowlist. -- 2026-06-18: the real production deployment runs on the user's Windows 11 gaming PC (CPU Ryzen 9 9950X3D, GPU RTX 5070Ti — Blackwell, same sm_120 arch as this host's RTX 5050) with Docker. Use `docker-compose.yml` + `docker-compose.gpu-windows.yml` (Docker Desktop + WSL2 GPU). This Linux host is only the LAN browser host (JARVIS_ROLE=browser); test changes here as a Blackwell-GPU proxy but remember the bot/brain role actually runs on the Windows box. The `1516396486265405548` value the user pasted on 2026-06-18 was just a Discord message ID in the chat channel, not a guild/voice-channel ID. +## Credentials -## Host RAM / disk safety (tmpfs rule) +Stored at `/home/claude/.config/ejclaw/secrets.json` (read with the Read tool when needed). Never paste a secret into chat or into memory files. Each entry's `note` explains its usage rules; see `ops/credentials.md`. -`/tmp` (16G) and `/dev/shm` (16G) are RAM-backed tmpfs. Anything large written there consumes RAM, not disk. +## Rooms -- 2026-06-11 incident: MeloTTS Korean-TTS venv (~2.4GB) was installed into `/tmp/melo311` (tmpfs), eating RAM and risking OOM. Fixed by moving it to disk at `/home/claude/jarvis-tts/melo311` and updating 43 script shebangs. -- Recurrence guards (in place, defence-in-depth): - 1. `~/.bashrc` exports `TMPDIR=$HOME/.tmp`, `UV_CACHE_DIR=$HOME/.cache/uv`, `PIP_CACHE_DIR=$HOME/.cache/pip` (all ext4 disk), above the interactive-shell guard so non-interactive agent shells inherit it. - 2. The `ejclaw.service` user unit (`~/.config/systemd/user/ejclaw.service`) sets the same three `Environment=` vars, so every child process (agent runners, install/build steps that never source `.bashrc`) inherits disk temp at the systemd root. Applied via `daemon-reload`; takes full effect on the next `ejclaw.service` restart. Do NOT restart it from inside a live agent turn (the agent is its descendant). - 3. Watchdog `/home/claude/tmpfs-ram-watchdog.sh` (user crontab `*/30`) posts once per episode to the EJClaw #상태 channel when `/tmp` or `/dev/shm` usage > 3GB or MemAvailable < 4GB, re-arms after recovery, and deletes `$HOME/.tmp` files older than 3 days. Token-free (Discord REST with the owner bot token, no agent run); log `/home/claude/tmpfs-ram-watchdog.log`, latches under `~/.tmpfs-ram-watchdog/`. Test with `DRYRUN=1 TMPFS_MAX_MB=100 /home/claude/tmpfs-ram-watchdog.sh`. This catches an explicit `/tmp/...` write that bypasses `TMPDIR`. (The original watchdog had silently disappeared; restored 2026-10-01.) Disk space is watched separately by `/home/claude/disk85-alert.sh` (cron `*/10`, @everyone alert at ≥85% root usage). -- Rule: NEVER install venvs, models, datasets, or any multi-GB artefact into `/tmp` or `/dev/shm`. Use disk paths under `$HOME` (e.g. `/home/claude/jarvis-tts`). If a tool ignores `TMPDIR`, pass an explicit disk path. EJClaw runs directly via `bun` under the user systemd unit (no Docker/compose in use here). -- Root filesystem `/` (ext4) is 506G total, ~221G free (55% used) as of 2026-10-01 (it was 156G in early June, then 353G). RAM is 30GB as seen by the guest (Proxmox shows 32GiB and counts guest page cache as "used", so its ~85% figure is misleading — judge by `free -h` → `available`). Adding RAM is not the real fix for tmpfs blowups; keeping big artefacts on disk is. -- The two TTS scratch scripts (`/home/claude/jarvis-tts/gen_melo.py`, `gen_xtts.py`) now route output to disk and hard-refuse any `/tmp` or `/dev/shm` write path via a `disk_path()` guard. -- 2026-07-26 OOM incident + `MemoryHigh` change: the `ejclaw.service` base unit set `MemoryHigh=3G` (soft cgroup cap; `MemoryMax` unset). An agent-launched GPU/diffusers job in the `anyway2` channel (`gen_banner.py`, ~2.9GB RSS on its own) ran INSIDE the bot's cgroup, pushed total past 3G, and `systemd-oomd` oom-killed `ejclaw.service` ~6× in a row (systemd auto-restarted each time → restart loop; the banner also never finished). Fix: raised the soft cap to `MemoryHigh=16G` via a documented drop-in `~/.config/systemd/user/ejclaw.service.d/10-memory-high.conf` (applied with `daemon-reload`, no restart needed for a cgroup-property change). Rationale: host has 30G, KAclaw is capped at 3G and other services ~2G, so 16G leaves ~11G+cache headroom while still capping runaways; `MemoryMax` stays infinity so a burst is throttled, not hard-killed. The base unit still reads 3G — the drop-in overrides it (`systemctl --user cat ejclaw` shows both). Cleaner long-term alternative (not done): keep the bot cap tight and launch heavy GPU jobs OUTSIDE the bot cgroup via `systemd-run --user --scope`. - -## Outbound attachment allowed folders - -Discord/outbound attachments are only delivered if the file lives inside an allowlisted directory; files elsewhere are rejected as `outside-allowed-dirs` and the bot now appends a visible "전송 실패" notice (it no longer drops silently). - -**Standing rule (2026-06-12 user decision):** the single canonical outbound attachment folder is `/home/claude/EJClaw/data/attachments` (the EJClaw DATA_DIR `attachments` dir). It is already allow-listed by default — no `.env` registration is needed. To send ANY file, first write or copy it into that folder (a subfolder is fine, e.g. `data/attachments/generated/`), then reference that path in the `MEDIA:` directive. Do NOT widen the allowlist to other folders (the earlier idea of allow-listing `/home/claude/jarvis-tts` was reverted at the user's request, and auto-copying arbitrary out-of-allowlist files in the delivery layer was also reverted because it bypasses cross-room / sensitive-file protection). - -- Concretely: when attaching generated media (TTS audio, images, etc.), `cp` it into `/home/claude/EJClaw/data/attachments/` (or a subfolder) and emit `MEDIA:/home/claude/EJClaw/data/attachments/...`. This needs no restart, no env change, no code change. -- The TTS scripts under `/home/claude/jarvis-tts` still write their working output there; copy the resulting `.wav` into `data/attachments` before attaching (jarvis-tts itself is NOT allow-listed). -- System defaults still allowed for internal use: the codex `generated_images` dir and the system temp dir (used for auto-generated screenshots). Only `data/attachments` is the place WE deliberately stage files to send. -- Safety net: if a file is ever referenced from outside an allowed folder, delivery now appends a visible "전송 실패 (사유)" notice instead of silently dropping it. -- **Auto-cleanup (2026-06-12 user decision):** prefer `/home/claude/EJClaw/data/attachments/generated/` for files we send. A user crontab entry sweeps that subfolder every 15 min and deletes files older than 60 min (`find .../data/attachments/generated -type f -mmin +60 -delete`). The sweep is scoped to `generated/` only, so system-managed attachments under `data/attachments` (e.g. `paired-turn-outputs/`, dashboard copies) are never touched. If a sent file must persist longer than ~1h, put it directly under `data/attachments/` (not `generated/`). - -## Stored credentials - -Shared credentials live at `/home/claude/.config/ejclaw/secrets.json` (chmod 600, owner-only). Read with the Read tool when a session in any channel asks to "저장해둔 계정토큰으로 로그인" or otherwise needs a registered token. - -Schema: `credentials..{type, host, token, note, added_at}`. - -Currently stored: -- `git.tkrmagid.kr` — Gitea personal access token. Use via `Authorization: token ` header, or embed in HTTPS URL as `https://:@git.tkrmagid.kr/...`. For `git clone/push`, prefer the URL form or `git -c http.extraHeader="Authorization: token " clone ...`. Do not paste the raw token into chat replies. -- `sudo` — local sudo password for the `claude` user on this host. Use via `echo "$PW" | sudo -S ` (read the password from `credentials.sudo.password` with the Read tool, then pipe). Do not paste the raw password into chat replies. -- `discord.com/bot/testbot` — Discord test bot (테스트봇, client/app ID `1538122882528321536`), stored 2026-08-15 for use across all chats. Read the token from `credentials["discord.com/bot/testbot"].token` in secrets.json with the Read tool (token is NOT written here). Use as `Authorization: Bot ` against `https://discord.com/api/v10`, or as the bot token for a discord.js/gateway login. Do not paste the raw token into chat replies. Token was chat-exposed on save; if used for anything sensitive/long-term, rotate it in the Discord Developer Portal first. Re-ask the user if Discord returns 401. -- Other entries (each has its own `note` with usage rules — read it before use): `docker5.ssh` (root password fallback for .5; prefer key auth), `kis.koreainvestment` + `dart.fss.or.kr` (stock-adviser APIs), `huggingface.co` (read token), `anthropic.api` (stock-adviser API key with a monthly budget cap; also usable to list models via `GET /v1/models`), `tossinvest.com` (IP-allowlisted), `washing_machine_app.keystore`, `api.smartthings.com`, `discord.com/oauth/virtual-stock-site-bot`, `youtube.oauth/music_bot_v2`, and per-repo GitHub PATs `github.com/Kacarong/{style-ai-site,flight-tracker,stock-adviser,virtual-stock-site-bot,cgv-macro}` (use only via ephemeral push URLs, never persist in `.git/config`). - -To add a credential: edit `secrets.json`, append a new entry under `credentials`, and add the key + intended use to this list. Never paste raw secret values into chat or into this file. - -Git remotes on git.tkrmagid.kr no longer embed the token in the URL (cleaned 2026-10-01 in EJClaw, music-bot-test, tts_site, voice_copy_work, vp_build, groups/mc_chat_answer_mod, groups/mc_domain_proxy, groups/tts_bot). Each repo has `credential.https://git.tkrmagid.kr.helper=/home/claude/.config/ejclaw/git-credential-tkrmagid`, which serves the `git.tkrmagid.kr` token from secrets.json (keeps the URL's username, default `claude-bot`). For a new clone: use a token-free URL and run `git config --local credential.https://git.tkrmagid.kr.helper /home/claude/.config/ejclaw/git-credential-tkrmagid`. Rotating the token now only needs a secrets.json edit. - -## User communication preferences - -- 2026-05-27: 모든 채팅에서 사용자에게 답변할 때 기본적으로 존댓말을 사용한다. 특별히 반말을 요청받지 않는 한 반말로 답변하지 않는다. - -## Room mode policy - -All rooms default to `tribunal` (paired) mode. Owner (claude-code) runs the work, the reviewer (codex, `REVIEWER_AGENT_TYPE=codex`) verifies, and the arbiter (claude-code) breaks deadlocks. New rooms registered via `bun setup/index.ts --step register` are `tribunal` by default and get their paired workspace (`work_dir`) provisioned at registration. A room that ends up tribunal with `work_dir = NULL` (e.g. created through another path) is now self-healed on its first turn (fix c7a8a43, 2026-09-19); before that fix such rooms silently dropped follow-up messages. - -If the user in a channel says any of these — "클로드 사용하지 말자", "paired 모드 끄자", "리뷰어 끄자", "이 방은 single 로 바꿔줘" — switch that channel back to `single` mode. Use: - -```bash -bun -e "import { initDatabase, setExplicitRoomMode } from './src/db.js'; initDatabase(); setExplicitRoomMode('', 'single');" -``` - -The reverse phrase ("paired 켜자", "리뷰어 다시 켜자") flips it back to `'tribunal'`. Acknowledge the change in chat and confirm the new mode. - -## Room deregistration (채팅 등록 해제 + 데이터 정리) - -When the user says any of — "채팅 아이디 등록해제 해줘 ", "이 채널 등록 해제해줘", "채널 등록 해제" (with a channel id / JID) — the standing behavior (user decision 2026-08-18) is: fully unregister the room AND purge all managed data, keeping ONLY the chat channel record and its message history, then hot-reload (SIGHUP, NOT a full restart — see below) so the channel goes unregistered in the live process. - -Use the canonical script (keeps `chats` + `messages`, removes everything else — room_settings/role/skill overrides, paired tasks/turns/attempts/outputs/reservations/leases/projects/handoffs, work_items, scheduled_tasks, task_run_logs, sessions, router cursor, and the on-disk `groups/`, `data/workspaces/`, `data/sessions/`, `data/ipc/` including any git worktree): - -```bash -cd /home/claude/EJClaw -bun scripts/deregister-room.ts --dry-run # preview what will be removed -bun scripts/deregister-room.ts # execute the purge -``` - -Accepts bare Discord channel IDs (auto-prefixed to `dc:`) or full JIDs; multiple ids allowed. Safety built in: a group folder still used by another room is NOT deleted on disk (only that chat's DB rows); `--dry-run` writes nothing. - -Applying to the live process is now automatic: after the purge the script calls `signalEjclawReload()`, which sends `SIGHUP` to the running service's main PID → `runtimeState.reloadRoomBindings()`, so the room drops from live bindings with NO full restart. Do NOT run a plain `systemctl restart` from inside a turn (it kills the agent mid-command and can't be confirmed). Verify by the `Room bindings reloaded` log or `getRegisteredGroup(jid)` → undefined. Always take a quick DB backup first for irreversible purges: `cp store/messages.db /home/claude/ejclaw-db-backup-$(date +%Y%m%d-%H%M%S).db`. - -## Channel registration + making it live (no false restarts) — MANDATORY - -The agent runs INSIDE `ejclaw.service`, so a plain `systemctl --user restart ejclaw.service` from within a turn kills the agent mid-command: the restart is unreliable and you CANNOT confirm it in the same turn. Never claim a restart/reload happened without evidence — the repeated "재시작 됐다고 했는데 안 됨" complaints came from claiming success without verifying. - -When you register (or deregister) a channel via a DB write (bun script / `setup --step register`) from a room, the running process does NOT see it until its in-memory room bindings are reloaded. Preferred way — hot-reload, NO full restart, agent survives so you can verify in the same turn: - -```bash -kill -HUP "$(systemctl --user show -p MainPID --value ejclaw.service)" -``` - -`src/index.ts` handles `SIGHUP` → `runtimeState.reloadRoomBindings()` (re-reads ONLY room bindings from the DB; message cursors/sessions untouched, so no re-processing). Sending HUP to the MainPID only (not the whole cgroup) leaves agent subprocesses alive. After it, VERIFY before reporting: look for the `Room bindings reloaded` log with the new `groupCount`, or check `getRegisteredGroup(jid)` resolves. Standing user rule: registering a channel must be followed immediately by this live-reload (this is the "등록하면 자동 재시작/반영" the user asked for) — do it once, verify, then report. - -Only when a change truly needs a full process restart (e.g. code/env changes), use the detached form so it survives the agent's death, and verify in a SEPARATE scheduled task (new PID / start time) — do it exactly once: - -```bash -systemd-run --user --collect --unit="ejclaw-restart-$$" systemctl --user restart ejclaw.service -``` - -## washing_machine_app (EasyAppliance Android) deployment - -- 2026-08-06 user standing rule: for the washing_machine_app project, ALWAYS deploy after a change and deliver via in-app update — do not just commit. Every meaningful change ends with a published Gitea release so the in-app updater offers it. -- Build env: default JDK on this host is Java 25, which the Gradle Kotlin compiler can't parse ("IllegalArgumentException: 25.0.3"). Build with `export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64`. Android SDK at `/home/claude/android-sdk` (build-tools 35.0.0 has aapt/apksigner). -- Release process: bump `versionCode`/`versionName` defaults in `app/build.gradle.kts` (also passable via `-PverCode -PverName`), `./gradlew assembleRelease` (auto-signed via `keystore.properties` → keystore at `/home/claude/.config/ejclaw/washing_machine_app.keystore`, release cert SHA-256 `56246bd2…`; every release MUST use this same key or in-place update breaks). Then create a Gitea release + upload the APK asset named `EasyAppliance-v.apk`. -- Release host: Gitea `git.tkrmagid.kr`, repo `tkrmagid/washing_machine_app`. Use the stored `git.tkrmagid.kr` token (secrets.json). API: `POST /api/v1/repos/tkrmagid/washing_machine_app/releases` then `POST /releases//assets?name=...` with the APK. The app's `UpdateRepository` reads releases via the public API and offers any release whose tag (minus `v`) is newer than installed. Latest published: v0.3.9 (verCode 12) as of 2026-08-06. Token now stored encrypted (AndroidKeyStore AES/GCM via TokenCrypto); allowBackup=false, usesCleartextTraffic=false. -- No git remote is configured in the owner worktree; the Gitea release (APK) is the delivery mechanism, separate from any source push. -- Real-device tuning: the user's SmartThings Personal Access Token is stored in secrets.json under `api.smartthings.com` (Bearer, `https://api.smartthings.com/v1`). Use it to read actual device capabilities/supported values so preset/status mappings stay accurate. Their real devices: 세탁기 `b9b0b550-…` (water temp enum none/cold/20/30/40/60/90; spin `custom.washerSpinLevel`; rinse `custom.washerRinseCycles`; NO manual water level — auto), 벽걸이/스탠드 에어컨 (mode cool/dry/wind/aIComfort; fan auto/1/2/3/4/max; setpoint 16–30). SmartThings commands are type-strict: appliance enum args (washer temp "40", fan "1", rinse "3") are STRINGS; only setpoint-style commands are NUMBERS — `SmartThingsDeviceRepository.encodeArg` handles this via a NUMERIC_COMMANDS allowlist. - -## Git backups - -- 2026-05-27 10:53 KST: EJClaw 정상 동작 상태를 git commit `1509108` (`backup current stable ejclaw state`)로 백업했다. "이번 요청만 리뷰어 사용" 기능 작업은 이 백업 커밋 이후에 시작한 변경이다. - -## Per-service Claude credential isolation (user changes, verified 2026-06-23) - -The user reworked credential handling so each service reads its OWN Claude credentials file instead of all sharing `~/.claude/.credentials.json`. Goal: run each service on a different Claude account. - -- Resolver: `src/claude-credentials-path.ts` — `getClaudeCredentialsPath(accountIndex, {allowHomeFallback})` and `hasExplicitClaudeCredentialsPath()`. Order: account 0 → `CLAUDE_CREDENTIALS_PATH`; account 1+ → `CLAUDE_ACCOUNTS_DIR//.credentials.json`; else home fallback (`~/.claude` or `~/.claude-accounts/`). The same module exists in both EJClaw and KAclaw. -- Wired into: `setup/login.ts` (writes to `CLAUDE_CREDENTIALS_PATH`; PKCE state file is now per-path-hashed `ejclaw-claude-login-.json` so different-account logins don't collide), `src/token-refresh.ts`, `src/claude-usage.ts`, `src/agent-runner-environment.ts` (pre-syncs creds into each session `.claude/.credentials.json`), `src/runtime-inventory.ts`. -- Credential paths in use: - - claude CLI: `~/.claude/.credentials.json` - - EJClaw: `/home/claude/EJClaw/data/claude/.credentials.json` (set via `CLAUDE_CREDENTIALS_PATH` in `EJClaw/.env`) - - KAclaw: `/home/claude/KAclaw/data/claude/.credentials.json` (set via `CLAUDE_CREDENTIALS_PATH` in `KAclaw/.env`) -- Relevant env flags (EJClaw and KAclaw both): `CLAUDE_USAGE_USE_HOME_CREDENTIALS=false`, `CLAUDE_TOKEN_REFRESH_USE_CREDENTIALS=true`, `CLAUDE_TOKEN_REFRESH_USE_HOME_CREDENTIALS=false`. The token-refresh loop only runs when `CLAUDE_TOKEN_REFRESH_USE_CREDENTIALS=true` OR `*_USE_HOME_CREDENTIALS=true`. -- To re-auth ONE service: run its login flow with that service's `CLAUDE_CREDENTIALS_PATH` exported, then send the new `code#state`. Do NOT copy one service's `.credentials.json` to another path — that collapses them onto one token + one shared refresh token, and when both refresh independently the refresh token rotates and the other side breaks with `invalid_grant`. -- Account-identity reality (verified via `https://api.anthropic.com/api/oauth/profile`, 2026-06-23 ~19:35 KST): despite the intent, all three paths still resolved to the SAME account `tkrmagid@gmail.com` (Max). CLI and KAclaw even held byte-identical access+refresh tokens (from an earlier `~/.claude`→KAclaw copy); EJClaw held a different token of the same account. Before assuming the services are on distinct accounts, verify the actual account by calling the profile API for each path's accessToken (compare `account.email`), not just by checking that the file paths differ. -- KAclaw is NOT a git repo (no `.git`), so its source changes can't be diffed/version-controlled. 2026-06-23 dashboard usage-row fix (utilization of exactly 1 rendering as 100%, `> 1` → `>= 1` in `dashboard-usage-rows.ts`) lives only in the KAclaw working tree. EJClaw's copy of that file may carry the same `> 1` pattern — worth checking if the EJClaw dashboard ever shows a window at exactly 1%. -- UPDATE 2026-06-23 ~21:26 KST: KAclaw was re-logged onto a DIFFERENT account at the user's request. Now EJClaw = `tkrmagid@gmail.com` (Max) and KAclaw = `hjj100411@gmail.com` (NOT Max — has_claude_max=false, so Pro/Free, lower limits). Done via KAclaw's own `setup/index.ts --step login` with `CLAUDE_CREDENTIALS_PATH=/home/claude/KAclaw/data/claude/.credentials.json` exported, then updated KAclaw `.env` CLAUDE_CODE_OAUTH_TOKEN(S) to the new access token and restarted `kaclaw.service`. claude CLI (`~/.claude`) is still `tkrmagid@gmail.com`. So EJClaw and KAclaw are now genuinely different accounts; verify with the profile-API email check if in doubt. - -## Codex (reviewer) credential isolation (2026-07-22) - -EJClaw's paired-room **reviewer** runs as Codex (gpt-5.5) with ChatGPT-OAuth auth. It shared `~/.codex/auth.json` with other consumers (manual `codex` CLI, KAclaw same-home), so refreshes rotated the token and the stale copy died with "access token could not be refreshed because your refresh token was already used." Every reviewer turn then failed → task ended `reviewer_codex_unavailable` and escalated straight to the arbiter, so chats showed owner + arbiter but the reviewer never posted. Diagnose via `paired_turn_attempts.last_error` in `store/messages.db`. - -Fix (mirrors the Claude per-service isolation above): EJClaw now uses a dedicated Codex account dir `~/.codex-accounts/0/` instead of shared `~/.codex`. `src/codex-token-rotation.ts` `initCodexTokenRotation()` loads numbered dirs under `~/.codex-accounts/` as the rotation pool and ignores the `~/.codex` fallback when the dir has entries; EJClaw leases + writes-back refreshes to `~/.codex-accounts/0/auth.json` (`syncCodexSessionAuthBack`), isolated from the CLI and KAclaw. - -- **Activation needs an ejclaw restart** — `initCodexTokenRotation()` runs once at process start (`src/index.ts`), so a running ejclaw keeps the old cached account until `systemctl --user restart ejclaw`. Do NOT restart from inside a live agent turn (the agent is a descendant). -- **Re-auth runbook (the procedure CHANGED):** refresh the reviewer's Codex login into the isolated dir, NOT default `~/.codex`: - `CODEX_HOME=/home/claude/.codex-accounts/0 /home/claude/EJClaw/node_modules/.bin/codex login --device-auth` - then open the printed `https://auth.openai.com/codex/device` URL and enter the one-time code. A plain `codex login` writes to `~/.codex` and will NOT update the account EJClaw uses. Account: ChatGPT `account_id 7e40ecd8-461b-49bd-bbe4-cdc6c7dc68cf`. -- Verify: `cd /home/claude/EJClaw && bun -e 'import {initCodexTokenRotation,getActiveCodexAuthPath} from "./src/codex-token-rotation.ts"; initCodexTokenRotation(); console.log(getActiveCodexAuthPath())'` → should print `/home/claude/.codex-accounts/0/auth.json`. - -## EJClaw / KAclaw model + runtime versions (updated 2026-10-01) - -Current state: -- EJClaw: `CLAUDE_MODEL=claude-opus-5-5` (Claude Opus 5.5), `CLAUDE_EFFORT=high`, `CLAUDE_THINKING=adaptive`. Runner SDK `@anthropic-ai/claude-agent-sdk` 0.3.286 → bundled Claude Code CLI 2.1.286 (pinned in `runners/agent-runner/package.json`). Reviewer: Codex `gpt-5.5` via `@openai/codex` 0.159.3 (pinned in `runners/codex-runner/package.json`). `.env` backup before the 5.5 switch: `/home/claude/ejclaw-env-backup-20261001-140746`. -- KAclaw: `CLAUDE_MODEL=claude-opus-5-5` since 2026-10-01 (account hjj100411, Pro). Its runner is a standalone package (no workspaces): SDK bumped 0.2.114 → 0.3.286 in `runners/agent-runner` (CLI 2.1.114 rejected Opus 5.5). KAclaw is not a git repo; pre-change backup (runner package.json, bun.lock, dist, node_modules, .env, expired credentials) at `/home/claude/kaclaw-backup-20261001-144805/`. -- KAclaw OAuth: its refresh token expired 2026-09-30 22:43 (`invalid_grant: Refresh token expired`; refresh tokens have an absolute lifetime), the access token died 23:38 and KAclaw silently could not answer until re-login. Re-login = `cd /home/claude/KAclaw && CLAUDE_CREDENTIALS_PATH=/home/claude/KAclaw/data/claude/.credentials.json bun setup/index.ts --step login` (prints URL) → account owner logs in → re-run with `--code ` → update `CLAUDE_CODE_OAUTH_TOKEN(S)` in KAclaw `.env` → `systemctl --user restart kaclaw` (safe from an EJClaw turn). -- Testing a model "with a service's account": the Claude Code CLI ignores `CLAUDE_CREDENTIALS_PATH`; without `CLAUDE_CODE_OAUTH_TOKEN` it silently uses `~/.claude` (tkrmagid Max). Pass `CLAUDE_CODE_OAUTH_TOKEN=` explicitly, or the test proves nothing about that account. - -Model upgrade checklist (both incidents below were caused by skipping it): -1. Get the exact model ID from the models API, never by guessing from the marketing name: `curl -s https://api.anthropic.com/v1/models -H "x-api-key: " -H "anthropic-version: 2023-06-01"`. Valid 5.x IDs as of 2026-10-01: `claude-opus-5-5`, `claude-sonnet-5-5`, `claude-opus-5`, `claude-sonnet-5`, `claude-fable-5-1`, `claude-fable-5`. Writing `5.0` as `claude-opus-5-0` is invalid — that typo broke every KAclaw channel 2026-09-17 → 09-21. -2. Before editing `.env`, run a real query through the service's own SDK + bundled binary with that service's access token in `CLAUDE_CODE_OAUTH_TOKEN` (`query({model, pathToClaudeCodeExecutable: /node_modules/@anthropic-ai/claude-agent-sdk-linux-x64/claude})`; `CLAUDE_CREDENTIALS_PATH` alone is ignored by the CLI and silently falls back to `~/.claude`). Newer models can require a newer Claude Code CLI: Opus 5.5 is rejected by CLI 2.1.153 with `400 ... version 2.1.280 or newer is required`, so the SDK had to be bumped first. -3. Opus/Sonnet 5.x support only `adaptive` thinking — keep `CLAUDE_THINKING=adaptive` (`enabled` + budget fails on 5.x). -4. Apply with the detached `systemd-run … systemctl --user restart` form and verify the new PID + zero `does not support this model` / `issue with the selected model` log lines. - -Gotchas found 2026-10-01: -- **Stale nested `node_modules` can shadow the lockfile version.** `runners/codex-runner/node_modules/@openai/codex` held an untracked 0.124.0 copy from 2026-04-26, so the reviewer kept running 0.124 no matter what the pin said (that old CLI failed every run with `failed to decode models response: unknown variant max` and dumped the whole models payload into the journal). It was removed (backup `/home/claude/.tmp/stale-codex-backup/`). After any runner dependency bump, verify what the runner really loads: `cd runners/codex-runner/dist && bun -e "const p=Bun.resolveSync('@openai/codex/package.json', process.cwd()+'/index.js'); console.log(p, require(p).version)"` — it must be the root `node_modules` path. -- **One Codex account = one Codex run at a time.** The per-account auth lease is exclusive (protects refresh-token rotation). Two rooms needing Codex at once used to fail the second run instantly ("Codex rotation pool unavailable") and close the task as `reviewer_codex_unavailable` — the review was silently skipped (3× on 2026-09-28). Since fix 30ba510 the second run waits up to 15 min for the lease; a dead/rate-limited pool still fails fast. Adding a second Codex account under `~/.codex-accounts/1/` would remove the waiting. -- The service runs `bun src/index.ts`, so root `dist/` staleness does not matter; runner `dist/` (spawned by path) does — rebuild with `bun run build:runners` after runner changes. - -EJClaw fixes 2026-09-19 → 10-01 (details in each commit message): -- c7a8a43 — tribunal room with `work_dir = NULL` self-heals instead of freezing its task revision and dropping follow-ups for 10 min. -- 69ae334 — superseded run's 5s progress ticker is stopped when it loses the turn attempt (room no longer looks frozen). -- 8b68885 — claude-agent-sdk 0.3.153 → 0.3.286 for Opus 5.5. -- 30ba510 — wait for a busy Codex lease instead of skipping the review. -- a7f2e02 — @openai/codex 0.133 → 0.159.3 (+ stale 0.124 nested copy removed); simple reviewer turn 49s → 5s, 0 app-server errors. - -## Windows CI 러너 / .9 중첩 가상화 (2026-09-23) — 모든 프로젝트 공용 - -`.9` 안에서 Windows VM 을 돌려 Windows 전용 경로를 자동 검증할 수 있다. -LiveSub 에서 처음 구축했지만 **어느 프로젝트든 그대로 재사용**하면 된다. - -### 중첩 가상화 — 된다 (실측) - -`.9` 는 Proxmox 위의 QEMU/KVM VM 이지만 중첩 가상화가 켜져 있다. -`/dev/kvm` 존재, `kvm_intel nested=Y`, `qemu -enable-kvm` → `kvm support: enabled`. - -- 설치해 둔 것: `qemu-system-x86`, `ovmf`, `swtpm`, `swtpm-tools`, `xorriso` -- **`claude` 계정을 `kvm` 그룹에 넣었다.** `/dev/kvm` 이 `root:kvm 0660` 이라 - 없으면 `Could not access KVM kernel module: Permission denied` 로 죽는다. - usermod 는 재로그인해야 적용되므로 그 전에는 `sg kvm -c '...'` 로 감싼다. -- **중첩 GPU 패스스루는 불가능하다.** IOMMU 그룹 0개, 커널 cmdline 에 iommu - 옵션 없음. GPU 검증은 `.9`(리눅스)에서 직접 해야 한다. - -### 자원 한계 — CPU 는 절대 늘리지 말 것 (실측 근거) - -물리 호스트는 **i7-11700 (8코어/16스레드), RAM 62.6GB** 다. Proxmox(`192.168.10.2`) -에는 SSH 키가 없어 접근 못 하지만, `.5` LXC 를 통해 호스트 스펙을 읽을 수 있다. -`/proc/meminfo` 는 lxcfs 가 컨테이너 한도로 가리므로 -**`/sys/devices/system/node/node0/meminfo`** 를 읽어야 진짜 값이 나온다. - -`.9` 의 vCPU 를 늘리면 `.5` 의 운영 컨테이너 19개가 느려진다. `.5` 응답 실측: - -``` -.9 한가할 때 616~628ms 기준 -.9 가 4코어 사용 657~678ms +7% <- VM 100% 인 정상 상황 -.9 가 8코어 전부 1487~1491ms +140% <- 이 상태를 만들면 안 된다 -``` - -그래서 VM 은 **vCPU 4 / RAM 8GB / 디스크 80GB** 로 띄우고, 최악을 막으려면 -`CPUQuota=400%` + `CPUWeight=50` 을 건다. CI 작업은 코어 4개에서 포화하므로 -더 줘도 빨라지지 않는다 (파이프라인 2코어 13초 / 4코어 10초 / 8코어 11초). - -### ⚠️ VM 은 반드시 `systemd-run --user --scope` 로 띄울 것 - -에이전트 턴에서 그냥 `qemu` 를 실행하면 VM 이 **봇의 cgroup 안에서** 돈다. -`ejclaw.service` 는 `MemoryHigh=16G` 이고 8GB VM 이 그 안에 들어가면 상한을 -쳐서 봇이 OOM 으로 죽는다 (2026-07-26 에 실제로 6회 연속 OOM 전례 있음). - -```bash -systemd-run --user --scope -p MemoryMax=10G -p CPUQuota=400% -p CPUWeight=50 \ - --unit win-ci qemu-system-x86_64 -enable-kvm -m 8192 ... -``` - -`app.slice` 의 형제 scope 로 빠지는 것을 실측 확인했다. - -### ISO 출처 (라이선스 키 불필요) - -- **Windows 11 Enterprise LTSC 2024 평가판** (90일, 키 없이 설치·사용) - `https://go.microsoft.com/fwlink/p/?linkid=2289029` → 4.8GB - 파일명 `26100.1742...CLIENT_LTSC_EVAL_x64FRE_en-us.iso` (빌드 26100 = 24H2) - LTSC 라 Store·소비자 앱이 없어 CI 러너로 오히려 유리하다. - 90일 후에는 `slmgr /rearm` 으로 최대 3회 연장하거나 VM 을 다시 만든다. -- **virtio-win** (virtio 드라이버) - `https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/virtio-win.iso` → 0.9GB -- 다운로드 속도 약 12.5MB/s → 둘 합쳐 약 8분. -- 사용자 지시: **설치 끝나면 ISO 는 지워서 디스크를 회수할 것.** - -### VM 구성 시 빠뜨리면 조용히 실패하는 것들 - -- `-device virtio-vga,max_outputs=2` — 화면 2개. 기본 `-vga std` 는 1개라 - **모니터 선택 기능을 검증할 수 없다.** 게스트에 virtio-gpu 드라이버 필요. -- 오디오 장치를 안 붙이면 WASAPI 장치 열거가 0개로 실패한다. -- **자동 로그온 필수.** 러너를 서비스로 돌리면 데스크톱 세션이 없어 GUI - 테스트와 스크린샷이 전부 실패한다. 자동 로그온 + 로그온 시 예약 작업. -- Windows 11 은 TPM 2.0 + SecureBoot 필요 → `swtpm` + `OVMF_CODE_4M.secboot.fd`. - -### Gitea Actions 러너 - -- 라벨 형식 `[:]`, schema 기본값 `host`. -- **라벨이 안 맞아도 작업이 실패하지 않고 기본 ubuntu 컨테이너에서 돈다.** - Windows 검증이 조용히 리눅스에서 도는 착각이 생기므로 워크플로 첫 단계에서 - `RUNNER_OS` 를 확인해 막을 것. -- `shell: bash` 를 쓰지 말 것. Git for Windows 를 깔아도 `bash.exe` 는 기본 - PATH 에 없다 (`Git\cmd` 만 등록). `pwsh`(PowerShell 7) 를 쓰고 winget 으로 - `Microsoft.PowerShell` 을 설치해야 한다. -- GPU 러너를 여러 프로젝트가 공유하면 **`capacity: 1`** 을 반드시 확인한다. - 워크플로 `concurrency` 는 같은 저장소 안에서만 막아준다. - -### .9 재부팅 시 확인 (2026-09-23 기준) - -`linger=yes` 이고 아래 유저 서비스가 전부 `enabled` 라 재부팅 후 자동 복구된다. -ejclaw, kaclaw, geoglobe-web/tunnel, unifi-site/tunnel, exvideo-tunnel, -wsai-voice, xmas-counter, vncserver, filter-chain. 시스템 쪽은 docker, -containerd, ollama 가 enabled. 재부팅 전에 이 목록을 다시 확인할 것. - -## .5 Docker host + remote docker context (2026-07-21) - -A second LAN machine runs Docker workloads; any Claude channel on .9 can drive it. - -- Identity: `192.168.10.5` — a Proxmox **LXC container** hostnamed `docker`, Debian 13, Docker 29.3.0, root-only shell, **no GPU**. This host (.9, `192.168.10.9`, "claude") is a QEMU/KVM VM with the RTX 5050 passed through — so .9 has the GPU, .5 does not. -- SSH: **passwordless key auth is installed** (`claude@.9` pubkey in `root@.5:~/.ssh/authorized_keys`) → `ssh root@192.168.10.5` needs no password. Fallback root password in `secrets.json` under `docker5.ssh`. -- **Docker context** `docker5` (`ssh://root@192.168.10.5`) exists on .9 and is the **active default context**, so a plain `docker ...`/`docker compose ...` on .9 runs on .5. Use `docker --context default ...` (or `docker context use default`) for .9's own local docker (where the GPU Ollama service for B would run); switch back with `docker context use docker5`. -- .5 runs ~19 production containers — do NOT disturb: projects `bot` (mamil, memil, music1/2/3, random), `site` (make_video_site, minecraft_launcher_site, signature), `virtual-stock-site-bot` (api/web/worker), `mc_domain_proxy` (mc-filter api/frontend/nginx/proxy), `act_runner`, `portainer`, `spotify-tokener`. Compose files under `root@.5:/root/other/*/docker-compose.yml`. -- GPU-from-.5: since .5 has no GPU device, GPU work for .5 containers must go over the network to .9 (e.g. Ollama on .9 `--gpus all` bound to `0.0.0.0:11434`, .5 calls `http://192.168.10.9:11434`). In-container CUDA (e.g. faster-whisper `device=cuda`) will NOT work on .5. - ---- +- All rooms are `tribunal` (owner + reviewer + arbiter) unless the user asks for single mode in that room ("리뷰어 끄자", "single로 바꿔줘"). Commands: `ops/ejclaw-ops.md`. +- "채널 등록 해제 " means a full purge with `bun scripts/deregister-room.ts` (dry-run first, DB backup first). Details: `ops/ejclaw-ops.md`. +- A short stop-only message ("그만해", "멈춰", "중단", "stop") stops the room's running work; this is built in. # CLAUDE.md