claude-bot/EJClaw
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Simplify verification runtime

Browse Source
This commit is contained in:
ejclaw
2026-04-08 05:30:14 +09:00
parent ead7c32870
commit edd5b45d99
20 changed files with 300 additions and 427 deletions
Download Patch File Download Diff File Expand all files Collapse all files

11
.dockerignore
View File

@@ -1,11 +0,0 @@
*
!container/
!container/Dockerfile
!shared/
!shared/verification-snapshot.d.ts
!shared/verification-snapshot.js
!runners/
!runners/agent-runner/
!runners/agent-runner/**
!runners/codex-runner/
!runners/codex-runner/**

5
CLAUDE.md
View File

@@ -38,7 +38,6 @@ Run commands directly—don't tell the user to run them.
bun run build # Build main project
bun run build:runners # Install + build both runners
bun run build:runtime # Build host runtime only
bun run build:container # Optional: rebuild reviewer Docker image
bun run dev # Dev mode with hot reload
```
@@ -54,8 +53,8 @@ Deploy:
bun run deploy
```
`deploy` rebuilds only the host runtime. Rebuild the reviewer Docker image
separately with `bun run build:container` only when you need the container path.
`deploy` rebuilds only the host runtime, and verification also runs directly on
the host runtime.
## Service Stack Architecture

3
README.md
View File

@@ -140,7 +140,6 @@ Discord ──► SQLite (WAL) ──► GroupQueue ──┬──► Owner (ho
- Linux (Ubuntu 22.04+) or macOS
- [Bun](https://bun.sh/) 1.3+
- Docker (currently used for isolated verification runs)
- [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code)
- [Codex CLI](https://github.com/openai/codex) (`npm install -g @openai/codex`)
- Discord bot tokens (3: owner, reviewer, arbiter)
@@ -153,7 +152,6 @@ cd EJClaw
bun install
bun run build:runners
bun run build
bun run build:container # Build verification container image
```
## Documentation
@@ -213,7 +211,6 @@ bun run deploy
```bash
bun run build # Build main project
bun run build:runners # Install + build both runners
bun run build:container # Rebuild verification container image
bun run dev # Dev mode with hot reload
bun test # Run tests
```

92
container/Dockerfile
View File

@@ -1,92 +0,0 @@
# EJClaw Reviewer Container
# Runs Claude/Codex agent in read-only isolated environment for code review.
# Build from project root: docker build -f container/Dockerfile -t ejclaw-reviewer:latest .
FROM node:22-slim
# System dependencies — verification/review tools included
RUN apt-get update --fix-missing && apt-get install -y --no-install-recommends \
chromium \
fonts-liberation \
fonts-noto-cjk \
fonts-noto-color-emoji \
curl \
git \
# Media
ffmpeg \
sox \
mediainfo \
imagemagick \
# Build
make \
gcc \
g++ \
python3 \
python3-pip \
# Utils
jq \
file \
sqlite3 \
rsync \
zip \
unzip \
strace \
ripgrep \
fd-find \
bat \
tree \
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg \
| dd of=/usr/share/keyrings/githubcli-archive-keyring.gpg \
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/githubcli-archive-keyring.gpg] https://cli.github.com/packages stable main" \
> /etc/apt/sources.list.d/github-cli.list \
&& apt-get update && apt-get install -y --no-install-recommends gh \
&& rm -rf /var/lib/apt/lists/*
# Chromium paths for Playwright/agent-browser
ENV AGENT_BROWSER_EXECUTABLE_PATH=/usr/bin/chromium
ENV PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH=/usr/bin/chromium
# Install Claude Code and Codex CLI globally (requires Node.js)
RUN npm install -g @anthropic-ai/claude-code @openai/codex
# Install Bun runtime (EJClaw runner uses bun:sqlite and runs via bun)
ENV BUN_INSTALL="/usr/local"
RUN curl -fsSL https://bun.sh/install | bash
# Create app directory
WORKDIR /app
# Copy shared verification snapshot helper used by agent-runner
COPY shared/verification-snapshot.* ./shared/
# Copy agent-runner (Claude Code)
COPY runners/agent-runner/package*.json ./runners/agent-runner/
RUN bun install --cwd ./runners/agent-runner
COPY runners/agent-runner/ ./runners/agent-runner/
RUN bun run --cwd ./runners/agent-runner build
# Copy codex-runner
COPY runners/codex-runner/package*.json ./runners/codex-runner/
RUN bun install --cwd ./runners/codex-runner
COPY runners/codex-runner/ ./runners/codex-runner/
RUN bun run --cwd ./runners/codex-runner build
# Create workspace directories
RUN mkdir -p /workspace/project /workspace/group /workspace/global \
/workspace/ipc/messages /workspace/ipc/input
# Entrypoint (fallback — persistent containers use docker exec with explicit runner path)
RUN printf '#!/bin/bash\nset -e\ncat > /tmp/input.json\nbun /app/runners/agent-runner/dist/index.js < /tmp/input.json\n' \
> /app/entrypoint.sh && chmod +x /app/entrypoint.sh
# Set ownership to node user (non-root)
RUN chown -R node:node /workspace /home/node
# Switch to non-root user
USER node
# Working directory is the read-only project mount
WORKDIR /workspace/project
# Entry point reads JSON from stdin, outputs JSON to stdout
ENTRYPOINT ["/app/entrypoint.sh"]

11
container/build.sh
View File

@@ -1,11 +0,0 @@
#!/usr/bin/env bash
# Build the EJClaw reviewer container image
# Runs from project root so Dockerfile can COPY runners/agent-runner/
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
echo "Building ejclaw-reviewer container image..."
docker build -f "$SCRIPT_DIR/Dockerfile" -t ejclaw-reviewer:latest "$PROJECT_ROOT"
echo "Done. Image: ejclaw-reviewer:latest"

14
docs/architecture.md
View File

@@ -80,12 +80,13 @@ User message
## Verification Isolation
Reviewer and arbiter now run as host processes with role-scoped read-only
guards and sandbox settings. Docker remains in use for verification profiles:
guards and sandbox settings. Verification profiles run directly on the host with
restricted environment variables and snapshot checks:
- Scratch workspace copied before execution
- Verification command run inside the container image
- Isolated filesystem view for test/typecheck/build checks
- Runtime image shared with host tooling and evidence inspection
- Fixed-profile direct execution (`test`, `typecheck`, `build`)
- Snapshot comparison before/after execution
- Root-level runtime/build directories excluded from verification snapshot
- Role-scoped reviewer safeguards remain independent of verification execution
## Key Files
@@ -93,8 +94,7 @@ guards and sandbox settings. Docker remains in use for verification profiles:
|------|---------|
| `src/index.ts` | Orchestrator: state, message loop, agent invocation |
| `src/agent-runner.ts` | Spawns agent processes, manages env/sessions/skills |
| `src/verification.ts` | Verification execution using the container image |
| `src/container-runtime.ts` | Shared Docker runtime helpers for verification |
| `src/verification.ts` | Fixed-profile verification execution with snapshot checks |
| `src/channels/discord.ts` | Discord channel (8s typing refresh, Whisper transcription) |
| `src/ipc.ts` | IPC watcher and task processing |
| `src/router.ts` | Message formatting and outbound routing |

2
package.json
View File

@@ -9,8 +9,6 @@
"build": "tsc",
"build:runners": "bun install --cwd runners/agent-runner && bun run --cwd runners/agent-runner build && bun install --cwd runners/codex-runner && bun run --cwd runners/codex-runner build",
"build:runtime": "bun run build && bun run build:runners",
"build:container": "docker build -f container/Dockerfile -t ejclaw-reviewer:latest .",
"build:all": "bun run build:runtime && bun run build:container",
"deploy": "git pull --ff-only && bun run build:runtime && systemctl --user restart ejclaw",
"start": "bun dist/index.js",
"dev": "bun --watch src/index.ts",

1
runners/agent-runner/src/host-evidence.ts
View File

@@ -4,7 +4,6 @@ import path from 'path';
export const HOST_EVIDENCE_ACTIONS = [
'ejclaw_service_status',
'ejclaw_service_logs',
'reviewer_image_inspect',
] as const;
export type HostEvidenceAction = (typeof HOST_EVIDENCE_ACTIONS)[number];

4
runners/agent-runner/src/ipc-mcp-stdio.ts
View File

@@ -357,12 +357,12 @@ server.tool(
server.tool(
'read_host_evidence',
'Read host-side deployment evidence through a narrow allowlist. Use this instead of broad shell access when reviewer/arbiter needs service status, recent logs, or reviewer image metadata.',
'Read host-side deployment evidence through a narrow allowlist. Use this instead of broad shell access when reviewer/arbiter needs service status or recent logs.',
{
action: z
.enum(HOST_EVIDENCE_ACTIONS)
.describe(
'ejclaw_service_status=systemctl --user show ejclaw, ejclaw_service_logs=recent journalctl lines, reviewer_image_inspect=docker image inspect for the reviewer image',
'ejclaw_service_status=systemctl --user show ejclaw, ejclaw_service_logs=recent journalctl lines',
),
tail_lines: z
.number()

6
runners/agent-runner/test/verification.test.ts
View File

@@ -45,7 +45,7 @@ describe('runner verification helpers', () => {
stderr: '',
exitCode: 0,
snapshotId: 'fs:abc123',
runtimeVersion: 'ejclaw-reviewer:latest@sha256:test',
runtimeVersion: 'host:bun@test',
}),
);
@@ -69,13 +69,13 @@ describe('runner verification helpers', () => {
stderr: 'build failed\n',
exitCode: 1,
snapshotId: 'fs:def456',
runtimeVersion: 'ejclaw-reviewer:latest@sha256:test',
runtimeVersion: 'host:bun@test',
error: 'command failed',
});
expect(text).toContain('Verification profile: build');
expect(text).toContain('Snapshot: fs:def456');
expect(text).toContain('Runtime: ejclaw-reviewer:latest@sha256:test');
expect(text).toContain('Runtime: host:bun@test');
expect(text).toContain('Exit code: 1');
expect(text).toContain('$ npm run build');
expect(text).toContain('[stderr]');

3
setup/environment.test.ts
View File

@@ -99,8 +99,7 @@ describe('credentials detection', () => {
describe('platform command detection', () => {
it('commandExists returns boolean', async () => {
const { commandExists } = await import('./platform.js');
expect(typeof commandExists('docker')).toBe('boolean');
expect(typeof commandExists('git')).toBe('boolean');
expect(typeof commandExists('nonexistent_binary_xyz')).toBe('boolean');
});
});

1
shared/verification-snapshot.js
View File

@@ -9,6 +9,7 @@ export const VERIFICATION_SNAPSHOT_EXCLUDE_NAMES = new Set([
const VERIFICATION_SNAPSHOT_ROOT_EXCLUDE_NAMES = new Set([
'.git',
'.env',
'dist',
'data',
'logs',
'cache',

3
src/config.ts
View File

@@ -207,9 +207,6 @@ const rawMaxRoundTrips = getEnv('PAIRED_MAX_ROUND_TRIPS') || '1000';
export const PAIRED_MAX_ROUND_TRIPS =
rawMaxRoundTrips === '0' ? Infinity : parseInt(rawMaxRoundTrips, 10) || 1000;
// ── Container isolation ───────────────────────────────────────────
export const REVIEWER_CONTAINER_IMAGE =
getEnv('REVIEWER_CONTAINER_IMAGE') || 'ejclaw-reviewer:latest';
export const RECOVERY_STAGGER_MS = parseInt(
getEnv('RECOVERY_STAGGER_MS') || '2000',
10,

111
src/container-runtime.ts
View File

@@ -1,111 +0,0 @@
/**
* Container runtime abstraction for EJClaw verification.
* Verification profiles and related host evidence still use Docker.
* Runtime-specific logic lives here so swapping runtimes means changing one file.
*/
import { execSync } from 'child_process';
import fs from 'fs';
import os from 'os';
import { logger } from './logger.js';
export const CONTAINER_RUNTIME_BIN = 'docker';
export const CONTAINER_HOST_GATEWAY = 'host.docker.internal';
/**
* Address the credential proxy binds to.
* Docker Desktop (macOS): 127.0.0.1
* Docker (Linux): docker0 bridge IP (containers can reach it via host.docker.internal)
*/
export const PROXY_BIND_HOST =
process.env.CREDENTIAL_PROXY_HOST || detectProxyBindHost();
function detectProxyBindHost(): string {
if (os.platform() === 'darwin') return '127.0.0.1';
if (fs.existsSync('/proc/sys/fs/binfmt_misc/WSLInterop')) return '127.0.0.1';
const ifaces = os.networkInterfaces();
const docker0 = ifaces['docker0'];
if (docker0) {
const ipv4 = docker0.find((a) => a.family === 'IPv4');
if (ipv4) return ipv4.address;
}
return '0.0.0.0';
}
export function hostGatewayArgs(): string[] {
if (os.platform() === 'linux') {
return ['--add-host=host.docker.internal:host-gateway'];
}
return [];
}
export function readonlyMountArgs(
hostPath: string,
containerPath: string,
): string[] {
return ['-v', `${hostPath}:${containerPath}:ro`];
}
export function writableMountArgs(
hostPath: string,
containerPath: string,
): string[] {
return ['-v', `${hostPath}:${containerPath}`];
}
export function tmpfsMountArgs(
containerPath: string,
options?: string | string[],
): string[] {
if (!options || (Array.isArray(options) && options.length === 0)) {
return ['--tmpfs', containerPath];
}
const optionText = Array.isArray(options) ? options.join(',') : options;
return ['--tmpfs', `${containerPath}:${optionText}`];
}
export function stopContainer(name: string): string {
return `${CONTAINER_RUNTIME_BIN} stop ${name}`;
}
export function ensureContainerRuntimeRunning(): void {
try {
execSync(`${CONTAINER_RUNTIME_BIN} info`, {
stdio: 'pipe',
timeout: 10000,
});
logger.debug('Container runtime already running');
} catch (err) {
logger.error({ err }, 'Failed to reach container runtime');
throw new Error(
'Container runtime (Docker) is required for reviewer isolation but failed to start. Run: docker info',
);
}
}
export function cleanupOrphans(): void {
try {
const output = execSync(
`${CONTAINER_RUNTIME_BIN} ps --filter name=ejclaw-reviewer- --format '{{.Names}}'`,
{ stdio: ['pipe', 'pipe', 'pipe'], encoding: 'utf-8' },
);
const orphans = output.trim().split('\n').filter(Boolean);
for (const name of orphans) {
try {
execSync(stopContainer(name), { stdio: 'pipe' });
} catch {
/* already stopped */
}
}
if (orphans.length > 0) {
logger.info(
{ count: orphans.length, names: orphans },
'Stopped orphaned reviewer containers',
);
}
} catch (err) {
logger.warn({ err }, 'Failed to clean up orphaned containers');
}
}

12
src/host-evidence.test.ts
View File

@@ -10,7 +10,6 @@ describe('host evidence helpers', () => {
it('recognizes only allowlisted actions', () => {
expect(isHostEvidenceAction('ejclaw_service_status')).toBe(true);
expect(isHostEvidenceAction('ejclaw_service_logs')).toBe(true);
expect(isHostEvidenceAction('reviewer_image_inspect')).toBe(true);
expect(isHostEvidenceAction('rm -rf /')).toBe(false);
});
@@ -39,16 +38,5 @@ describe('host evidence helpers', () => {
expect(logs.args).toEqual(
expect.arrayContaining(['--user', '-u', 'ejclaw', '-n', '42']),
);
const inspect = buildHostEvidenceCommand({
requestId: 'req-3',
action: 'reviewer_image_inspect',
});
expect(inspect.file).toBe('docker');
expect(inspect.args.slice(0, 3)).toEqual([
'image',
'inspect',
'ejclaw-reviewer:latest',
]);
});
});

18
src/host-evidence.ts
View File

@@ -2,13 +2,11 @@ import { execFile } from 'child_process';
import fs from 'fs';
import path from 'path';
import { REVIEWER_CONTAINER_IMAGE } from './config.js';
import { resolveGroupIpcPath } from './group-folder.js';
export const HOST_EVIDENCE_ACTIONS = [
'ejclaw_service_status',
'ejclaw_service_logs',
'reviewer_image_inspect',
] as const;
export type HostEvidenceAction = (typeof HOST_EVIDENCE_ACTIONS)[number];
@@ -110,22 +108,6 @@ export function buildHostEvidenceCommand(
commandText: `journalctl ${args.join(' ')}`,
};
}
case 'reviewer_image_inspect': {
const args = [
'image',
'inspect',
REVIEWER_CONTAINER_IMAGE,
'--format',
'Id={{.Id}}\nRepoTags={{json .RepoTags}}\nCreated={{.Created}}\nSize={{.Size}}',
];
return {
file: 'docker',
args,
commandText: `docker ${args
.map((part) => (part.includes(' ') ? JSON.stringify(part) : part))
.join(' ')}`,
};
}
}
}

2
src/paired-workspace-manager.ts
View File

@@ -778,7 +778,7 @@ export function markPairedTaskReviewReady(taskId: string): {
);
}
// Reviewer runs in a container with the owner workspace mounted read-only.
// Reviewer uses the owner workspace directly in read-only mode.
// No file-level snapshot copy needed — just register the path.
const reviewerWorkspace = makeWorkspaceRecord({
taskId,

2
src/verification-ipc.test.ts
View File

@@ -62,7 +62,7 @@ describe('verification IPC', () => {
stderr: '',
exitCode: 0,
snapshotId: 'fs:abc123',
runtimeVersion: 'ejclaw-reviewer:latest@sha256:test',
runtimeVersion: 'host:bun@test',
});
await processTaskIpc(

178
src/verification.test.ts
View File

@@ -5,7 +5,6 @@ import path from 'path';
import { describe, expect, it } from 'vitest';
import {
buildDockerRunArgs,
buildVerificationCommand,
computeVerificationSnapshot,
isVerificationProfile,
@@ -72,10 +71,13 @@ describe('verification helpers', () => {
path.join(repoDir, 'src', 'index.ts'),
'export const x = 1;\n',
);
fs.mkdirSync(path.join(repoDir, 'dist'), { recursive: true });
fs.writeFileSync(path.join(repoDir, '.env'), 'SECRET=1\n');
fs.writeFileSync(path.join(repoDir, 'dist', 'index.js'), 'export const x = 1;\n');
const first = computeVerificationSnapshot(repoDir).snapshotId;
fs.writeFileSync(path.join(repoDir, '.env'), 'SECRET=2\n');
fs.writeFileSync(path.join(repoDir, 'dist', 'index.js'), 'export const x = 2;\n');
const second = computeVerificationSnapshot(repoDir).snapshotId;
fs.writeFileSync(
path.join(repoDir, 'src', 'index.ts'),
@@ -195,9 +197,53 @@ describe('verification helpers', () => {
expect(result.error).toContain('Snapshot mismatch before verification');
});
it('runs verification commands via docker entrypoint override', () => {
it('runs typecheck directly on the host for non-build profiles', async () => {
const repoDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-docker-'),
path.join(os.tmpdir(), 'ejclaw-verification-direct-'),
);
fs.mkdirSync(path.join(repoDir, 'node_modules', '.bin'), {
recursive: true,
});
fs.mkdirSync(path.join(repoDir, 'src'), { recursive: true });
fs.writeFileSync(
path.join(repoDir, 'package.json'),
JSON.stringify({
name: 'verification-direct',
packageManager: 'bun@1.3.11',
scripts: {
typecheck:
'node -e "process.stdout.write(\'typecheck:\' + (process.env.CI || \'missing\'))"',
},
}),
);
fs.writeFileSync(path.join(repoDir, 'bun.lock'), '');
fs.writeFileSync(path.join(repoDir, 'node_modules', '.bin', 'placeholder'), '');
fs.writeFileSync(
path.join(repoDir, 'src', 'index.ts'),
'export const value = 1;\n',
);
ensureWorkspaceDependenciesInstalled(repoDir);
const expectedSnapshotId = computeVerificationSnapshot(repoDir).snapshotId;
const result = await runVerificationRequest(
{
requestId: 'req-direct-typecheck',
profile: 'typecheck',
expectedSnapshotId,
},
{ repoDir },
);
expect(result.ok).toBe(true);
expect(result.exitCode).toBe(0);
expect(result.stdout).toContain('typecheck:1');
expect(result.runtimeVersion).toMatch(/^host:bun@/);
expect(result.snapshotId).toBe(expectedSnapshotId);
});
it('does not leak host secrets into direct verification commands', async () => {
const repoDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-direct-secret-'),
);
fs.mkdirSync(path.join(repoDir, 'node_modules', '.bin'), {
recursive: true,
@@ -205,27 +251,129 @@ describe('verification helpers', () => {
fs.writeFileSync(
path.join(repoDir, 'package.json'),
JSON.stringify({
name: 'verification-docker',
name: 'verification-direct-secret',
packageManager: 'bun@1.3.11',
scripts: {
typecheck: 'tsc --noEmit',
typecheck:
'node -e "process.stdout.write(process.env.EJCLAW_VERIFICATION_SECRET || \'missing\')"',
},
}),
);
fs.writeFileSync(path.join(repoDir, 'bun.lock'), '');
fs.writeFileSync(path.join(repoDir, 'node_modules', '.bin', 'tsc'), '');
fs.writeFileSync(path.join(repoDir, 'node_modules', '.bin', 'placeholder'), '');
ensureWorkspaceDependenciesInstalled(repoDir);
const command = buildVerificationCommand('typecheck', repoDir);
const args = buildDockerRunArgs('/tmp/verify-workspace', repoDir, command);
const imageIndex = args.findIndex((value) => value === 'ejclaw-reviewer:latest');
const previousSecret = process.env.EJCLAW_VERIFICATION_SECRET;
process.env.EJCLAW_VERIFICATION_SECRET = 'top-secret';
expect(args).toContain('--entrypoint');
expect(args[args.indexOf('--entrypoint') + 1]).toBe(command.file);
expect(imageIndex).toBeGreaterThan(args.indexOf('--entrypoint'));
expect(args.slice(imageIndex + 1)).toEqual(command.args);
expect(args).toContain(
'/workspace/project/node_modules/.vite-temp:uid=1000,gid=1000,mode=1777',
try {
const expectedSnapshotId = computeVerificationSnapshot(repoDir).snapshotId;
const result = await runVerificationRequest(
{
requestId: 'req-direct-typecheck-secret',
profile: 'typecheck',
expectedSnapshotId,
},
{ repoDir },
);
expect(result.ok).toBe(true);
expect(result.stdout).toContain('missing');
expect(result.stdout).not.toContain('top-secret');
} finally {
if (previousSecret == null) {
delete process.env.EJCLAW_VERIFICATION_SECRET;
} else {
process.env.EJCLAW_VERIFICATION_SECRET = previousSecret;
}
}
});
it('fails direct verification if the command mutates the workspace', async () => {
const repoDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-direct-mutate-'),
);
fs.mkdirSync(path.join(repoDir, 'node_modules', '.bin'), {
recursive: true,
});
fs.mkdirSync(path.join(repoDir, 'src'), { recursive: true });
fs.writeFileSync(
path.join(repoDir, 'package.json'),
JSON.stringify({
name: 'verification-direct-mutate',
packageManager: 'bun@1.3.11',
scripts: {
test: 'node -e "require(\'node:fs\').writeFileSync(\'src/generated.txt\', \'x\\\\n\')"',
},
}),
);
fs.writeFileSync(path.join(repoDir, 'bun.lock'), '');
fs.writeFileSync(path.join(repoDir, 'node_modules', '.bin', 'placeholder'), '');
fs.writeFileSync(
path.join(repoDir, 'src', 'index.ts'),
'export const value = 1;\n',
);
ensureWorkspaceDependenciesInstalled(repoDir);
const expectedSnapshotId = computeVerificationSnapshot(repoDir).snapshotId;
const result = await runVerificationRequest(
{
requestId: 'req-direct-test-mutate',
profile: 'test',
expectedSnapshotId,
},
{ repoDir },
);
expect(result.ok).toBe(false);
expect(result.exitCode).toBe(1);
expect(result.error).toContain('Workspace changed during verification');
expect(result.snapshotId).not.toBe(expectedSnapshotId);
expect(fs.existsSync(path.join(repoDir, 'src', 'generated.txt'))).toBe(true);
});
it('allows build verification to write excluded output directories', async () => {
const repoDir = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-build-direct-'),
);
fs.mkdirSync(path.join(repoDir, 'node_modules', '.bin'), {
recursive: true,
});
fs.mkdirSync(path.join(repoDir, 'src'), { recursive: true });
fs.writeFileSync(
path.join(repoDir, 'package.json'),
JSON.stringify({
name: 'verification-build-direct',
packageManager: 'bun@1.3.11',
scripts: {
build: 'node -e "require(\'node:fs\').mkdirSync(\'dist\', { recursive: true }); require(\'node:fs\').writeFileSync(\'dist/output.js\', \'ok\\\\n\')"',
},
}),
);
fs.writeFileSync(path.join(repoDir, 'bun.lock'), '');
fs.writeFileSync(path.join(repoDir, 'node_modules', '.bin', 'placeholder'), '');
fs.writeFileSync(
path.join(repoDir, 'src', 'index.ts'),
'export const value = 1;\n',
);
ensureWorkspaceDependenciesInstalled(repoDir);
const expectedSnapshotId = computeVerificationSnapshot(repoDir).snapshotId;
const result = await runVerificationRequest(
{
requestId: 'req-direct-build',
profile: 'build',
expectedSnapshotId,
},
{ repoDir },
);
expect(result.ok).toBe(true);
expect(result.exitCode).toBe(0);
expect(result.runtimeVersion).toMatch(/^host:bun@/);
expect(result.snapshotId).toBe(expectedSnapshotId);
expect(fs.readFileSync(path.join(repoDir, 'dist', 'output.js'), 'utf-8')).toBe(
'ok\n',
);
});
});

220
src/verification.ts
View File

@@ -3,24 +3,15 @@ import fs from 'fs';
import os from 'os';
import path from 'path';
import { REVIEWER_CONTAINER_IMAGE, TIMEZONE } from './config.js';
import {
CONTAINER_RUNTIME_BIN,
hostGatewayArgs,
readonlyMountArgs,
tmpfsMountArgs,
writableMountArgs,
} from './container-runtime.js';
import { TIMEZONE } from './config.js';
import { resolveGroupIpcPath } from './group-folder.js';
import {
buildWorkspaceScriptCommand,
detectPnpmStorePath,
ensureWorkspaceDependenciesInstalled,
hasInstalledNodeModules,
} from './workspace-package-manager.js';
import {
computeVerificationSnapshotId,
isVerificationSnapshotExcludedPath,
resolveVerificationResponsesDir,
} from '../shared/verification-snapshot.js';
@@ -61,11 +52,39 @@ interface VerificationCommandSpec {
commandText: string;
requiredScript: string;
}
type DirectExecutionEnvKey =
| 'PATH'
| 'HOME'
| 'USER'
| 'LOGNAME'
| 'SHELL'
| 'LANG'
| 'LC_ALL'
| 'LC_CTYPE'
| 'NODE_PATH'
| 'NODE_OPTIONS'
| 'TERM'
| 'COLORTERM'
| 'FORCE_COLOR';
const PRIMARY_PROJECT_MOUNT = '/workspace/project';
const MAX_OUTPUT_CHARS = 24_000;
const COMMAND_TIMEOUT_MS = 20 * 60 * 1000;
const COMMAND_MAX_BUFFER = 20 * 1024 * 1024;
const DIRECT_EXECUTION_ENV_KEYS: readonly DirectExecutionEnvKey[] = [
'PATH',
'HOME',
'USER',
'LOGNAME',
'SHELL',
'LANG',
'LC_ALL',
'LC_CTYPE',
'NODE_PATH',
'NODE_OPTIONS',
'TERM',
'COLORTERM',
'FORCE_COLOR',
];
export function isVerificationProfile(
value: unknown,
@@ -95,10 +114,6 @@ export function buildVerificationCommand(
};
}
function shouldExcludePath(repoDir: string, source: string): boolean {
return isVerificationSnapshotExcludedPath(repoDir, source);
}
export function computeVerificationSnapshot(
repoDir: string,
): VerificationSnapshot {
@@ -125,102 +140,73 @@ function readPackageScripts(repoDir: string): Record<string, string> {
return packageJson.scripts || {};
}
function copyWorkspaceToScratch(repoDir: string, scratchDir: string): void {
fs.cpSync(repoDir, scratchDir, {
recursive: true,
filter: (source) => {
if (source === repoDir) return true;
return !shouldExcludePath(repoDir, source);
},
});
}
function detectRuntimeVersion(): string {
function detectDirectRuntimeVersion(
command: Pick<VerificationCommandSpec, 'file'>,
): string {
try {
const imageId = execFileSync(
CONTAINER_RUNTIME_BIN,
['image', 'inspect', REVIEWER_CONTAINER_IMAGE, '--format', '{{.Id}}'],
{
switch (command.file) {
case 'bun': {
const bunVersion = execFileSync('bun', ['--version'], {
encoding: 'utf-8',
stdio: ['ignore', 'pipe', 'pipe'],
timeout: 5000,
},
).trim();
return `${REVIEWER_CONTAINER_IMAGE}@${imageId}`;
}).trim();
return `host:bun@${bunVersion}`;
}
case 'node':
return `host:node@${process.version}`;
default:
return `host:${command.file}`;
}
} catch {
return REVIEWER_CONTAINER_IMAGE;
return `host:${command.file}`;
}
}
export function buildDockerRunArgs(
scratchWorkspace: string,
sourceRepoDir: string,
command: Pick<VerificationCommandSpec, 'file' | 'args'>,
): string[] {
const args = [
'run',
'--rm',
'-i',
'--workdir',
PRIMARY_PROJECT_MOUNT,
'--entrypoint',
command.file,
'-e',
`TZ=${TIMEZONE}`,
'-e',
'CI=1',
'-e',
'VITEST_CACHE_DIR=/tmp/.vitest',
'-e',
'JEST_CACHE_DIR=/tmp/.jest',
'-e',
'npm_config_cache=/tmp/.npm',
];
args.push(...hostGatewayArgs());
const hostUid = process.getuid?.();
const hostGid = process.getgid?.();
if (hostUid != null && hostUid !== 0 && hostUid !== 1000) {
args.push('--user', `${hostUid}:${hostGid}`);
args.push('-e', 'HOME=/home/node');
}
args.push(...writableMountArgs(scratchWorkspace, PRIMARY_PROJECT_MOUNT));
const sourceNodeModulesDir = path.join(sourceRepoDir, 'node_modules');
if (
hasInstalledNodeModules(sourceRepoDir) &&
fs.existsSync(sourceNodeModulesDir)
) {
args.push(
...readonlyMountArgs(
sourceNodeModulesDir,
path.join(PRIMARY_PROJECT_MOUNT, 'node_modules'),
),
);
args.push(
...tmpfsMountArgs(
path.join(PRIMARY_PROJECT_MOUNT, 'node_modules', '.vite-temp'),
['uid=1000', 'gid=1000', 'mode=1777'],
),
function buildDirectExecutionEnvironment(): {
env: NodeJS.ProcessEnv;
tempRoot: string;
} {
const tempRoot = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-runtime-'),
);
const env: NodeJS.ProcessEnv = {
TZ: TIMEZONE,
CI: '1',
TMPDIR: tempRoot,
TMP: tempRoot,
TEMP: tempRoot,
VITEST_CACHE_DIR: path.join(tempRoot, '.vitest'),
JEST_CACHE_DIR: path.join(tempRoot, '.jest'),
npm_config_cache: path.join(tempRoot, '.npm'),
npm_config_userconfig: path.join(tempRoot, '.npmrc'),
BUN_INSTALL_CACHE_DIR: path.join(tempRoot, '.bun'),
XDG_CACHE_HOME: path.join(tempRoot, '.cache'),
COREPACK_HOME: path.join(tempRoot, '.corepack'),
PNPM_HOME: path.join(tempRoot, '.pnpm-home'),
YARN_CACHE_FOLDER: path.join(tempRoot, '.yarn-cache'),
};
for (const key of DIRECT_EXECUTION_ENV_KEYS) {
const value = process.env[key];
if (value) {
env[key] = value;
}
}
const pnpmStore = detectPnpmStorePath(sourceRepoDir);
if (pnpmStore) {
args.push(...readonlyMountArgs(pnpmStore, pnpmStore));
}
args.push(...tmpfsMountArgs('/tmp'));
args.push(REVIEWER_CONTAINER_IMAGE, ...command.args);
return args;
return {
tempRoot,
env,
};
}
function execFileCapture(
file: string,
args: string[],
options?: {
cwd?: string;
env?: NodeJS.ProcessEnv;
},
): Promise<{ stdout: string; stderr: string }> {
return new Promise((resolve, reject) => {
execFile(
@@ -230,6 +216,8 @@ function execFileCapture(
encoding: 'utf8',
timeout: COMMAND_TIMEOUT_MS,
maxBuffer: COMMAND_MAX_BUFFER,
cwd: options?.cwd,
env: options?.env,
},
(error, stdout, stderr) => {
if (error) {
@@ -264,8 +252,8 @@ export async function runVerificationRequest(
},
): Promise<VerificationResult> {
const repoDir = options?.repoDir || process.cwd();
const runtimeVersion = detectRuntimeVersion();
const command = buildVerificationCommand(request.profile, repoDir);
const runtimeVersion = detectDirectRuntimeVersion(command);
const scripts = readPackageScripts(repoDir);
if (!scripts[command.requiredScript]) {
@@ -333,36 +321,29 @@ export async function runVerificationRequest(
};
}
const scratchRoot = fs.mkdtempSync(
path.join(os.tmpdir(), 'ejclaw-verification-'),
);
const scratchWorkspace = path.join(scratchRoot, 'workspace');
const directExecution = buildDirectExecutionEnvironment();
try {
copyWorkspaceToScratch(repoDir, scratchWorkspace);
try {
const { stdout, stderr } = await execFileCapture(command.file, command.args, {
cwd: repoDir,
env: directExecution.env,
});
const afterSnapshot = computeVerificationSnapshot(repoDir);
if (afterSnapshot.snapshotId !== beforeSnapshot.snapshotId) {
return {
ok: false,
profile: request.profile,
command: command.commandText,
stdout: '',
stderr: '',
stdout: truncateOutput(stdout),
stderr: truncateOutput(stderr),
exitCode: 1,
snapshotId: afterSnapshot.snapshotId,
runtimeVersion,
error: `Workspace changed while preparing verification scratch. expected=${beforeSnapshot.snapshotId} current=${afterSnapshot.snapshotId}`,
error: `Workspace changed during verification. expected=${beforeSnapshot.snapshotId} current=${afterSnapshot.snapshotId}`,
};
}
const dockerArgs = buildDockerRunArgs(scratchWorkspace, repoDir, command);
try {
const { stdout, stderr } = await execFileCapture(
CONTAINER_RUNTIME_BIN,
dockerArgs,
);
return {
ok: true,
profile: request.profile,
@@ -382,6 +363,9 @@ export async function runVerificationRequest(
typeof error === 'object' && error !== null && 'stderr' in error
? String((error as { stderr?: unknown }).stderr ?? '')
: '';
const afterSnapshot = computeVerificationSnapshot(repoDir);
const workspaceChanged =
afterSnapshot.snapshotId !== beforeSnapshot.snapshotId;
return {
ok: false,
@@ -390,13 +374,19 @@ export async function runVerificationRequest(
stdout: truncateOutput(stdout),
stderr: truncateOutput(stderr),
exitCode: extractExitCode(error),
snapshotId: beforeSnapshot.snapshotId,
snapshotId: workspaceChanged
? afterSnapshot.snapshotId
: beforeSnapshot.snapshotId,
runtimeVersion,
error: error instanceof Error ? error.message : String(error),
error: workspaceChanged
? `Workspace changed during verification. expected=${beforeSnapshot.snapshotId} current=${afterSnapshot.snapshotId}`
: error instanceof Error
? error.message
: String(error),
};
}
} finally {
fs.rmSync(scratchRoot, { recursive: true, force: true });
fs.rmSync(directExecution.tempRoot, { recursive: true, force: true });
}
}