feat: robust controlBrowser — real xdotool input, tabs/back/forward/popups

- Install xdotool + wmctrl (late Docker layer, preserves the melo cache) so
  the on-screen Chrome gets real X input (visible cursor, char-by-char typing)
  instead of synthetic events; falls back to the Playwright API if absent.
- Fix active-tab detection (probe document.visibilityState instead of assuming
  tab 0) so sequential ops target the right tab.
- Add back / forward / refresh; new/switch/close tabs via real keyboard
  (Ctrl+T / Ctrl+<n> / Ctrl+W) when xdotool is present.
- Auto-dismiss native JS dialogs; closePopups clears blank/popup tabs.
- Report broadcast (Go-Live) state in status from the turn context.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-14 02:33:59 +09:00
parent b18217fcdd
commit 109dbc7e16
3 changed files with 140 additions and 94 deletions

View File

@@ -65,6 +65,14 @@ RUN ls -d /opt/venv/lib/python*/site-packages/nvidia/cublas/lib \
COPY docker/setup-melo.sh /app/docker/setup-melo.sh
RUN bash /app/docker/setup-melo.sh
# --- Human input + window management for the on-screen Chrome control tool.
# Placed AFTER the heavy melo layer so it doesn't bust that cache. xdotool
# injects real X pointer/keyboard events (visible cursor, char-by-char
# typing) into the broadcast; wmctrl lists/moves windows. ---
RUN apt-get update && apt-get install -y --no-install-recommends \
xdotool wmctrl \
&& rm -rf /var/lib/apt/lists/*
# --- Discord bot deps (cache layer on lockfile) ---
COPY bot/package.json bot/bun.lock /app/bot/
RUN cd /app/bot && bun install --frozen-lockfile || bun install