feat(controlBrowser): add moveMouse (hover) action for the visible cursor

The tool had no cursor-move/hover action, so "move the mouse to the search box"
had nothing to call and a weak model just claimed it had moved it. Add a
moveMouse action wired to human.humanHover (real xdotool cursor), targetable by
CSS selector or site= (that site's search box). Also clarify in the tool
description that 'navigate' types into the address bar (no mouse) while
'search'/'type' move the real cursor to the on-page box and click before
typing, so the model picks the visible-cursor path when the user wants it.
This commit is contained in:
javis-bot
2026-06-24 19:14:04 +09:00
parent 83999a5b0b
commit 5629da7e9f
3 changed files with 83 additions and 13 deletions

View File

@@ -0,0 +1,37 @@
"""Tests for the controlBrowser tool's action surface.
These are deterministic schema/summary checks — they do not drive a real
browser. The actual cursor movement is exercised live on the browser host
(xdotool + CDP), which these tests cannot reach.
"""
import pytest
from jarvis.tools.builtin.control_browser import ControlBrowserTool
@pytest.fixture
def tool():
return ControlBrowserTool()
def test_movemouse_is_an_exposed_action(tool):
# A weak model confabulated "moved the mouse" because no move/hover action
# existed to call. The cursor-move capability must be a real action so the
# request "move the mouse to the search box" maps to a tool call.
enum = tool.inputSchema["properties"]["action"]["enum"]
assert "moveMouse" in enum
def test_movemouse_summary_reports_the_target(tool):
summary = tool._summarise("moveMouse", {"site": "naver"}, {"ok": True, "target": "naver"})
assert "마우스" in summary and "naver" in summary
def test_description_distinguishes_cursor_paths(tool):
# The model must know navigate is address-bar only (no mouse) while
# search/type/moveMouse move the real cursor — that distinction is the
# whole point of the fix.
desc = tool.description
assert "moveMouse" in desc
assert "address bar" in desc # navigate is described as address-bar typing