feat: controlBrowser 'search' action + one-sentence voice replies

- Add a one-shot `search` action (site=naver/google/daum/youtube/bing) that
  navigates the on-screen browser straight to the results page, so a small
  model can satisfy "search X on Naver" in a single tool call instead of a
  fragile navigate->type->enter chain.
- Sharpen the tool description to steer the router to controlBrowser (not
  webSearch) for anything that should happen IN the visible browser.
- System prompt: answer in one short sentence (voice assistant) — also cuts
  TTS time.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-06-14 02:52:47 +09:00
parent 109dbc7e16
commit c21c5b225f
3 changed files with 44 additions and 8 deletions

View File

@@ -41,12 +41,14 @@ class ControlBrowserTool(Tool):
@property
def description(self) -> str:
return (
"Operate the on-screen Chrome shown on the broadcast like a person: open a "
"website/URL, list/open/close/switch tabs, close popups, click an element, "
"type text, scroll, or screenshot. Use whenever the user asks to go to a site "
"(e.g. 'open Naver'), check what's on screen, or interact with the page. "
"Only available in screen-share mode. Do NOT claim you did any of this unless "
"this tool returns success."
"Drive the on-screen Chrome that is shown on the broadcast, like a person would. "
"Use this (NOT webSearch) whenever the user wants something done or shown IN the "
"browser on screen: open a website or URL, search on a specific site (action "
"'search' with site=naver/google/daum/youtube/bing), go back/forward, refresh, "
"manage tabs (list/new/close/switch), close popups, click, type, scroll, or "
"screenshot. webSearch only returns text and shows nothing on screen; this tool "
"actually navigates the visible browser. Only available in screen-share mode. "
"Never claim you did any of this unless this tool returns success."
)
@property
@@ -57,14 +59,16 @@ class ControlBrowserTool(Tool):
"action": {
"type": "string",
"enum": [
"status", "listTabs", "navigate", "back", "forward",
"refresh", "newTab", "closeTab", "activateTab",
"status", "listTabs", "navigate", "search", "back",
"forward", "refresh", "newTab", "closeTab", "activateTab",
"closePopups", "click", "type", "scroll", "pressKey",
"screenshot",
],
"description": "What to do in the browser.",
},
"url": {"type": "string", "description": "Target URL/site for navigate/newTab (e.g. 'naver.com')."},
"query": {"type": "string", "description": "Search text for action 'search'."},
"site": {"type": "string", "description": "Search site for action 'search': naver, google, daum, youtube, bing."},
"index": {"type": "integer", "description": "Tab index for closeTab/activateTab (from listTabs)."},
"selector": {"type": "string", "description": "CSS selector for click/type."},
"text": {"type": "string", "description": "Text to type."},
@@ -129,6 +133,10 @@ class ControlBrowserTool(Tool):
def _summarise(action: str, args: Dict[str, Any], data: Dict[str, Any]) -> str:
if action == "navigate":
return f"브라우저에서 {data.get('url', args.get('url'))} 로 이동했습니다."
if action == "search":
return f"{data.get('site', '')}에서 '{data.get('query', args.get('query'))}'를 검색해 화면에 띄웠습니다."
if action in ("back", "forward", "refresh"):
return f"브라우저: {action} 완료 ({data.get('url', '')})."
if action in ("status", "listTabs"):
tabs = data.get("tabs", [])
lines = "\n".join(f" [{t['index']}] {t.get('title') or t['url']}" for t in tabs) or " (탭 없음)"