Commit Graph

5 Commits

Author SHA1 Message Date
javis-bot
decd7ae477 fix(v2): YouTube no longer stops 45 s in, consent and the bot's video keep their tools
From the live conversation on 0.1.40 (2026-11-18):

- The bot browser told pages a program drives it (navigator.webdriver), and
  YouTube stopped every video about 45 s in with "문제가 발생했습니다"
  (ump.spsrejectfailure, reproduced twice on the game PC). Chrome now starts
  with --disable-blink-features=AutomationControlled, as the M0 video check
  already did; the live Chrome test checks webdriver is false.
- "네." to "계산기를 실행해도 될까요?" got webSearch and botBrowser from the
  router, so the consent call was refused and the calculator never started.
  Waiting tools: pcControl stays on offer while a request asked in the
  previous reply waits, botBrowser while it holds the asker's page.
- "소리 반으로 줄여 줘" for the bot's video worked on the person's own Edge
  window. botBrowser gets a volume action (media elements and YouTube's player),
  the page shows its volume, its description names its actions first (the part
  the planner sees), pcControl's says the bot's browser is not among its
  windows, and a turn whose plan has no step while a tool holds something for
  the asker gets the one-step nudge for it.

Evals (qwen3.6 on the game PC): volume 0/3 before, 8/8 after; "네." consent
10/10; pcControl 10/10; botBrowser 16/18 (the NewJeans play case fails 0/6
before this change too). Unit: Python 2542 passed, 13 failed (the same 13 as
before); app 348 passed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-19 00:47:38 +09:00
javis-bot
cda8db7fce fix: play on YouTube in one step, Korea as the default location, searches in the user's language
- botBrowser `play`: searches YouTube for the words as said, opens the first video (not an ad,
  without the playlist) and starts it, in one call (an address given opens that page); `type`
  also takes `query`. Live 0.1.19: the model stopped on YouTube's home page or the results, a
  retry was refused as a third call, and the answer still said the video played.
- `brain.location` (기본 위치, default 대한민국) goes to the brain as `location_name`: the time line
  says it without GeoIP (no database on the game PC), and getWeather without a place uses it.
- Planner and webSearch keep search words in the user's language (never translated into
  English), so recommendations fit where the user is.

Evals (qwen3:8b, 3 runs): play ending on a playing video found with the words as said 0/6 -> 5/6;
Korean searches for Korean requests 6/9 -> 8/9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 17:26:30 +09:00
javis-bot
420c9c73f3 fix: stopped sentences recorded only as far as heard, search words as plain query, paused video said so
- Playback sends how long the clip playing when a request was stopped had played (tts_played
  last_ms); the worker cuts the last sentence of that recording to it.
- botBrowser open takes the search words as plain `query`: the tool puts them, encoded, into the
  address's empty search parameter or types them into the site's search box. The description
  tells the model to search this way and never put words into an address itself (live: 아이유
  hand-encoded into 아이우). Eval (qwen3:8b, 3 runs): right words reach YouTube 1/3 -> 3/3.
- When a page's main video does not play (also after the tool pressed play, or it failed to load),
  a note after the page says it is not playing and not to say it plays. Eval: a paused video
  reported as playing 1/3 -> 0/3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 16:12:04 +09:00
javis-bot
56212c06dc fix(brain): botBrowser keeps a tab per person, one deadline, and never repeats an action
- each asker has a tab and connection of their own (at most 5, least recently used closes first)
- links, forms and window.open that would open a new tab load in the same tab
- one 15 s deadline covers connecting, the action and the page; events do not extend it
- a broken connection retries only open and read; click, type and back may have run and are not repeated
- <<< and >>> in page content are replaced so it cannot close the fence
- the description says how to search on a site (open, then type with submit=true); eval case 0/3 -> 3/3 on qwen3:8b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:33:00 +09:00
javis-bot
077db5031c feat(brain): botBrowser tool drives the bot's own Chrome over CDP
open, read, click (label or [n]), type (optionally Enter) and back; page content comes back fenced
as untrusted data; only http(s) addresses; no LLM call. Offered only when the app started the bot
browser (JARVIS_BROWSER_CDP) and the browser feature is on; meanwhile fetchWebPage, which has no
address guard, is not offered. Eval evals/test_bot_browser_tool.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 22:18:05 +09:00