Commit Graph

43 Commits

Author SHA1 Message Date
javis-bot
cda8db7fce fix: play on YouTube in one step, Korea as the default location, searches in the user's language
- botBrowser `play`: searches YouTube for the words as said, opens the first video (not an ad,
  without the playlist) and starts it, in one call (an address given opens that page); `type`
  also takes `query`. Live 0.1.19: the model stopped on YouTube's home page or the results, a
  retry was refused as a third call, and the answer still said the video played.
- `brain.location` (기본 위치, default 대한민국) goes to the brain as `location_name`: the time line
  says it without GeoIP (no database on the game PC), and getWeather without a place uses it.
- Planner and webSearch keep search words in the user's language (never translated into
  English), so recommendations fit where the user is.

Evals (qwen3:8b, 3 runs): play ending on a playing video found with the words as said 0/6 -> 5/6;
Korean searches for Korean requests 6/9 -> 8/9.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 17:26:30 +09:00
javis-bot
a20ebe4eae feat(brain): keep how each answer was made (records, tools, plan, thinking, tool calls)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-16 03:00:10 +09:00
javis-bot
53bccb620d feat: screen viewing, Jarvis sees the whole screen, a monitor or a window for registered people
- screenView tool (screen_view.spec.md): whole screen, monitor by number, window by part of its
  title or program, or the list of monitors and windows; fast, normal or detailed pictures.
- The app captures with Windows Graphics Capture (app/src/screen.rs, admin.spec.md "Screen
  viewing"), decides for the asker (only people registered for PC control), never picks, lists
  or shows excluded programs (pcControl.screenExclude, password managers by default), refuses
  minimised windows and black pictures, and records screen.view in the activity log.
- Pictures ride on the tool message (ToolExecutionResult.images); llm.py sends them to
  OpenAI-compatible servers as image parts after the tool messages. They stay in that reply only:
  not in the tool carryover, logs or prompt dumps.
- The LLM reads pictures only when screen viewing is available (--vision gpu normally, cpu in a
  game), measured apart for the placement; otherwise screenView is a disabled tool.
- Worker: screen_action / screen_action_result, 4 MiB messages.
- Eval evals/test_screen_view_tool.py; Windows check tests/screen_windows.rs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 17:02:59 +09:00
javis-bot
a5bd25b86f feat: personal notes, each person's standing instructions for how Jarvis answers them
A personalNotes tool (add, list, delete the asker's own notes) offered only
with a notes backend. The app keeps the notes in config/notes.json and sends
the asker's with every voice, chat and task reply; the reply engine puts them
in a <person_notes> fence as that person's preferences, which never permit an
action or a tool. The worker's schedule bridge becomes a shared app bridge
for schedule_action and note_action. A 사람별 지침 screen edits and deletes
any note; features.notes turns the tool and the notes off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 17:10:53 +09:00
javis-bot
ea2d3ba958 feat: reminders and scheduled tasks (schedule tool, app store and delivery)
The brain gets a schedule tool (add, list, delete the asker's own reminders,
timers and tasks) offered only with a schedule backend. The voice worker asks
the app through schedule_action and runs fired tasks as reply via task with
read-only tools, no MCP and a fresh short-term memory. The app keeps the
schedules in config/schedules.json, fires them every 5 s, speaks them when the
person is in Jarvis's voice channel and otherwise posts in the chat channel
they asked in (or schedule.chatChannel for voice). A feature switch and a
예약 screen list and delete everyone's schedules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-14 16:27:42 +09:00
javis-bot
8b3905f1c2 fix(app): MCP discovery never blocks a turn; secrets masked as lines are written; TTS child gets no MCP secrets
- Discovery runs outside the MCP tool cache's lock (one at a time), so a turn reading the cached
  tools never waits for a slow server; a new conversation refreshes in the background.
- The log hub masks secrets before a line is kept or written, so a key replaced or removed later
  stays hidden on the screen, in copies, files and exports, also after a restart.
- The worker takes JARVIS_MCP_ENV out of its environment at start, before MeloTTS runs, and hands
  it to the brain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 10:07:52 +09:00
javis-bot
3ef4281180 feat(app): features screen with tool switches and MCP servers; v2 drops host file and screen tools
- Settings `features` (weather, webSearch, meals) and `mcpServers` (name, command, args, env names,
  enabled), edited on the new 기능 screen. The tools of a feature that is off go into the brain's
  `disabled_tools`; only enabled MCP servers go into its `mcps`.
- MCP environment values (API keys) live only in the encrypted secrets (`mcp:<server>:<NAME>`),
  shown as stored or not, masked in logs, removed with their server, and reach the worker only in
  `JARVIS_MCP_ENV`, which the brain takes out of its environment before giving each server its own.
- `enabled_builtin_tools(cfg)` is the one list the router, toolSearchTool and the executor use, so
  a turned-off tool is neither offered nor run, also when the model names it.
- The v2 brain never offers v1's `localFiles` (home folder read/write/delete) and `screenshot`:
  anyone in the voice or chat channels could ask for them. Screen viewing comes in M7.
- The brain connects MCP servers on its own thread and logs each outcome without debug logging.
- Stored secrets are masked in the log hub as soon as the data folder is found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:56:17 +09:00
javis-bot
07c701be54 fix(reply): a tool call written in Qwen's XML keeps its argument types
Every <parameter> value was passed as text, so a delete confirmed with
confirm=true reached chatChannels as "true" and asked again. Values now take
the type the tool's input schema declares (boolean, integer, number, array,
object); a value that does not fit stays text. The roadmap's eval summary now
names the reply-length failure apart from the later passing runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:21:45 +09:00
javis-bot
e6ee152774 fix(voice_worker): plan chat tool turns, and keep v2 answers to the core
- A chatChannels pick from the router counts as a tool turn for the intent
  judge ("post this in the schedule channel" needs the planner).
- The default-language directive names an explicit language request as its
  one exception; the history block says plans and appointments people
  mentioned need no calendar tool; why/how answers give only the core reason.

Evals (Qwen3.6, v2 options): reply length 3/3 (v2 185-205 chars, v1 315-356),
language, history, chat channels and tool-skip 22/22; full suite 219 passed,
6 failed (paris/london, graph branch and four v1 single-user memory cases).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:06:40 +09:00
javis-bot
46e4e75d05 feat(voice_worker): a short voice prompt and one judgement before each v2 answer
- compact_prompt: the v2 persona in a tenth of v1's words, answers as short as
  possible, the full tool guidance only on turns with a data tool. A greeting's
  reply prompt went from about 4,500 to 1,400 tokens on Strata.
- intent_judge: the tool router judges whether the turn needs tools; the
  planner runs only when it does. prepare_intent runs the pick ahead on a
  partial transcript (worker message prepare), and the reply reuses it.
- Without stream_browser the screen-share tools are not offered.
- The time context carries the 12-hour form (10:37 was read as pm).
- A <tool_call> block written as the reply is parsed as a call, not spoken.

Evals (Qwen3.6 on Strata, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1):
221 passed, 4 failed (known failures); new evals/test_voice_reply_length.py
(v2 answers 234 chars against v1's 416) and test_reply_language_default.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 12:25:15 +09:00
javis-bot
7623d6d002 feat(voice_worker): the v2 brain answers in Korean unless asked for another language
The brain passes default_language (worker --reply-language, default Korean) to
the reply engine; unlike the output_language lock it gives way when the user
explicitly asks for a language. Tools get the ko code (--tool-language).

Eval evals/test_reply_language_default.py (Qwen3.6 on Strata): 6/6, English
questions answered in Korean, an explicit English request in English, and the
control without the default answers English in English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:34:12 +09:00
javis-bot
3ec430fd25 fix(reply): known user location reaches getWeather; retries are not blocked by the previous reply
- Drop the engine's deterministic Korean weather shortcut. It bypassed the
  chat model and the tool runner, so a location the user had stated could
  never be used. The tool already returns one clean Korean sentence.
- getWeather and the tool-argument prompts pass the place the user has
  said they are (conversation or memory) and leave the argument empty
  otherwise, so the detected location is not copied into it.
- getWeather retries a "district, city, country" name with its leading
  parts dropped when the geocoder finds nothing.
- The planner gets the live time/location line and plans
  searchMemory topic='where the user lives' only when the location is
  unknown or disabled.
- The per-reply repeat guard counts only this reply's tool messages, so
  calls carried over from the previous reply no longer block a retry the
  user asks for.
- Memory extractor: the context block sits after the examples and the
  no-location example uses a different query, so a location already in
  context is not asked for again. New counterpart eval checks that it is
  still asked for when the context has none.
- Tool retry eval fails every call in turn 1, so a model that retries by
  itself inside turn 1 still exercises the user's "try again".
- Specs (planner, reply), docs/llm_contexts.md and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:55:45 +09:00
javis-bot
fe4e296361 feat(chat): chatChannels tool for the registered Discord chat channels
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
  delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
  backend, and then on every turn. Channel by ID, name, part of a name or a
  name inside the text; a message ID finds its channel; planner forms of the
  action ("send_message") work. Read and search results are fenced as
  untrusted data. delete only reports until called again with confirm=true
  in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
  store and sends chat_action, waiting for chat_action_result (15 s timeout,
  failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
  sent on worker ready and after login; chat_action runs send, edit (own
  messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
  "router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
  refuses in a voice channel's chat, reported as an error), delete, refuse an
  unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
  message and never deleted in the same turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:01:33 +09:00
javis-bot
3dbf1e8c56 feat(nutrition): meals belong to the person who logged them
- meals.user_id (v1 databases gain it on open, DEFAULT '' keeps their meals
  the single user's); insert_meal, get_meals_between and delete_meal take
  the person, and delete only removes that person's meal.
- turn_state.set_user / ToolContext.user_id carry the asker; logMeal,
  fetchMeals and deleteMeal use it. The v2 brain sets it for each reply.
- Tests: two people never see or delete each other's meals, no user is v1's
  single user, and a v1 database opens with its meals kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:32:36 +09:00
javis-bot
b3dcd12ec9 fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:28:25 +09:00
javis-bot
7a288129de fix(memory): rank the whole history and keep relevant recent records
- Search scored only the newest 5,000 matching records, so an old record
  behind enough newer chatter was never found. Both passes (word rarity counts
  and the score) now run in SQLite over every record in scope and only the
  best rows come back.
- context() excluded the recent window from search and then cut that window
  for the budget, so a relevant recent record could vanish. Search now covers
  the recent window, matches keep their half of the budget, and a matched
  record is removed from the recent block so it appears once.
- Matching used SQLite's ASCII-only lower() against Python's lowercasing.
  Stored text gets a folded column (NFKC + casefold) and query words are
  folded the same way, so case is ignored in every script.
- Prompt label "Records related to this question" (matches may now be recent);
  eval re-run 3x on qwen2.5:7b, all pass. Spec, memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:50:37 +09:00
javis-bot
54a465948a feat(memory): v2 conversation history store and its context in the reply prompt
- src/jarvis/memory/history.py keeps voice transcripts, Jarvis's answers,
  registered chat channel messages (with edits and deletes) and voice channel
  presence in one SQLite file. Every read is scoped to the asker: own records
  always, chat channels and others' words (only while the asker was in the
  same voice channel) behind settings, plus an optional since and off.
- Search is language-agnostic: Unicode words, prefixes keeping 60% of the
  word, rarer words weigh more, newer wins ties. 100k records search in
  24-61 ms here. context() returns the recent window and the matches cut to a
  token budget, matches at most half so chatter cannot push out the record
  the question is about.
- run_reply_engine(history=...) puts that context into the system prompt
  under "Information shared in previous conversations", fenced as data.
  evals/test_history_context_recall.py: with the block qwen2.5:7b answers the
  stored day in Korean and English (3 of 3 runs), without it the day is unknown.
- Spec history.spec.md, registry entry in CLAUDE.md, reply.spec.md,
  docs/llm_contexts.md, docs/v2/memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:45:33 +09:00
javis-bot
6b23a8adc7 feat(llm): talk to OpenAI-compatible servers such as Strata
A base URL that ends in /v1 now selects the OpenAI-compatible API: chat goes
to /chat/completions and embeddings to /embeddings. The conversion lives in
src/jarvis/llm.py only, so the reply engine and every other caller keep their
Ollama-shaped messages and responses.

- Requests drop private engine keys, send tool-call arguments as JSON strings,
  map num_predict to max_tokens and the thinking switch to
  chat_template_kwargs.enable_thinking, and leave out num_ctx and keep_alive.
- Responses come back with message.content, tool calls with dict arguments and
  an id, and message.thinking from reasoning_content.
- Streaming, the warm-up (one-token chat completion), the intent judge and the
  dictation clean-up use the same path. A server without an embedding model
  fails open to the text paths.
- Specs and docs/llm_contexts.md describe the transport; roadmap records M3
  step 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:32:48 +09:00
javis-bot
8cb7e57cda fix(reply): never deliver the tool-less answer when the planned call cannot run
When the skipped-tool guard could not run the planned call (not a literal
call, or arguments the tool does not take) it delivered the answer it had
held back, which was given without the tool and could be a made-up value.
The planned call is now recorded as a failed tool result and the model
answers from that, so the user hears that it could not check or do it.

M0 kit: a case passes only when the final answer uses the tool result
(13 degrees, not a made-up 24) or, for the no-tool request, says it
cannot do it; the no-tool request now counts. The kit mirrors the
engine's failed-result path, and a failing model moves on to the next
candidate on these stricter checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:19:30 +09:00
javis-bot
903e0eb9de fix(reply): stop models answering past an available tool
M0 on Strata caught both v2 models skipping a tool they had been given:
the recommended model made up "24 degrees, sunny" instead of calling the
weather tool, and the fast model said "timer set" without calling the
timer tool (3 of 3 each). qwen2.5:7b does the same through the engine.

- Prompt: TOOL_HONESTY_RULES in the persona (live facts only from a tool
  result this turn, actions only claimed after a tool reports success, no
  answer text next to a tool call), restated at the end of the system
  message by tool_honesty_reminder() with the turn's real tool names.
- Engine: skipped-tool guard. When a native-tool model answers before any
  tool has run and the plan names a callable tool as a concrete call, the
  answer is dropped and the plan steps are direct-executed, then the model
  answers from the results. Only literal, schema-valid steps run (regex
  fast path, no LLM repair); otherwise the held-back answer is delivered.
- Planner: the AVAILABLE TOOLS lines now carry argument names
  (planner_tool_catalog), so concrete steps use the tool's real keys.
- Eval evals/test_tool_skip_confabulation.py; on qwen2.5:7b, 3 runs each,
  available-tool cases went from 0/12 to 9/12 calling the tool.
- M0 kit: tools_fast / tools_recommended stages (run_m0_tools.bat) rerun
  tool calling before and after the fix on the real models, loading the
  production system_prompt.py and planner.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:05:40 +09:00
javis-bot
83999a5b0b fix(prompts): classify every sub-8B model (2b/4b/5b/6b) as SMALL
Some checks failed
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m21s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
detect_model_size only matched 1b/3b/7b, so a genuinely small model such as
qwen*:4b fell through to LARGE and got the terse, less-guided prompt set it
can't follow — contributing to off-tone, rambling replies. Extend the small
patterns to cover all sub-8B sizes (1b-7b) across :/-/_ separators and sync the
spec table.
2026-06-24 18:31:54 +09:00
javis-bot
7bb9718c34 feat(reply): cap spoken replies at a single sentence
Replies stayed long because the prompt stack gave conflicting length signals:
the persona said "one sentence (two at the very most)" AND told the model to
"state the answer in a sentence, then add a dry observation" (a 2nd sentence),
while voice_style said "two to three sentences maximum". The model followed the
longest. Make all three sources agree on exactly one sentence: the persona's
aside must now fold into the same sentence as a trailing clause (never a 2nd
sentence), voice_style caps at one sentence, and agents/llm.md says 한 문장.
Shorter replies also cut Edge-TTS latency, since synth time scales with text
length. Specs (prompts.spec.md) and docs/llm_contexts.md updated; deterministic
prompt-contract tests added.
2026-06-24 17:55:27 +09:00
javis-bot
5ee47827f3 perf: cap chat output tokens via ollama_num_predict to bound reply latency
Spoken (TTS) replies are 1-2 sentences, so an unbounded num_predict only
exposes the worst case where the chat model rambles or loops. Add an
ollama_num_predict config (default 512, 0 disables) wired into the reply
loop's chat call on both the native- and text-tool paths. The 512-token
headroom stays well above this app's short tool-call JSON, so capping never
truncates a tool call. This keeps the user's quality model instead of
downgrading it. Configurable in the container via OLLAMA_NUM_PREDICT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00
javis-bot
2f000ac6c8 feat: load operator instructions from agents/*.md into the reply prompt
Some checks failed
Release / semantic-release (push) Successful in 30s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
Drop Markdown files into an agents/ folder and their contents are appended to
the main reply LLM's system prompt, so an operator can extend the assistant's
rules/tone without code changes. Files are concatenated in filename order
(use 00-, 10- prefixes to control ordering) and re-read once per turn, so edits
apply on the next reply with no rebuild/restart. Fail-open: a missing, empty,
or unreadable folder yields no instructions and never breaks a reply.

- load_agent_instructions() in system_prompt.py (AGENTS_DIR env, default
  /app/agents); reads *.md only, skips blanks, ignores non-dir paths
- engine.py appends it alongside the existing settings-UI llm_instructions,
  under the same "Additional instructions from the operator:" framing
- docker-compose.yml bind-mounts ./agents:/app/agents:ro and sets AGENTS_DIR
- agents/example.md.sample starter template (.sample is not loaded)
- tests cover ordering, md-only filtering, blank-skip, env/arg resolution,
  and fail-open paths
- README, .env.example, docs/llm_contexts.md updated

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 00:57:54 +09:00
javis-bot
7870a76314 fix: persona uses settings output_language, matching the reply directive
The persona prompt was built from the raw OUTPUT_LANGUAGE env while the
reply-language directive read the settings-web UI value (config JSON), so
changing the language in the settings page was honoured by the directive
but ignored by the persona, leaving them contradicting each other.

Add _resolve_output_language() as the single source of truth (config wins
over env) and feed the same resolved value to both build_system_prompt()
and reply_language_directive(). Update docs/llm_contexts.md to match.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 19:47:26 +09:00
javis-bot
b3088dd75f fix: settings output_language overrides the compose env default
The settings-UI output_language was ignored because the OUTPUT_LANGUAGE env took
precedence. Read the config value first, fall back to env, so changing the
language in /settings actually applies.
2026-06-15 16:57:01 +09:00
javis-bot
84e435f916 feat: settings web UI (models / STT / TTS speed / language / LLM instructions)
Adds /settings (served by the bridge) to change the LLM model (from installed
Ollama models), Whisper model, TTS engine + MeloTTS speed, output language,
agentic max-turns, thinking mode, and free-form LLM instructions — live, with a
'apply' that restarts the bridge + TTS worker. Settings persist to the runtime
config JSON; engine reads output_language + llm_instructions and the TTS worker
reads melo_speed from it. Bridge port publishable for access.
2026-06-15 13:05:46 +09:00
javis-bot
bdb012fc7c fix: weather passes named city from utterance; clean navigate reply
GeoIP auto-detect is unavailable in the container, so '부산 날씨' failed
(no location). Extract the city from the utterance and pass it to getWeather.
Also report navigate by site name instead of the mid-load about:blank url.
2026-06-15 01:02:35 +09:00
javis-bot
49061d30f0 fix: deterministic browser navigate for 'go back to <site>' (구글로 돌아가)
The 7B narrated '구글 메인으로 돌아갑니다' without acting, so the screen stayed on
Naver. Split site intent into SEARCH vs NAV: nav words (돌아가/이동/열어/메인/go
back) now drive controlBrowser.navigate to the site homepage directly, search
words run controlBrowser.search — both deterministically, no LLM.
2026-06-15 00:58:53 +09:00
javis-bot
c522e1b285 fix: deterministic weather → one clean Korean sentence (no 'Celsius')
getWeather now returns only the Korean sentence (지금 <곳> 날씨는 <상태>, 기온 N도
(체감 M도)입니다) with no English/°C source. A deterministic weather path in the
engine returns it verbatim, bypassing the 7B which was rephrasing into multiple
sentences and leaking 'Celsius'.
2026-06-14 22:46:03 +09:00
javis-bot
d970bf276e fix: harden Korean-only output lock (front+end, explicit script ban)
qwen2.5:7b leaked Chinese/Cyrillic mid-reply despite the OUTPUT_LANGUAGE
lock, which was buried mid-prompt. Repeat the lock at the END of the system
prompt (recency) and ban specific foreign scripts explicitly.
2026-06-14 22:25:33 +09:00
javis-bot
11a72cb296 fix: deterministic on-screen site search + lock replies to Korean
- Site-specified search ("네이버에서 X 검색해줘") now runs controlBrowser.search
  directly in the engine when broadcasting, instead of relying on the 3B model
  to emit the tool call (it kept narrating "검색하겠습니다" without acting).
- Set OUTPUT_LANGUAGE=ko so replies are Korean-only — stops the small model
  leaking CJK/Hanja and English fragments (每, 朗, "feels like") into weather
  and other answers, and keeps them concise.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 21:07:26 +09:00
javis-bot
37759f2b2c perf: conversational fast-path (skip enrichment) + shorter silence wait
Greetings/small-talk routed no data tool yet still ran the episodic memory
enrichment (LLM keyword extract + diary/graph search, ~1s) every turn. Skip it
when the router picked no external-data tool — the always-injected warm profile
still personalises the reply. Also drop the voice silence-detection wait
800ms -> 600ms for snappier turn-taking. Warm "안녕" now lands well under the
3-4s target.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 03:21:52 +09:00
javis-bot
4bc7f836ae fix: native tool-calling for qwen2.5 (tools actually fire) + kill Chrome infobars
Root cause of "weather/search do nothing": the engine forced TEXT tool-calling
for all <=7B models, but qwen2.5:3b emits clean NATIVE tool calls and fails at
the text format — so it just narrated ("부산 날씨는 맑습니다") and never called
getWeather/webSearch/controlBrowser. Use native tool-calling for tool-capable
small families (qwen2.5/qwen3/llama3.x/mistral); native still auto-falls back
to text on HTTP 400, so non-tool models (gemma) are unaffected.

Also launch Chrome with --test-type (removes the "--no-sandbox unsupported
flag" infobar) and disable the Translate feature/popup.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 03:08:49 +09:00
javis-bot
642fa42561 fix: drop webSearch when a site is named in screen-share, forcing controlBrowser
The 3B model kept choosing webSearch over controlBrowser even when offered, so
'네이버에서 X 검색' still used the invisible web path. When broadcasting and the
user explicitly names a site, remove webSearch from the allow-list so the
on-screen browser is the only search route.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:57:51 +09:00
javis-bot
f3d34bab4d fix: always offer controlBrowser in screen-share so on-screen search works
The small router reflexively routed every "search/open" intent to webSearch
and never surfaced controlBrowser, so "네이버에서 X 검색해줘" did nothing on the
broadcast. Union controlBrowser (+browseAndPlay) into the allow-list every
turn in screen-share mode (like setBroadcast), and steer the model in the
system prompt to prefer the on-screen browser over webSearch when available.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 02:55:33 +09:00
javis-bot
2c38e7576d perf: unify Ollama num_ctx so a voice turn keeps one resident model
Ollama keeps a separate loaded model instance per (model, num_ctx). The
main agentic chat used num_ctx=8192 while the router/enrichment/digest
passes used 4096, so every voice turn forced at least one cold reload
(~3.4s) when switching context sizes — the dominant per-turn latency
(measured: resident chat call 0.27s vs cold 3.4s).

Introduce a single OLLAMA_NUM_CTX (default 8192, env-tunable for tight
VRAM) used by call_llm_direct, chat_with_messages, call_llm_streaming and
the planner, collapsing a turn to one resident instance.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-14 00:19:53 +09:00
javis-bot
c8a04a110f fix(brain): recover colon-JSON and single tool_call object forms
qwen2.5:3b emits tool calls in text shapes the parser dropped, breaking
two reviewer-reported behaviours:

- `getWeather: {"location": "Seoul"}` (a JSON object after the colon) was
  dumped wholesale into {"query": "{...}"}, so `location` never reached the
  tool. getWeather then ran with empty args, returned the auto-detected
  location's weather, the model noticed the mismatch and retried — looping up
  to 8 times before giving up with an English error. Now the JSON object after
  the colon is parsed directly as the argument dict.
- `call_stop: {"id":..., "function": {"name": "setBroadcast",
  "arguments": "{\"action\": \"stop\"}"}}` — a single tool_call object without
  the `tool_calls: [...]` array wrapper, behind a `call_xxx:` label — matched
  no form, so the raw JSON leaked to the user AND setBroadcast never ran
  ("방송 꺼줘" did nothing). Now name + arguments are pulled from the embedded
  `function` object when the name is in the allow-list.

Field-captured from the live qwen2.5:3b brain (2026-06-12). Tests cover both
shapes, non-ASCII args, dict/string arguments, and unknown-tool rejection.
2026-06-12 21:59:26 +09:00
javis-bot
8a2a109d5e feat(brain): make OUTPUT_LANGUAGE lock robust on small models
Harden the reply-language lock so qwen2.5:3b reliably stays in the locked
language instead of leaking the query language back in:

- reply_language_directive(): single resolver with clear precedence —
  explicit OUTPUT_LANGUAGE lock wins over the Piper/Chatterbox English-only
  fallback (this deployment's actual TTS is Korean MeloTTS, so the legacy
  English lock was both wrong and contradicting the Korean lock).
- Stronger, override-explicit directive wording, inserted near the FRONT of
  the system prompt so a small model gives it primacy over the persona.
- build_system_prompt(output_language=...): rewrite the persona's "in the
  user's language" clause to the locked language so the persona stops
  fighting the lock.
- docs/llm_contexts.md: document the resolver, precedence, and placement.

Live-verified on the running brain (qwen2.5:3b): Korean voice-style input
and a cold English query both return fully Korean replies with no CJK/Hanja
leak. Tests cover unset/set/agnostic/whitespace + precedence + persona rewrite.
2026-06-12 21:18:47 +09:00
javis-bot
006a32276a feat(brain): add OUTPUT_LANGUAGE reply-language lock
Add an optional OUTPUT_LANGUAGE env var that forces every reply into a
single language. When set, output_language_directive() injects a "respond
only in <language>" instruction (also forbidding other scripts) into the
chat loop's system prompt, next to the existing TTS English-only lock.
Empty (default) keeps the multilingual "reply in the user's language"
behaviour, so upstream is unaffected.

For the Korean-only deployment this also suppresses the occasional trailing
CJK/Hanja fragment qwen2.5:3b leaks on free-form chit-chat.

- system_prompt.py: language-agnostic output_language_directive() helper
- engine.py: read OUTPUT_LANGUAGE, append directive in _build_initial_system_message
- docker-compose.yml + .env.example: document/pass the new var
- docs/llm_contexts.md: note the new gating on the main reply context
- tests: cover unset/set/agnostic/whitespace cases
2026-06-12 21:08:44 +09:00
javis-bot
03375a77d0 fix(docker+brain): make container Chrome search work, harden, improve toggle
Addresses review findings on the dockerized stack:
- Container Chrome search was dead: add --remote-debugging-port + a non-default
  --user-data-dir (Chrome 136+ refuses CDP on the default profile), add the
  playwright dep (browse-search.mjs connectOverCDP) with browser download
  skipped, and connect to 127.0.0.1 not "localhost" (container localhost -> ::1
  while Chrome binds IPv4). Verified: browse-search returns real results.
- Broadcast toggle reliability: always offer setBroadcast in screen-share mode
  (the embedding/keyword router dropped it for non-English utterances) and make
  its description force a tool call. "방송 꺼줘"->stop now 5/5; no false triggers.
- Stop the broadcast on voice leave (no orphaned stream).
- Security: bind VNC/noVNC to loopback by default (VNC_BIND override) and the
  bridge to the container loopback (BRIDGE_HOST=127.0.0.1), not published.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-11 10:44:56 +09:00
javis-bot
ca86390407 feat: couple broadcast to voice + voice-controlled broadcast toggle
Completes the STREAM_BROWSER=true behaviour:
- handleJoin auto-starts the broadcast on voice join and wires the session to
  the guild streamer; each turn reports the live state to the brain so search
  routes Chrome (live) vs Gemini (off).
- New setBroadcast tool lets the user toggle the broadcast by voice ("방송
  켜줘/꺼줘") via the LLM (no hardcoded phrases); it refuses when
  STREAM_BROWSER=false. The directive flows brain -> bridge (broadcast_action)
  -> bot streamer.start/stop, guarded by isActive() so it's idempotent.
- Per-turn IPC uses a thread-local (reply/turn_state.py) instead of threading
  params through the whole engine chain: bridge sets broadcasting in, tool
  records the directive out; Tool.execute exposes broadcasting on ToolContext.

Bot typecheck clean; brain covered by tests/test_set_broadcast.py (+ existing
routing tests). Specs + docs updated.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-11 10:08:30 +09:00
javis-bot
c4abf63f38 Add Discord-native hybrid front-end for Jarvis (bot + bridge)
Some checks failed
Release / semantic-release (push) Successful in 59s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 13m45s
Release / build-linux (push) Failing after 7m47s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Transform isair/jarvis into a Discord-controlled voice assistant running on
the Ubuntu VNC desktop, keeping the mature ~39k-line Python brain intact.

- bot/ (Node + bun, discord.js): /자비스 slash commands (ephemeral),
  voice channel join + voice receive/playback, pluggable VNC screen broadcast
  (selfbot live / noVNC / screenshot)
- bridge/ (Python, Flask): wraps jarvis STT + run_reply_engine + Piper TTS
  behind a thin localhost HTTP API
- .env.example, scripts/ (start_bridge/start_bot/dev), README rewrite,
  docs/language-comparison.md and docs/vnc-xfce-setup.md

Language decision: hybrid (Python brain + Node/bun Discord layer) because
Discord blocks bot video; native screen broadcast only works via a Node
selfbot library.
2026-06-09 14:51:05 +09:00