Commit Graph

80 Commits

Author SHA1 Message Date
javis-bot
d0cb540fa0 fix(app): collection's 15 s limit also covers the name lookup, TLS and headers
The name lookup runs on a daemon thread waited for only until the deadline, and a watchdog
shuts the connection down at the deadline, so a server that sends its status line or headers
byte by byte within the per-read limit is cut off too. A response cut short is a failure.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:46:10 +09:00
javis-bot
9185265010 fix(app): collection connects only to the addresses it checked, within 15 s, logging no address
- the fetcher resolves once, requires every address to be public and connects only to those
  (TLS still checks the name), so DNS rebinding cannot reach the local network; redirects too
- 15 s for the whole fetch also against servers that send slowly; no URL in any log line
- a site whose title alone changed is kept again
- target names that differ only in case are refused (one Windows folder)
- the worker always gets the collection file, so results older than the keep days go also
  with no targets or the feature off
- the results screen lists every target with results, not only those in the newest 100

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:39:04 +09:00
javis-bot
b94e9dc112 feat(app): background collection frame for sites, search words and RSS feeds
Targets (none by default), interval and keep days on a new 백그라운드 수집 screen with the
results; the worker collects on its own thread through the web search's address check and
writes only to the data folder's collect/ (logs carry target names and counts).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 21:29:49 +09:00
javis-bot
8b3905f1c2 fix(app): MCP discovery never blocks a turn; secrets masked as lines are written; TTS child gets no MCP secrets
- Discovery runs outside the MCP tool cache's lock (one at a time), so a turn reading the cached
  tools never waits for a slow server; a new conversation refreshes in the background.
- The log hub masks secrets before a line is kept or written, so a key replaced or removed later
  stays hidden on the screen, in copies, files and exports, also after a restart.
- The worker takes JARVIS_MCP_ENV out of its environment at start, before MeloTTS runs, and hands
  it to the brain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 10:07:52 +09:00
javis-bot
3ef4281180 feat(app): features screen with tool switches and MCP servers; v2 drops host file and screen tools
- Settings `features` (weather, webSearch, meals) and `mcpServers` (name, command, args, env names,
  enabled), edited on the new 기능 screen. The tools of a feature that is off go into the brain's
  `disabled_tools`; only enabled MCP servers go into its `mcps`.
- MCP environment values (API keys) live only in the encrypted secrets (`mcp:<server>:<NAME>`),
  shown as stored or not, masked in logs, removed with their server, and reach the worker only in
  `JARVIS_MCP_ENV`, which the brain takes out of its environment before giving each server its own.
- `enabled_builtin_tools(cfg)` is the one list the router, toolSearchTool and the executor use, so
  a turned-off tool is neither offered nor run, also when the model names it.
- The v2 brain never offers v1's `localFiles` (home folder read/write/delete) and `screenshot`:
  anyone in the voice or chat channels could ask for them. Screen viewing comes in M7.
- The brain connects MCP servers on its own thread and logs each outcome without debug logging.
- Stored secrets are masked in the log hub as soon as the data folder is found.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-13 09:56:17 +09:00
javis-bot
5c510bb602 feat(app): start Strata from the data folder and point the brain at it
M5 step 4, second part. The services start the LLM first: the installed
Strata with the M0 kit's model and normal placement (models in
models/llm, parallel 1, or 2 when a chat channel answers every message,
pictures off, --restart), wait until /v1/models lists the model, and stop
it last. The first start downloads the model; the dashboard shows
Strata's last line and percentage, and stop also cancels a start (the
server is cleared). llm.url uses a server the app does not start.

On every start the app writes config/brain.json (Strata's /v1, the
listed model id, memory/jarvis.db) and hands it to the worker as
JARVIS_CONFIG_PATH. Loading a config with its own db_path no longer
creates ~/.local/share/jarvis.

service_start/service_restart run in the background. Settings gain
llm.context and llm.port (model screen) and llm.url (file only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:45:38 +09:00
javis-bot
a4436698c8 feat(app): M5 step 4a, install the runtime and voice models into the data folder
- download: resumed downloads checked against a pinned SHA-256.
- install: uv 0.11.20, the worker/brain venv (Python 3.12, the new
  src/voice_worker/requirements-brain.txt), the MeloTTS venv (Python 3.11,
  PyTorch cu128), Whisper large-v3-turbo, the Korean MeloTTS voice and
  Strata, the M0 kit's recipe; markers per step and version, cancel, retry,
  everything (caches, temp, HF and NLTK data) inside the data folder, the uv
  cache cleaned and the worker settings pointed at the venvs when done.
- The services give the worker the same environment once installed.
- The wizard's model step shows and drives the install.

Installed from an empty folder on Linux and started the services on it:
worker connected, STT and TTS ready, nothing written outside the folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 17:11:23 +09:00
javis-bot
eb0fa2f159 fix(app): log hub masks dotted tokens, keeps worker errors and drops the oldest lines by time
- redact: a token with sentence dots around it is masked too.
- Worker error messages are always logged (code and id), so the log screen
  and the error report show them without debug logging.
- The voice worker logs with stt, llm and tts categories, so its lines land
  in those tabs.
- The zip leaves out the oldest lines by the time they were logged, across
  every tab's file, not by file name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:43:01 +09:00
javis-bot
07c701be54 fix(reply): a tool call written in Qwen's XML keeps its argument types
Every <parameter> value was passed as text, so a delete confirmed with
confirm=true reached chatChannels as "true" and asked again. Values now take
the type the tool's input schema declares (boolean, integer, number, array,
object); a value that does not fit stays text. The roadmap's eval summary now
names the reply-length failure apart from the later passing runs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:21:45 +09:00
javis-bot
e6ee152774 fix(voice_worker): plan chat tool turns, and keep v2 answers to the core
- A chatChannels pick from the router counts as a tool turn for the intent
  judge ("post this in the schedule channel" needs the planner).
- The default-language directive names an explicit language request as its
  one exception; the history block says plans and appointments people
  mentioned need no calendar tool; why/how answers give only the core reason.

Evals (Qwen3.6, v2 options): reply length 3/3 (v2 185-205 chars, v1 315-356),
language, history, chat channels and tool-skip 22/22; full suite 219 passed,
6 failed (paris/london, graph branch and four v1 single-user memory cases).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 13:06:40 +09:00
javis-bot
46e4e75d05 feat(voice_worker): a short voice prompt and one judgement before each v2 answer
- compact_prompt: the v2 persona in a tenth of v1's words, answers as short as
  possible, the full tool guidance only on turns with a data tool. A greeting's
  reply prompt went from about 4,500 to 1,400 tokens on Strata.
- intent_judge: the tool router judges whether the turn needs tools; the
  planner runs only when it does. prepare_intent runs the pick ahead on a
  partial transcript (worker message prepare), and the reply reuses it.
- Without stream_browser the screen-share tools are not offered.
- The time context carries the 12-hour form (10:37 was read as pm).
- A <tool_call> block written as the reply is parsed as a call, not spoken.

Evals (Qwen3.6 on Strata, EVAL_COMPACT_PROMPT=1 EVAL_INTENT_JUDGE=1):
221 passed, 4 failed (known failures); new evals/test_voice_reply_length.py
(v2 answers 234 chars against v1's 416) and test_reply_language_default.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 12:25:15 +09:00
javis-bot
7623d6d002 feat(voice_worker): the v2 brain answers in Korean unless asked for another language
The brain passes default_language (worker --reply-language, default Korean) to
the reply engine; unlike the output_language lock it gives way when the user
explicitly asks for a language. Tools get the ko code (--tool-language).

Eval evals/test_reply_language_default.py (Qwen3.6 on Strata): 6/6, English
questions answered in Korean, an explicit English request in English, and the
control without the default answers English in English.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 10:34:12 +09:00
javis-bot
9ff307d7fe fix(voice_worker): the v2 brain answers in the asker's language
v1's tts_engine default (English-only Piper) made the reply engine force
English, so Korean questions got English answers. The v2 brain speaks
through the worker's MeloTTS and chat answers are not spoken, so it runs
with tts_engine=melo; an output_language lock still wins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 09:16:34 +09:00
javis-bot
9e75df266b fix: a delete question counts as asked only once the answer reached the person
A person's reply stopped counting as "unanswered" as soon as the worker sent
reply_text, so a delete question that could not be spoken (no TTS, a failed
or stopped speech) or that the app could not post still let the next
"응, 지워" confirm the delete.

- Worker: an answer reaches the person on a tts_done neither failed nor
  cancelled (spoken answers), on the app's reply_delivered ok (chat
  answers), or at reply_text (unspoken voice answers). tts_unavailable, a
  failed or cancelled tts_done, reply_delivered ok=false, no report within
  30 s and the session ending all mean it never did: a turn number is
  skipped, so the delete asked in it lapses. A reply sent before the
  previous answer reached them is still not their next turn.
- App: a chat answer reports reply_delivered with whether Discord took the
  post; a late, failed or empty answer reports it was not posted.
- Tests: no TTS, a failed speech, a chat post the app reports failed, and
  no report at all do not delete; a spoken and a posted question do.
  Conversation and worker client tests for the report.
- Specs: voice_worker.spec.md (reply_delivered, when an answer reaches the
  person), discord_core.spec.md, conversation.spec.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:39:48 +09:00
javis-bot
c328bda699 fix(voice_worker): a yes sent before the delete question was answered does not delete
A delete is confirmed only in the person's very next turn, but turn numbers
were given when a reply started. A "응, 지워" sent while the delete question
was still being written waited for the person's lock and then counted as the
next turn, so the delete went through.

- The worker counts each person's unanswered replies. A reply that arrived
  before their previous answer was given skips a turn number, so a delete
  asked in that answer lapses.
- After an answer the person never got (failed, or stopped before it was
  spoken) a turn number is skipped as well: it asked them nothing.
- Tests with the real chatChannels tool and an agreeing consent check: a yes
  sent before the answer does not delete, a yes after a failed answer does
  not delete, a yes after the answer deletes.
- Specs: voice_worker.spec.md (turn numbering), chat_channels.spec.md (the
  next turn is a message sent after the question was received).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:32:25 +09:00
javis-bot
19f984ae73 fix(voice_worker): one person's voice and chat answers are written one at a time
The voice and chat lanes let a person's voice reply start while their chat
reply was still running. The chat delete consent counts "the very next
message" by turn number, so a message that started before the delete was
asked could pass as the confirmation, and the person's dialogue memory
was updated out of order.

- A reply takes its asker's lock, then its lane's: one person's replies
  stay one at a time in the order asked, different people's voice and
  chat answers still overlap, and a reply waiting for its asker holds no
  lane. Test: a person's voice question starts only after their chat
  answer.
- Specs and roadmap: the lanes are per person; the roadmap no longer says
  voice answers wait behind chat answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-12 01:27:13 +09:00
javis-bot
2f69557e1b fix: voice answers no longer wait behind chat answers; late chat answers are dropped
- The voice worker writes voice and chat replies in separate lanes (one
  of each at a time), so a chat answer being written never holds up a
  voice answer. Tests: a voice reply finishes while a chat reply is still
  running; replies of one kind stay one at a time.
- A chat answer arriving 5 minutes or more after it was asked is not
  posted, checked when it arrives.
- Specs: the lanes, and that Strata needs --parallel 2 for both lanes'
  LLM calls to run at once until the app starts Strata itself (M5).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:42:30 +09:00
javis-bot
f892a1dce9 feat: answer calls in the voice channel through the response policy
- conversation.rs: the loop between the core's events and the rest.
  Finals go through call detection and the response policy; answers are
  asked of the brain and spoken from the app's queue, an interruption
  cancels only that answer, a voice stop silences everything, and missed
  requests are kept without text (last 100). Tested with a fake voice side.
- Core: answer(user, text) adopts a playback request the worker speaks
  under the same id, cancel(id) stops one request, ask_addressed and
  ask_rank pass the conversation's questions; new SpeakingStopped,
  Addressed and Ranked events.
- Worker client: reply, addressed and rank messages and their results; a
  reply with no answer, refused or not sent ends its speech as failed.
- Voice worker: addressed and rank messages on their own threads, beside
  a running reply; a spoken reply cancelled while the brain is answering
  is not spoken. judges.py holds the two short LLM judgements with the
  heard text fenced as data; eval evals/test_voice_judges.py.
- Specs (conversation, discord core, voice worker), docs/llm_contexts.md
  and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 22:26:47 +09:00
javis-bot
3ec430fd25 fix(reply): known user location reaches getWeather; retries are not blocked by the previous reply
- Drop the engine's deterministic Korean weather shortcut. It bypassed the
  chat model and the tool runner, so a location the user had stated could
  never be used. The tool already returns one clean Korean sentence.
- getWeather and the tool-argument prompts pass the place the user has
  said they are (conversation or memory) and leave the argument empty
  otherwise, so the detected location is not copied into it.
- getWeather retries a "district, city, country" name with its leading
  parts dropped when the geocoder finds nothing.
- The planner gets the live time/location line and plans
  searchMemory topic='where the user lives' only when the location is
  unknown or disabled.
- The per-reply repeat guard counts only this reply's tool messages, so
  calls carried over from the previous reply no longer block a retry the
  user asks for.
- Memory extractor: the context block sits after the examples and the
  no-location example uses a different query, so a location already in
  context is not asked for again. New counterpart eval checks that it is
  still asked for when the context has none.
- Tool retry eval fails every call in turn 1, so a model that retries by
  itself inside turn 1 still exercises the user's "try again".
- Specs (planner, reply), docs/llm_contexts.md and the roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 21:55:45 +09:00
javis-bot
4621e6c1fd fix(chat): the delete consent check knows the pending message and reads the whole reply
The consent classifier is now told which message and channel are pending, so a yes for another
message ("yes, delete message 22" while 21 is pending) is a no and cancels the request. A reply
longer than 600 characters is no longer cut and judged by its start (which could drop a later
"don't"); it counts as no without a model call.

Eval evals/test_chat_delete_consent.py gains the different-message and long-reply cases:
17/17 on qwen2.5:7b, three runs. The previous prompt said YES 3/3 to
"응, 일정 채널에 있는 거 지워" while a message in 잡담 was pending.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 15:26:58 +09:00
javis-bot
0182823592 fix(chat): delete needs the person's own yes; chat action ids never repeat
- chatChannels delete with confirm=true now runs only in the person's very
  next turn and only when their own message of that turn agrees, checked by a
  small fail-closed YES/NO call on the router model chain (the message fenced
  as data). A refusal, question, other topic or error cancels the request;
  an unconfirmed request lapses after one turn. The reply model's confirm
  flag alone can no longer delete.
- chat_action ids carry a per-run nonce and the session, so a late answer
  meant for an earlier connection (or worker run) never completes a newer
  action.
- Tests: agree, refuse, other topic, lapse, classifier failure, late answer
  across reconnects. Eval test_chat_delete_consent: 12 of 12 on qwen2.5:7b,
  3 runs. Spec and llm_contexts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:08:35 +09:00
javis-bot
fe4e296361 feat(chat): chatChannels tool for the registered Discord chat channels
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
  delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
  backend, and then on every turn. Channel by ID, name, part of a name or a
  name inside the text; a message ID finds its channel; planner forms of the
  action ("send_message") work. Read and search results are fenced as
  untrusted data. delete only reports until called again with confirm=true
  in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
  store and sends chat_action, waiting for chat_action_result (15 s timeout,
  failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
  sent on worker ready and after login; chat_action runs send, edit (own
  messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
  "router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
  refuses in a voice channel's chat, reported as an error), delete, refuse an
  unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
  message and never deleted in the same turn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 14:01:33 +09:00
javis-bot
3dbf1e8c56 feat(nutrition): meals belong to the person who logged them
- meals.user_id (v1 databases gain it on open, DEFAULT '' keeps their meals
  the single user's); insert_meal, get_meals_between and delete_meal take
  the person, and delete only removes that person's meal.
- turn_state.set_user / ToolContext.user_id carry the asker; logMeal,
  fetchMeals and deleteMeal use it. The v2 brain sets it for each reply.
- Tests: two people never see or delete each other's meals, no user is v1's
  single user, and a v1 database opens with its meals kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:32:36 +09:00
javis-bot
b3dcd12ec9 fix(voice_worker): keep history and the brain's output scoped to who was there
- A final is stored under the channel, speaker name and time fixed when its
  end was decided. Decoding takes a while; a move, rename or late joiner
  meanwhile used to change where and when it was said, and who heard it.
- At start the worker closes presence a killed run left open
  (HistoryStore.close_open_presence), so a person who is gone no longer
  hears new speech after a restart.
- run_reply_engine(legacy_memory=False) skips v1's single-user memory (warm
  profile, extractor, diary, graph, digest); the v2 brain uses it so another
  user's profile or diary never reaches the prompt.
- While a brain reply runs, stdout and stderr are muted (also helper threads
  such as the web fetch pool); the worker logs through voice_worker/log.py to
  the original stderr.
- Live check: Ollama's /v1 fixes the context at 2,048 tokens and cut the
  prompt in earlier runs; with an 8k tag the chat-record question was answered
  4 of 4 and nothing reached the output. Noted in llm_contexts and the eval.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:28:25 +09:00
javis-bot
864b2e7fdb feat(voice_worker): record history from the app and answer reply requests with the brain
Worker (voice_worker.spec.md "History and brain"):
- --history <file> keeps the conversation history store; --brain hosts the
  jarvis reply engine (JarvisBrain, settings point ollama_base_url at /v1).
- voice_join / voice_leave / member_join / member_leave keep presence; the
  session end closes it. Non-empty finals are stored under the speaker's name
  in the current voice channel.
- chat_message / chat_edit / chat_delete keep registered chat channels.
- reply builds the asker's scoped history context, runs the brain on a thread
  (one at a time), answers reply_text, stores a voice answer, and with
  speak=true says it under the same id. memory settings in hello and
  memory_config. brain_unavailable without --brain.

App (discord_core.spec.md "History reporting"):
- WorkerClient sends the new messages; the core reports the voice channel
  and its heard members after a join, a move and every worker reconnect,
  member joins and leaves, voice_leave when a session ends, and every message
  of a readable registered channel (its own too, with `to` on replies), edits
  and deletes.
- live_discord runs the worker with --history and checks the store at the
  end: 8 finals under the speaker's name, the edited chat message only,
  Jarvis's own messages, 5 presence intervals, none left open (passed, 107.8 s).

Eval: history recall gains a chat-record case and fails a reply that reads a
record stamp out (field run with qwen2.5:7b did once in four).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 13:16:24 +09:00
javis-bot
7a288129de fix(memory): rank the whole history and keep relevant recent records
- Search scored only the newest 5,000 matching records, so an old record
  behind enough newer chatter was never found. Both passes (word rarity counts
  and the score) now run in SQLite over every record in scope and only the
  best rows come back.
- context() excluded the recent window from search and then cut that window
  for the budget, so a relevant recent record could vanish. Search now covers
  the recent window, matches keep their half of the budget, and a matched
  record is removed from the recent block so it appears once.
- Matching used SQLite's ASCII-only lower() against Python's lowercasing.
  Stored text gets a folded column (NFKC + casefold) and query words are
  folded the same way, so case is ignored in every script.
- Prompt label "Records related to this question" (matches may now be recent);
  eval re-run 3x on qwen2.5:7b, all pass. Spec, memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:50:37 +09:00
javis-bot
54a465948a feat(memory): v2 conversation history store and its context in the reply prompt
- src/jarvis/memory/history.py keeps voice transcripts, Jarvis's answers,
  registered chat channel messages (with edits and deletes) and voice channel
  presence in one SQLite file. Every read is scoped to the asker: own records
  always, chat channels and others' words (only while the asker was in the
  same voice channel) behind settings, plus an optional since and off.
- Search is language-agnostic: Unicode words, prefixes keeping 60% of the
  word, rarer words weigh more, newer wins ties. 100k records search in
  24-61 ms here. context() returns the recent window and the matches cut to a
  token budget, matches at most half so chatter cannot push out the record
  the question is about.
- run_reply_engine(history=...) puts that context into the system prompt
  under "Information shared in previous conversations", fenced as data.
  evals/test_history_context_recall.py: with the block qwen2.5:7b answers the
  stored day in Korean and English (3 of 3 runs), without it the day is unknown.
- Spec history.spec.md, registry entry in CLAUDE.md, reply.spec.md,
  docs/llm_contexts.md, docs/v2/memory.md and roadmap updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:45:33 +09:00
javis-bot
6b23a8adc7 feat(llm): talk to OpenAI-compatible servers such as Strata
A base URL that ends in /v1 now selects the OpenAI-compatible API: chat goes
to /chat/completions and embeddings to /embeddings. The conversion lives in
src/jarvis/llm.py only, so the reply engine and every other caller keep their
Ollama-shaped messages and responses.

- Requests drop private engine keys, send tool-call arguments as JSON strings,
  map num_predict to max_tokens and the thinking switch to
  chat_template_kwargs.enable_thinking, and leave out num_ctx and keep_alive.
- Responses come back with message.content, tool calls with dict arguments and
  an id, and message.thinking from reasoning_content.
- Streaming, the warm-up (one-token chat completion), the intent judge and the
  dictation clean-up use the same path. A server without an embedding model
  fails open to the text paths.
- Specs and docs/llm_contexts.md describe the transport; roadmap records M3
  step 1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 12:32:48 +09:00
javis-bot
5838f9edd8 feat(voice_worker): free a speaker's place when they leave the voice channel
- New speaker_leave message: ends the speaker's open utterance (its final is still sent) and drops
  their state, so a long session never hits the 32-speaker limit as people come and go.
- Spec and test cover more than 32 speakers in one session, each leaving mid-utterance.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 11:14:25 +09:00
javis-bot
1309525aa8 fix(voice_worker): the wake word hint may never add, drop or create a wake word
Hotword occurrences were counted with fuzzy matching, which never counted a one-letter wake word
(so a hinted "봇 ..." was accepted over a plain decode without it) and counted the real word
"서비스" as "자비스" (so the hint could turn it into the wake word). Occurrences are now counted
exactly (case, spacing and punctuation ignored); near misses are left as heard for call detection.
The real-noise eval now plays each recording as recorded and at -25 dBFS and reports how many clips
reached the model.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:40:42 +09:00
javis-bot
590cd07268 fix(voice_worker): even out speaker loudness and stop transcribing noise into words
- Loudness equalisation replaces peak normalisation: rumble below 60 Hz removed, level measured on
  the VAD-voiced windows only, brought to -20 dBFS (gain -20..+40 dB), soft limiter below 0.98.
- Only clearly voiced windows (VAD >= 0.5) count towards the 250 ms needed before an utterance
  reaches the model, so noise the VAD half-believes never calls Whisper.
- Hallucination gate: mean token log-probability >= -0.7 (the old len/(len+1) correction let short
  hallucinations through), compression ratio <= 2.4; no_speech_prob dropped (always 0.0 on turbo).
- With hotwords, the plain decode of the same encoder pass decides; the hint may fix a wake word's
  spelling but cannot add or drop one (it wrote the wake word on laughter and after real sentences).
- Opt-in real-model evals: voice-like babble with the hint, speech over babble, quiet and clipped
  speakers, and a JARVIS_NOISE_DIR folder of real noise recordings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:27:38 +09:00
javis-bot
a7685e9078 feat(voice_worker): M1 multi-speaker streaming STT and sentence TTS over a local WebSocket
- One shared faster-whisper model for every speaker, batched; finals go before partials and
  partial batches are capped at two so a final never waits behind a ten-speaker pass.
- Per-speaker 48k stereo to 16k mono conversion, streaming Silero VAD, utterance ends on VAD
  silence, Discord speaking_end, missing audio, or the 28 s Whisper window.
- MeloTTS in its own venv through a child process; sentences stream out one chunk each, with
  incremental text, cancel and failure reporting.
- Loopback-only server with a token from the environment; logs carry no transcript text.
- Spec in src/voice_worker/voice_worker.spec.md; unit tests with fake engines and an opt-in
  real-GPU test that streams ten speakers at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-11 10:00:06 +09:00
javis-bot
8cb7e57cda fix(reply): never deliver the tool-less answer when the planned call cannot run
When the skipped-tool guard could not run the planned call (not a literal
call, or arguments the tool does not take) it delivered the answer it had
held back, which was given without the tool and could be a made-up value.
The planned call is now recorded as a failed tool result and the model
answers from that, so the user hears that it could not check or do it.

M0 kit: a case passes only when the final answer uses the tool result
(13 degrees, not a made-up 24) or, for the no-tool request, says it
cannot do it; the no-tool request now counts. The kit mirrors the
engine's failed-result path, and a failing model moves on to the next
candidate on these stricter checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:19:30 +09:00
javis-bot
903e0eb9de fix(reply): stop models answering past an available tool
M0 on Strata caught both v2 models skipping a tool they had been given:
the recommended model made up "24 degrees, sunny" instead of calling the
weather tool, and the fast model said "timer set" without calling the
timer tool (3 of 3 each). qwen2.5:7b does the same through the engine.

- Prompt: TOOL_HONESTY_RULES in the persona (live facts only from a tool
  result this turn, actions only claimed after a tool reports success, no
  answer text next to a tool call), restated at the end of the system
  message by tool_honesty_reminder() with the turn's real tool names.
- Engine: skipped-tool guard. When a native-tool model answers before any
  tool has run and the plan names a callable tool as a concrete call, the
  answer is dropped and the plan steps are direct-executed, then the model
  answers from the results. Only literal, schema-valid steps run (regex
  fast path, no LLM repair); otherwise the held-back answer is delivered.
- Planner: the AVAILABLE TOOLS lines now carry argument names
  (planner_tool_catalog), so concrete steps use the tool's real keys.
- Eval evals/test_tool_skip_confabulation.py; on qwen2.5:7b, 3 runs each,
  available-tool cases went from 0/12 to 9/12 calling the tool.
- M0 kit: tools_fast / tools_recommended stages (run_m0_tools.bat) rerun
  tool calling before and after the fix on the real models, loading the
  production system_prompt.py and planner.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-10 15:05:40 +09:00
javis-bot
ffc16665e5 fix(controlBrowser): never report moveMouse/search success without a real move
Some checks failed
Release / semantic-release (push) Successful in 19s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m15s
Release / build-linux (push) Failing after 7m7s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
moveMouse returned ok:true even when humanHover did nothing (no on-screen box)
or the selector never matched — recreating the "claims it moved but didn't"
bug. Now: humanHover returns a boolean (and brings the element into view first);
moveMouse returns ok:false when the target isn't found or has no on-screen box,
and when site=naver/... whose box isn't on the current page it navigates to the
site home first before hovering. search now reports input=human|api-fallback|api
so a silent fallback to cursor-less DOM input is visible, and the tool surfaces
that note in the reply instead of implying a human-like search happened.
2026-06-24 19:17:46 +09:00
javis-bot
5629da7e9f feat(controlBrowser): add moveMouse (hover) action for the visible cursor
The tool had no cursor-move/hover action, so "move the mouse to the search box"
had nothing to call and a weak model just claimed it had moved it. Add a
moveMouse action wired to human.humanHover (real xdotool cursor), targetable by
CSS selector or site= (that site's search box). Also clarify in the tool
description that 'navigate' types into the address bar (no mouse) while
'search'/'type' move the real cursor to the on-page box and click before
typing, so the model picks the visible-cursor path when the user wants it.
2026-06-24 19:14:04 +09:00
javis-bot
83999a5b0b fix(prompts): classify every sub-8B model (2b/4b/5b/6b) as SMALL
Some checks failed
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m21s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
detect_model_size only matched 1b/3b/7b, so a genuinely small model such as
qwen*:4b fell through to LARGE and got the terse, less-guided prompt set it
can't follow — contributing to off-tone, rambling replies. Extend the small
patterns to cover all sub-8B sizes (1b-7b) across :/-/_ separators and sync the
spec table.
2026-06-24 18:31:54 +09:00
javis-bot
d5fd218c86 fix(reply): stop weak models parroting persona example facts
A 4b model replied to "하이" with "테니스 연습을 Trenches Gym에서..." — it copied
the literal "box at Trenches Gym" few-shot example embedded in the persona
prompt and mangled boxing into tennis, presenting a prompt example as if it
were a real user fact. Remove the copyable proper-noun example and add an
explicit guard: use ONLY names/places/activities that literally appear in the
memory section, never borrow them from the instructions or example wording.
2026-06-24 18:31:54 +09:00
javis-bot
7bb9718c34 feat(reply): cap spoken replies at a single sentence
Replies stayed long because the prompt stack gave conflicting length signals:
the persona said "one sentence (two at the very most)" AND told the model to
"state the answer in a sentence, then add a dry observation" (a 2nd sentence),
while voice_style said "two to three sentences maximum". The model followed the
longest. Make all three sources agree on exactly one sentence: the persona's
aside must now fold into the same sentence as a trailing clause (never a 2nd
sentence), voice_style caps at one sentence, and agents/llm.md says 한 문장.
Shorter replies also cut Edge-TTS latency, since synth time scales with text
length. Specs (prompts.spec.md) and docs/llm_contexts.md updated; deterministic
prompt-contract tests added.
2026-06-24 17:55:27 +09:00
javis-bot
140fc56f18 feat: play the Nth YouTube result in browseAndPlay via an index arg
Some checks failed
Release / semantic-release (push) Successful in 22s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m10s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
agents/llm.md promises "play the Nth video from the top", but browseAndPlay
only ever clicked the first result. Add an optional 1-based index argument
(default 1, backward-compatible) threaded to the Node helper, which now clicks
the Nth a#video-title and clamps to the number of results returned so asking
beyond the list plays the last available video instead of failing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00
javis-bot
5ee47827f3 perf: cap chat output tokens via ollama_num_predict to bound reply latency
Spoken (TTS) replies are 1-2 sentences, so an unbounded num_predict only
exposes the worst case where the chat model rambles or loops. Add an
ollama_num_predict config (default 512, 0 disables) wired into the reply
loop's chat call on both the native- and text-tool paths. The 512-token
headroom stays well above this app's short tool-call JSON, so capping never
truncates a tool call. This keeps the user's quality model instead of
downgrading it. Configurable in the container via OLLAMA_NUM_PREDICT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 15:33:45 +09:00
javis-bot
086dd5cde7 fix: accept edge as a valid tts_engine and migrate stale persisted engines
Some checks failed
Release / semantic-release (push) Successful in 34s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m17s
Release / build-linux (push) Failing after 7m9s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
load_settings() coerced any tts_engine outside {piper, chatterbox} to piper, so
with TTS_ENGINE=edge the reply engine saw "piper" and treated the voice as
English-only in reply_language_directive() (only the OUTPUT_LANGUAGE lock kept
replies Korean). Add "edge" (and "melo") to the accepted set so the engine is
labelled multilingual correctly.

Also: a stale tts_engine in the persistent /data/jarvis-settings.json (melo/xtts
from an earlier voice, no longer built) would override the configured engine via
the entrypoint merge and leave the bot silent. Reset those to the env engine
during the merge.

Verified: load_settings() with tts_engine=edge now returns "edge"; the merge
maps melo/xtts -> edge; reply_language_directive("edge") is multilingual; 27
tests pass.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 03:49:53 +09:00
javis-bot
2f000ac6c8 feat: load operator instructions from agents/*.md into the reply prompt
Some checks failed
Release / semantic-release (push) Successful in 30s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / build-linux (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
tests / Unit tests (Linux, Python 3.11) (push) Has been cancelled
Drop Markdown files into an agents/ folder and their contents are appended to
the main reply LLM's system prompt, so an operator can extend the assistant's
rules/tone without code changes. Files are concatenated in filename order
(use 00-, 10- prefixes to control ordering) and re-read once per turn, so edits
apply on the next reply with no rebuild/restart. Fail-open: a missing, empty,
or unreadable folder yields no instructions and never breaks a reply.

- load_agent_instructions() in system_prompt.py (AGENTS_DIR env, default
  /app/agents); reads *.md only, skips blanks, ignores non-dir paths
- engine.py appends it alongside the existing settings-UI llm_instructions,
  under the same "Additional instructions from the operator:" framing
- docker-compose.yml bind-mounts ./agents:/app/agents:ro and sets AGENTS_DIR
- agents/example.md.sample starter template (.sample is not loaded)
- tests cover ordering, md-only filtering, blank-skip, env/arg resolution,
  and fail-open paths
- README, .env.example, docs/llm_contexts.md updated

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-23 00:57:54 +09:00
javis-bot
c56ce1eb30 feat: human-like typing for browser Google and YouTube search
Some checks failed
Release / semantic-release (push) Successful in 26s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m11s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
Make the browser search helper search the way a person does: load the
site home page, type the query into the search box one key at a time, and
press Enter — for both Google `search` and `youtube` — instead of jumping
straight to a results URL. Supports the goal of a human-like assistant.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-22 21:09:15 +09:00
javis-bot
597207dd33 feat: reuse a signed-in Chrome profile for browser web search
Add CHROME_USER_DATA_DIR so the browser search fallback can open Chrome
against a dedicated, Google-signed-in profile instead of a fresh anonymous
session. A returning signed-in profile is what actually avoids Google's
/sorry bot-detection page, so this is the reliable way to get browser
Google search in plain text turns. Fallback order is now CDP (broadcast
Chrome) -> persistent profile (when configured) -> ephemeral headless,
all still fail-open to the DDG/Brave/Wikipedia cascade.

Document the profile in .env.example and web_search.spec.md.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-22 20:57:25 +09:00
javis-bot
98a1825d01 feat: headless Chrome fallback for browser web search outside broadcast
browse-search.mjs only connected to the on-screen broadcast Chrome over
CDP, so browser-based Google search worked only during a live broadcast;
plain text turns fell through to the DDG cascade. Add a headless fallback
(system Chrome via channel:'chrome', else Playwright's bundled chromium)
for `search` mode so general conversation can use Google at no API cost.
`youtube` still requires the visible broadcast Chrome.

Detect Google's /sorry bot-detection interstitial structurally by URL and
fail fast so the caller fail-opens to DDG/Brave/Wikipedia instead of
treating the challenge page as empty results.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-22 20:52:54 +09:00
javis-bot
5b6a67963a feat: make GEMINI_AUTH=oauth authenticate in Docker
Some checks failed
Release / semantic-release (push) Successful in 25s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m17s
Release / build-linux (push) Failing after 7m8s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled
OAuth cannot be done interactively in the headless container, so the login
must be seeded into the mounted ~/.gemini. Three problems are fixed:

- Mount fragility on the Windows Docker Desktop target: the creds mount
  defaulted to ${HOME}/.config/javis/gemini, but ${HOME} is often unset when
  compose runs outside a WSL shell, silently mounting the wrong dir. Default is
  now the project-local ./docker/gemini-oauth (cross-platform), GEMINI_OAUTH_DIR
  still overrides.
- No visibility: when oauth is selected but no login is seeded, the path
  silently degraded to DDG/Brave. Added gemini_oauth_ready() + a one-time debug
  hint and a startup entrypoint warning (skipped on the browser role, fail-open).
- Seeding guidance: oauth_creds.json is the essential credential (refresh token;
  GOOGLE_GENAI_USE_GCA=true forces OAuth), which is what the readiness check and
  warning verify; docs recommend copying the whole ~/.gemini for convenience.

Adds docker/gemini-oauth/ seed dir (.gitkeep) with the login files gitignored,
GEMINI_OAUTH_DIR in .env.example, and updates DEPLOY.md, stream_browser_modes.md
and llm_contexts.md. Covered by 3 new tests (10 passed total).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-22 18:05:22 +09:00
javis-bot
7870a76314 fix: persona uses settings output_language, matching the reply directive
The persona prompt was built from the raw OUTPUT_LANGUAGE env while the
reply-language directive read the settings-web UI value (config JSON), so
changing the language in the settings page was honoured by the directive
but ignored by the persona, leaving them contradicting each other.

Add _resolve_output_language() as the single source of truth (config wins
over env) and feed the same resolved value to both build_system_prompt()
and reply_language_directive(). Update docs/llm_contexts.md to match.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-16 19:47:26 +09:00
javis-bot
b3088dd75f fix: settings output_language overrides the compose env default
The settings-UI output_language was ignored because the OUTPUT_LANGUAGE env took
precedence. Read the config value first, fall back to env, so changing the
language in /settings actually applies.
2026-06-15 16:57:01 +09:00
javis-bot
84e435f916 feat: settings web UI (models / STT / TTS speed / language / LLM instructions)
Adds /settings (served by the bridge) to change the LLM model (from installed
Ollama models), Whisper model, TTS engine + MeloTTS speed, output language,
agentic max-turns, thinking mode, and free-form LLM instructions — live, with a
'apply' that restarts the bridge + TTS worker. Settings persist to the runtime
config JSON; engine reads output_language + llm_instructions and the TTS worker
reads melo_speed from it. Bridge port publishable for access.
2026-06-15 13:05:46 +09:00