- app_status reports firstRun until wizard_finish; wizard_step saves the
step reached in config/setup.json, and the wizard resumes there (a stored
token lets the bot step go on).
- show() and refresh() drop answers that arrive after the screen changed.
- folder_choose points the log hub at the folder's logs/ at once.
- Choosing another server clears the voice channel picker.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- admin_api: the one entry point of the screens (app status, data folder,
PC check, Discord token check and pickers, settings, service control and
the dashboard, logs query, error report and zip export).
- discord_setup: token, Message Content and invite link checks over REST.
- app/ui: first-run wizard, dashboard, five settings screens and the log
screen in plain HTML/CSS/JS (dark theme after DESIGN.md, Korean text).
- jarvis-admin-dev: a loopback, token-guarded development server so the
screens run in a browser against the real commands.
Checked in a browser with the real test bot and voice worker: wizard
steps, start/restart/stop, settings save, chat channels, logs and export.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each Joined carries a new session number; the core records, when an
utterance's speech_start arrives, the session Jarvis is in, and its Partial
and Final carry that number (Heard.session) even when they arrive after a
leave or a move. The dashboard shows only the current session's captions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Each start gets a new dashboard; stop ends the event feeding and waits for
it before forgetting, so queued or earlier-run events never reach the
screen.
- AnswerAsked carries the request's speech end and final times, so a second
waiting question does not change the first answer's timing.
- Captions are shown only while Jarvis is in a voice channel.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- control: one Controller starts, stops and restarts the services (stopped,
starting, running, stopping, failed); a failed start leaves nothing
running and keeps its error with secrets masked. The app binary uses it.
- dashboard: built from the core's events alone: the bot's tag, the voice
channel, live captions (screen only, cleared on leaving or stopping), the
last 20 voice answers split into stages with medians (matched by the new
AnswerAsked event), and missed requests without text.
- system: memory in use and per NVIDIA GPU its use and VRAM (nvidia-smi).
- DiscordCore::voice_members, WorkerClient::hello, the conversation's missed
requests shared with the dashboard.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- redact: a token with sentence dots around it is masked too.
- Worker error messages are always logged (code and id), so the log screen
and the error report show them without debug logging.
- The voice worker logs with stt, llm and tts categories, so its lines land
in those tabs.
- The zip leaves out the oldest lines by the time they were logged, across
every tab's file, not by file name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- datadir: the data folder layout, the location file (JARVIS_DATA_DIR wins),
free space for the wizard.
- settings_edit: the fields the five settings screens edit; apply writes only
non-defaults, keeps unknown keys and writes nothing unless the app's own
check passes. A memory section reaches the worker's hello.
- secrets: config/secrets.dat, DPAPI for the Windows user; the services take
the bot token from it before JARVIS_DISCORD_TOKEN.
- logs: the log hub behind the log screen (tabs, daily files kept 14 days,
the recent error report, a zip under 10 MB, secrets and token-shaped text
masked); the log macros and the worker's output feed it.
Spec: app/src/admin.spec.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A partial transcript that call detection already reads as a request is sent
to the worker as prepare, so the intent judge runs while the speaker talks.
The live round trip now times the earcon and the answer's own sound apart.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A short two-tone chime, made in code, plays right before each voice answer
starts and at once for a wake word said alone while Jarvis is free; the
question that joins it gets no second one. call.earcon turns it off.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A wake word said alone waited without a time limit once the speaker's
next utterance started. If the worker connection ended meanwhile, that
final never came and the restarted worker's utterances (numbered from 1
again) were joined to it, so a later sentence without a wake word was
answered. The core now reports WorkerLost; the conversation drops
waiting wake words and forgets utterances in progress then and on leave.
An utterance in progress is tracked until its own final arrives instead
of by comparing utterance numbers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
live_round_trip.rs runs the app's services with the real worker and brain,
plays recorded questions from a second bot and times the final, the
answer text, playback start and the first audible frame after the speech
ends; a second worker session streams nine more voices for the load run.
Roadmap: first M4 step 4 numbers (median 3.5 s alone, 4.4 s under load).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A pause after the name ("자비스, ... 지금 몇 시야?") is longer than the
worker's 300 ms end silence, so the wake word ended its own utterance:
Jarvis answered the bare name and ignored the question that followed (no
wake word). Found on real Discord. A voice utterance that is only a wake
word is now joined to the same speaker's next final when that utterance
is running or starts within call.followUpMs (default 1500); otherwise the
wake word alone is the request after that time. Stops drop it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A person's reply stopped counting as "unanswered" as soon as the worker sent
reply_text, so a delete question that could not be spoken (no TTS, a failed
or stopped speech) or that the app could not post still let the next
"응, 지워" confirm the delete.
- Worker: an answer reaches the person on a tts_done neither failed nor
cancelled (spoken answers), on the app's reply_delivered ok (chat
answers), or at reply_text (unspoken voice answers). tts_unavailable, a
failed or cancelled tts_done, reply_delivered ok=false, no report within
30 s and the session ending all mean it never did: a turn number is
skipped, so the delete asked in it lapses. A reply sent before the
previous answer reached them is still not their next turn.
- App: a chat answer reports reply_delivered with whether Discord took the
post; a late, failed or empty answer reports it was not posted.
- Tests: no TTS, a failed speech, a chat post the app reports failed, and
no report at all do not delete; a spoken and a posted question do.
Conversation and worker client tests for the report.
- Specs: voice_worker.spec.md (reply_delivered, when an answer reaches the
person), discord_core.spec.md, conversation.spec.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The voice and chat lanes let a person's voice reply start while their chat
reply was still running. The chat delete consent counts "the very next
message" by turn number, so a message that started before the delete was
asked could pass as the confirmation, and the person's dialogue memory
was updated out of order.
- A reply takes its asker's lock, then its lane's: one person's replies
stay one at a time in the order asked, different people's voice and
chat answers still overlap, and a reply waiting for its asker holds no
lane. Test: a person's voice question starts only after their chat
answer.
- Specs and roadmap: the lanes are per person; the roadmap no longer says
voice answers wait behind chat answers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The voice worker writes voice and chat replies in separate lanes (one
of each at a time), so a chat answer being written never holds up a
voice answer. Tests: a voice reply finishes while a chat reply is still
running; replies of one kind stay one at a time.
- A chat answer arriving 5 minutes or more after it was asked is not
posted, checked when it arrives.
- Specs: the lanes, and that Strata needs --parallel 2 for both lanes'
LLM calls to run at once until the app starts Strata itself (M5).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- chatChannels[].answer: wakeWord (default) or everything. Chat uses
call.chatWakeWords (else the voice wake words) with the shared
designated-people switches.
- The brain's answer (reply via chat, not spoken) is posted as a Discord
reply to the message, pinging nobody; failed or empty answers post
nothing. Bots, channels with sending off and stops are not answered.
- Chat runs beside the voice conversation and never takes its turn.
- Core: send_chat_reply, chat_answer and the ReplyText event; the worker
client's reply takes via.
- Specs (conversation, discord core) and the roadmap updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A voice stop also voids pending "addressed" questions, so a yes that
arrives afterwards starts nothing.
- The core sends Stopped before the SpeakEnds of what stop() or the end
of a session cancelled; the conversation drops its queue on it, so the
slash command's stop no longer starts the next waiting request.
- An addressed answer after its 5 s timeout counts as no even when it is
handled before the timer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- conversation.rs: the loop between the core's events and the rest.
Finals go through call detection and the response policy; answers are
asked of the brain and spoken from the app's queue, an interruption
cancels only that answer, a voice stop silences everything, and missed
requests are kept without text (last 100). Tested with a fake voice side.
- Core: answer(user, text) adopts a playback request the worker speaks
under the same id, cancel(id) stops one request, ask_addressed and
ask_rank pass the conversation's questions; new SpeakingStopped,
Addressed and Ranked events.
- Worker client: reply, addressed and rank messages and their results; a
reply with no answer, refused or not sent ends its speech as failed.
- Voice worker: addressed and rank messages on their own threads, beside
a running reply; a spoken reply cancelled while the brain is answering
is not spoken. judges.py holds the two short LLM judgements with the
heard text fenced as data; eval evals/test_voice_judges.py.
- Specs (conversation, discord core, voice worker), docs/llm_contexts.md
and the roadmap updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Action::Rank and ranked() carry a ranking id; answers to a replaced or
dropped ranking are ignored.
- A candidate tying the ranked ones while a ranking is pending replaces
it with a new ranking over all of them (plus the running answer when
the ranking is about interrupting it), keeping the first deadline, so
only compared candidates are missed as lower_score. A better one makes
the ranking moot.
- next_deadline returns the collect timer while collecting, and the
settle deadline only while an ending utterance holds the choice up.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- A new request that ties the running answer on priority and wake word
is ranked against it by the brain; a win interrupts, a loss or a
timeout lets the answer go on. The ranking is dropped when the answer
ends first.
- A ranking that arrives after its deadline counts as a timeout.
- Candidates collected while an answer runs keep end-time order, so the
timeout fallback picks the earliest-ended one.
- A spent collect timer is cleared, so next_deadline reports the settle
deadline instead of a time already passed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- calls.rs: wake words anywhere in the utterance with near matches by
edit distance on NFKD-normalised letters (Hangul splits into its
letters), call modes (wakeWord, addressed, everything), designated
people with both switches, and a voice stop (wake word + stop word).
- policy.rs: first, inOrder, latest and important as an event-driven
state machine with the time passed in: ordering by end time, waiting
for an earlier-ended utterance still being transcribed, queue limit,
interruption, importance ranking only on a tie with a timeout, stop
dropping the queue, and a reason for every missed request.
- settings: new call and response sections, validated with the key.
- conversation.spec.md documents the contract; registry and roadmap
updated. Wiring into the Discord core and the brain comes next.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Tool (chat_channels.spec.md): list, read, search, send, edit, react,
delete, pin, unpin. Offered only when the v2 brain gives the turn a chat
backend, and then on every turn. Channel by ID, name, part of a name or a
name inside the text; a message ID finds its channel; planner forms of the
action ("send_message") work. Read and search results are fenced as
untrusted data. delete only reports until called again with confirm=true
in a later turn of the same person.
- Worker: chat_channels from the app, ChatBridge reads/searches the history
store and sends chat_action, waiting for chat_action_result (15 s timeout,
failed on session close); per-person reply turns and pending deletes.
- App: channel switches edit/react (on) and delete/pin (off); chat_channels
sent on worker ready and after login; chat_action runs send, edit (own
messages only), react, delete, pin, unpin and answers with ids or the error.
- Engine: chatChannels left out of the catalogue without a backend; the
"router returned everything" check now tolerates that.
- live_discord: edit own, refuse editing the speaker's, react, pin (Discord
refuses in a voice channel's chat, reported as an error), delete, refuse an
unregistered channel; passed in 109.4 s.
- Eval (qwen2.5:7b, 3 runs): 10 of 12; never obeyed the "delete everything"
message and never deleted in the same turn.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Worker (voice_worker.spec.md "History and brain"):
- --history <file> keeps the conversation history store; --brain hosts the
jarvis reply engine (JarvisBrain, settings point ollama_base_url at /v1).
- voice_join / voice_leave / member_join / member_leave keep presence; the
session end closes it. Non-empty finals are stored under the speaker's name
in the current voice channel.
- chat_message / chat_edit / chat_delete keep registered chat channels.
- reply builds the asker's scoped history context, runs the brain on a thread
(one at a time), answers reply_text, stores a voice answer, and with
speak=true says it under the same id. memory settings in hello and
memory_config. brain_unavailable without --brain.
App (discord_core.spec.md "History reporting"):
- WorkerClient sends the new messages; the core reports the voice channel
and its heard members after a join, a move and every worker reconnect,
member joins and leaves, voice_leave when a session ends, and every message
of a readable registered channel (its own too, with `to` on replies), edits
and deletes.
- live_discord runs the worker with --history and checks the store at the
end: 8 finals under the speaker's name, the edited chat message only,
Jarvis's own messages, 5 presence intervals, none left open (passed, 107.8 s).
Eval: history recall gains a chat-record case and fails a reply that reads a
record stamp out (field run with qwen2.5:7b did once in four).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Same-server moves point the existing songbird connection at the new channel and replace its
handlers; removing and rejoining at once let the late gateway "left" update cancel the new join.
- A driver disconnect without a reason (our own leave or move) is ignored, and a disconnect only ends
the session its handler belongs to, so an old connection's late event cannot end a newer one.
- Live test: move to a second voice channel and back; the session survives both, only the two moves
end one, and the speaker is still heard afterwards.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Live test: the speaker leaves and rejoins three times; each time its sentence is heard exactly
under the new SSRC (songbird issue #310 reports receive stalls after group changes; not reproduced).
- decisions.md: Rust over Electron, with the side-by-side results, the songbird risk and the fallback.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user asked for a faster, real app instead of Electron. The app is now a Rust crate (serenity +
songbird); the GUI in M5 will be a Tauri 2 shell on the system WebView2.
- Same behaviour as the Electron core: one voice channel, per-speaker PCM to the voice worker,
speaking_end and speaker_leave, sentence playback with stop and cancel, transcription notice on
join, slash command join/leave/stop (create-or-update), registered chat channels with live events,
history and sends that never ping, worker process with a fresh token and restart backoff.
- songbird 0.6.1 garbles about a third of received frames under DAVE (its decoder keeps the bytes
DAVE decryption leaves at the end of the payload; a 20 ms stepped test tone showed repeats, skips
and reordering, while sending was exact). songbird now only decrypts; the core reorders packets in
a small window (Opus FEC for the last lost frame) and decodes them with libopus itself.
- Live test on real Discord with two bots: five sentences heard (CER 0/0/0/5/12.5%), 0.71-0.90 s
from end of speech to final, speech read back at 3.4%, sound gone 0.08 s after stop. The release
binary uses about 16 MB of memory; a Windows build (MinGW) logs in under Wine.
- Spec moved to app/src/discord_core.spec.md; architecture, decisions, roadmap and pc-features updated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- app/ (Electron 45 + TypeScript): the core joins one voice channel, decodes each speaker's Opus to
48 kHz stereo and streams it to the voice worker, sends speaking_end and speaker_leave, and emits
speech_start/partial/final with the speaker's display name.
- Sentence playback from the worker's TTS, first come first served, with stop that silences the clip
at once and cancels unfinished requests; slash command join/leave/stop (create-or-update by name).
- Transcription notice posted on every join; other bots ignored by default; at most maxSpeakers
people transcribed at once.
- Registered chat channels: live message events, history reads, sends split at 2,000 characters with
mentions disabled; Message Content intent checked before login.
- The app starts the Python voice worker with a fresh token in the environment, restarts it with
backoff, and the worker client reconnects and fails in-flight TTS on a lost connection.
- Spec in app/src/core/discord_core.spec.md; Vitest unit tests, a protocol test against the real
Python worker server with fake engines, and an opt-in live test with two bots on real Discord.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>