feat(app): M5 step 2, service control and the dashboard data

- control: one Controller starts, stops and restarts the services (stopped,
  starting, running, stopping, failed); a failed start leaves nothing
  running and keeps its error with secrets masked. The app binary uses it.
- dashboard: built from the core's events alone: the bot's tag, the voice
  channel, live captions (screen only, cleared on leaving or stopping), the
  last 20 voice answers split into stages with medians (matched by the new
  AnswerAsked event), and missed requests without text.
- system: memory in use and per NVIDIA GPU its use and VRAM (nvidia-smi).
- DiscordCore::voice_members, WorkerClient::hello, the conversation's missed
  requests shared with the dashboard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-12 13:49:51 +09:00
parent eb0fa2f159
commit 0973ad9815
13 changed files with 799 additions and 36 deletions

View File

@@ -88,6 +88,39 @@ STT, LLM and TTS lines go to `stt`, `llm`, `tts` by their worker category), `bro
- Errors the voice worker reports to the app (`error` messages: code and request id) are always
logged, debug logging or not, so the log screen and the error report show them.
## Service control (`app/src/control.rs`)
The dashboard's start and stop buttons, and the app binary itself, run the services through one
`Controller` for the data folder: states `stopped`, `starting`, `running`, `stopping`, `failed`
(with the error, secrets masked). `start` from stopped or failed only, `stop` from running only;
`restart` is stop then start. A start that fails leaves nothing running. Starting reads the
settings and the stored token again, so saved changes take effect on the next start.
## Dashboard data (`app/src/dashboard.rs`)
`Controller::snapshot()` is everything the dashboard shows, as one JSON-ready value:
- `state` and `error` (above); `discord`: the bot's tag once connected; `worker`: connected, STT
and TTS ready (from the worker's `hello_ok`), restarts of its process.
- `voice`: the voice channel Jarvis is in and the people there (ID and display name), as the core
reports them to the worker.
- `captions`: live captions, the last 30 lines: a speaker's partial transcript is replaced by
their next partial and then by the final. On the screen only: never logged, never written,
cleared when Jarvis leaves the voice channel or the services stop.
- `latency`: the last 20 voice answers, each split into its stages: speech end to final transcript,
the brain (answer asked to answer text), the voice (answer text to the first sound) and the
total; with the median of each. An answer is matched to its question by the core's
`AnswerAsked { id, user }` event (the speech id and the asker) and that user's latest final.
- `missed`: the conversation's missed requests (`conversation.spec.md`), newest first, at most 20:
who, why and how long ago, never the text.
- `system`: memory in use and total, and per NVIDIA GPU its use and VRAM (from `nvidia-smi`,
missing when there is none); sampled when the snapshot is taken, at most once a second.
- `performance`: the performance stage (normal, gaming, paused, and why) comes with game
detection in M7; until then `null`.
The dashboard's state is built from the core's events alone (`Dashboard::on_event` with the time),
so it is tested with made-up events.
## Tests
`cargo test`: data folder layout, location file and the environment override, a refused relative
@@ -95,5 +128,7 @@ path, free space; settings view with defaults, apply writing only non-defaults,
back to its default, keeping unknown keys, refusing a bad value or a result `parse_settings` refuses
without touching the file; secrets round trip, file permissions on Unix, masking; log tabs from
categories, query filters and follow, files per day and retention, error report bounds, zip size
cap, redaction of stored secrets and token-shaped text. Windows-only paths (DPAPI, `%APPDATA%`) are
cap, redaction of stored secrets and token-shaped text; the controller's state rules with a fake
service starter; the dashboard from made-up events (captions replaced and cleared, latency stages
and medians, an answer matched to its asker's final, missed requests without text). Windows-only paths (DPAPI, `%APPDATA%`) are
checked on the Windows VM in the Windows build step.

321
app/src/control.rs Normal file
View File

@@ -0,0 +1,321 @@
//! Starting and stopping the services, and the dashboard snapshot (`app/src/admin.spec.md`,
//! "Service control" and "Dashboard data").
use std::path::PathBuf;
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
use async_trait::async_trait;
use serde_json::{json, Value};
use tokio::sync::mpsc;
use crate::dashboard::Dashboard;
use crate::discord::CoreEvent;
use crate::system::Usage;
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
#[serde(rename_all = "lowercase")]
pub enum State {
Stopped,
Starting,
Running,
Stopping,
Failed,
}
/// The running services as the controller sees them.
#[async_trait]
pub trait Running: Send + Sync + 'static {
async fn stop(&self);
/// Worker, voice channel members and missed requests, for the snapshot.
fn status(&self, now: Instant) -> Value;
}
/// What starts the services (the real ones, or a fake in tests).
#[async_trait]
pub trait Host: Send + Sync + 'static {
async fn start(&self) -> Result<(Arc<dyn Running>, mpsc::UnboundedReceiver<CoreEvent>), String>;
/// Stored secrets, masked in error messages.
fn secrets(&self) -> Vec<String>;
}
struct Inner {
state: State,
error: Option<String>,
running: Option<Arc<dyn Running>>,
}
pub struct Controller {
host: Box<dyn Host>,
inner: Mutex<Inner>,
dashboard: Arc<Mutex<Dashboard>>,
usage: tokio::sync::Mutex<Option<(Instant, Usage)>>,
}
impl Controller {
pub fn new(host: impl Host) -> Arc<Self> {
Arc::new(Self { host: Box::new(host), inner: Mutex::new(Inner { state: State::Stopped, error: None, running: None }),
dashboard: Arc::default(), usage: tokio::sync::Mutex::new(None) })
}
pub fn state(&self) -> State {
self.inner.lock().unwrap().state
}
/// Start from stopped or failed; a failed start leaves nothing running.
pub async fn start(&self) -> Result<(), String> {
{
let mut i = self.inner.lock().unwrap();
if !matches!(i.state, State::Stopped | State::Failed) {
return Err(format!("cannot start while {:?}", i.state).to_lowercase());
}
i.state = State::Starting;
i.error = None;
}
crate::info_log!("app", "starting the services");
match self.host.start().await {
Ok((running, mut events)) => {
let dash = self.dashboard.clone();
tokio::spawn(async move {
while let Some(ev) = events.recv().await {
dash.lock().unwrap().on_event(&ev, Instant::now());
}
});
let mut i = self.inner.lock().unwrap();
i.state = State::Running;
i.running = Some(running);
Ok(())
}
Err(e) => {
let e = crate::logs::redact(&e, &self.host.secrets());
crate::error_log!("app", "the services did not start: {e}");
let mut i = self.inner.lock().unwrap();
i.state = State::Failed;
i.error = Some(e.clone());
Err(e)
}
}
}
/// Stop from running; what was said is forgotten.
pub async fn stop(&self) -> Result<(), String> {
let running = {
let mut i = self.inner.lock().unwrap();
if i.state != State::Running {
return Err(format!("cannot stop while {:?}", i.state).to_lowercase());
}
i.state = State::Stopping;
i.running.take()
};
if let Some(r) = running {
r.stop().await;
}
self.dashboard.lock().unwrap().clear();
self.inner.lock().unwrap().state = State::Stopped;
crate::info_log!("app", "services stopped");
Ok(())
}
pub async fn restart(&self) -> Result<(), String> {
if self.state() == State::Running {
self.stop().await?;
}
self.start().await
}
/// Everything the dashboard shows.
pub async fn snapshot(&self) -> Value {
let now = Instant::now();
let (state, error, running) = {
let i = self.inner.lock().unwrap();
(i.state, i.error.clone(), i.running.clone())
};
let status = running.map(|r| r.status(now)).unwrap_or_else(|| json!({"worker": null, "voiceMembers": [],
"missed": []}));
let mut view = self.dashboard.lock().unwrap().view(&[], now);
view["missed"] = status["missed"].clone();
view["worker"] = status["worker"].clone();
view["voiceMembers"] = status["voiceMembers"].clone();
view["state"] = json!(state);
view["error"] = json!(error);
view["system"] = json!(self.system().await);
view["performance"] = Value::Null; // game detection comes in M7
view
}
/// Memory and GPU use, sampled at most once a second.
async fn system(&self) -> Usage {
let mut cache = self.usage.lock().await;
if let Some((at, u)) = cache.as_ref() {
if at.elapsed() < Duration::from_secs(1) {
return u.clone();
}
}
let u = crate::system::usage().await;
*cache = Some((Instant::now(), u.clone()));
u
}
}
/// The real services from a data folder.
pub struct ServicesHost {
pub data_dir: PathBuf,
}
#[async_trait]
impl Host for ServicesHost {
async fn start(&self) -> Result<(Arc<dyn Running>, mpsc::UnboundedReceiver<CoreEvent>), String> {
let mut s = crate::services::start_services(&self.data_dir).await.map_err(|e| e.to_string())?;
let (tx, rx) = mpsc::unbounded_channel();
let mut events = std::mem::replace(&mut s.events, mpsc::unbounded_channel().1);
tokio::spawn(async move {
while let Some(e) = events.recv().await {
if tx.send(e).is_err() {
break;
}
}
});
Ok((Arc::new(s), rx))
}
fn secrets(&self) -> Vec<String> {
crate::secrets::Secrets::new(&self.data_dir.join("config").join("secrets.dat")).values()
}
}
#[async_trait]
impl Running for crate::services::Services {
async fn stop(&self) {
crate::services::Services::stop(self).await;
}
fn status(&self, now: Instant) -> Value {
let hello = self.worker.hello();
let missed: Vec<_> = self.missed.lock().unwrap().iter().cloned().collect();
json!({
"worker": {"connected": hello.is_some(), "sttReady": hello.as_ref().map(|h| h.stt_ready),
"ttsReady": hello.as_ref().map(|h| h.tts_ready), "restarts": self.worker_process.restarts()},
"voiceMembers": self.core.voice_members().into_iter()
.map(|(id, name)| json!({"id": id.to_string(), "name": name})).collect::<Vec<_>>(),
"missed": crate::dashboard::missed_view(&missed, now),
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::atomic::{AtomicU32, Ordering};
#[derive(Default)]
struct Counts {
starts: AtomicU32,
stops: AtomicU32,
}
struct FakeRunning(Arc<Counts>);
#[async_trait]
impl Running for FakeRunning {
async fn stop(&self) {
self.0.stops.fetch_add(1, Ordering::SeqCst);
}
fn status(&self, _: Instant) -> Value {
json!({"worker": {"connected": true}, "voiceMembers": [{"id": "11", "name": "민수"}], "missed": []})
}
}
struct FakeHost {
counts: Arc<Counts>,
fail_first: AtomicU32,
events: Mutex<Option<mpsc::UnboundedSender<CoreEvent>>>,
}
#[async_trait]
impl Host for FakeHost {
async fn start(&self) -> Result<(Arc<dyn Running>, mpsc::UnboundedReceiver<CoreEvent>), String> {
self.counts.starts.fetch_add(1, Ordering::SeqCst);
tokio::time::sleep(Duration::from_millis(20)).await;
if self.fail_first.load(Ordering::SeqCst) > 0 {
self.fail_first.fetch_sub(1, Ordering::SeqCst);
return Err("login failed for token Mt0kenSecretValue".into());
}
let (tx, rx) = mpsc::unbounded_channel();
*self.events.lock().unwrap() = Some(tx);
Ok((Arc::new(FakeRunning(self.counts.clone())), rx))
}
fn secrets(&self) -> Vec<String> {
vec!["Mt0kenSecretValue".into()]
}
}
fn controller(fail_first: u32) -> (Arc<Controller>, Arc<Counts>, Arc<FakeHost>) {
let counts = Arc::new(Counts::default());
let host = Arc::new(FakeHost { counts: counts.clone(), fail_first: AtomicU32::new(fail_first),
events: Mutex::new(None) });
struct Shared(Arc<FakeHost>);
#[async_trait]
impl Host for Shared {
async fn start(&self) -> Result<(Arc<dyn Running>, mpsc::UnboundedReceiver<CoreEvent>), String> {
self.0.start().await
}
fn secrets(&self) -> Vec<String> {
self.0.secrets()
}
}
(Controller::new(Shared(host.clone())), counts, host)
}
#[tokio::test]
async fn start_and_stop_follow_the_state_rules() {
let (c, counts, _) = controller(0);
assert!(c.stop().await.is_err(), "nothing to stop");
let c2 = c.clone();
let first = tokio::spawn(async move { c2.start().await });
tokio::time::sleep(Duration::from_millis(5)).await;
assert_eq!(c.state(), State::Starting);
assert!(c.start().await.unwrap_err().contains("starting"), "one start at a time");
first.await.unwrap().unwrap();
assert_eq!(c.state(), State::Running);
assert!(c.start().await.is_err());
c.restart().await.unwrap();
assert_eq!((counts.starts.load(Ordering::SeqCst), counts.stops.load(Ordering::SeqCst)), (2, 1));
c.stop().await.unwrap();
assert_eq!(c.state(), State::Stopped);
}
#[tokio::test]
async fn a_failed_start_is_reported_masked_and_can_be_retried() {
let (c, counts, _) = controller(1);
let e = c.start().await.unwrap_err();
assert_eq!(e, "login failed for token [비밀값]");
let snap = c.snapshot().await;
assert_eq!((snap["state"].clone(), snap["error"].clone()), (json!("failed"), json!(e)));
c.start().await.unwrap();
assert_eq!(c.state(), State::Running);
assert_eq!(counts.stops.load(Ordering::SeqCst), 0, "a failed start leaves nothing to stop");
}
#[tokio::test]
async fn the_snapshot_carries_the_events_and_stopping_forgets_what_was_said() {
let (c, _, host) = controller(0);
c.start().await.unwrap();
let tx = host.events.lock().unwrap().clone().unwrap();
tx.send(CoreEvent::Ready { user_id: 1, tag: "Jarvis#0001".into() }).unwrap();
tx.send(CoreEvent::Joined { guild_id: 1, channel_id: 77 }).unwrap();
tx.send(CoreEvent::Partial(crate::discord::Heard { speaker_id: 11, speaker_name: "민수".into(), utt: 1 },
"자비스 안녕".into())).unwrap();
tokio::time::sleep(Duration::from_millis(50)).await;
let s = c.snapshot().await;
assert_eq!(s["state"], json!("running"));
assert_eq!(s["discord"], json!("Jarvis#0001"));
assert_eq!(s["voiceChannel"], json!("77"));
assert_eq!(s["captions"][0]["text"], json!("자비스 안녕"));
assert_eq!(s["voiceMembers"][0]["name"], json!("민수"));
assert_eq!(s["worker"]["connected"], json!(true));
assert!(s["system"]["ramTotalMb"].is_u64() || !cfg!(any(target_os = "linux", windows)));
assert_eq!(s["performance"], Value::Null);
c.stop().await.unwrap();
let s = c.snapshot().await;
assert_eq!(s["captions"], json!([]));
assert_eq!(s["worker"], Value::Null);
}
}

View File

@@ -117,6 +117,9 @@ impl ChatCalls {
/// Chat answers waited for longer than this are forgotten (the brain or the worker went away).
const CHAT_ANSWER_KEPT: Duration = Duration::from_secs(300);
/// The missed requests, shared between the conversation loop and the dashboard.
pub type MissedLog = std::sync::Arc<std::sync::Mutex<VecDeque<Missed>>>;
/// A request that was not answered, or an answer that was stopped. No text: only who, why and when.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Missed {
@@ -136,7 +139,8 @@ pub struct Conversation<V: Voice> {
by_speech: HashMap<String, u64>,
/// Ask id -> the request it becomes on a yes, and when it was asked.
asks: HashMap<String, (Request, Instant)>,
missed: VecDeque<Missed>,
/// Shared with the dashboard (`missed_handle`).
missed: MissedLog,
chat: ChatCalls,
/// Chat answer id -> the message it answers (channel, message) and when it was asked.
chat_answers: HashMap<String, (u64, u64, Instant)>,
@@ -152,7 +156,7 @@ pub struct Conversation<V: Voice> {
impl<V: Voice> Conversation<V> {
pub fn new(voice: V, calls: CallSettings, policy: PolicySettings) -> Self {
Self { voice, calls, policy: Policy::new(policy), next_id: 0, answers: HashMap::new(),
by_speech: HashMap::new(), asks: HashMap::new(), missed: VecDeque::new(), chat: ChatCalls::default(),
by_speech: HashMap::new(), asks: HashMap::new(), missed: MissedLog::default(), chat: ChatCalls::default(),
chat_answers: HashMap::new(), started: HashMap::new(), wakes: HashMap::new(),
chimed: HashSet::new() }
}
@@ -206,8 +210,14 @@ impl<V: Voice> Conversation<V> {
}
}
pub fn missed(&self) -> impl Iterator<Item = &Missed> {
self.missed.iter()
/// The missed requests kept, oldest first.
pub fn missed(&self) -> std::vec::IntoIter<Missed> {
self.missed.lock().unwrap().iter().cloned().collect::<Vec<_>>().into_iter()
}
/// The same records for the dashboard, readable while the loop runs.
pub fn missed_handle(&self) -> MissedLog {
self.missed.clone()
}
pub fn on_event(&mut self, ev: &CoreEvent, now: Instant) {
@@ -413,10 +423,11 @@ impl<V: Voice> Conversation<V> {
Action::Missed(r, reason) => {
self.chimed.remove(&r.id);
debug_log!("conversation", "missed req={} speaker={} reason={reason:?}", r.id, r.speaker);
if self.missed.len() == MISSED_KEPT {
self.missed.pop_front();
let mut missed = self.missed.lock().unwrap();
if missed.len() == MISSED_KEPT {
missed.pop_front();
}
self.missed.push_back(Missed { speaker: r.speaker, reason, at: now, chars: r.text.chars().count() });
missed.push_back(Missed { speaker: r.speaker, reason, at: now, chars: r.text.chars().count() });
}
Action::Rank { id, candidates } => {
let c: Vec<(String, u64, String)> =
@@ -724,7 +735,7 @@ mod tests {
c.on_event(&fin(A, "자비스 날씨", 100), ms(t, 0));
c.on_event(&fin(B, "자비스 안녕", 100), ms(t, 100));
assert_eq!(f.take().len(), 1);
let m: Vec<_> = c.missed().cloned().collect();
let m: Vec<_> = c.missed().collect();
assert_eq!(m, vec![Missed { speaker: B, reason: MissReason::Busy, at: ms(t, 100), chars: "자비스 안녕".chars().count() }]);
}

277
app/src/dashboard.rs Normal file
View File

@@ -0,0 +1,277 @@
//! What the dashboard shows, built from the core's events (`app/src/admin.spec.md`, "Dashboard data").
//!
//! Pure: `on_event` takes the time, so every rule is tested with made-up events. Captions hold what
//! people said for the screen only; nothing here is logged or written.
use std::collections::{HashMap, VecDeque};
use std::time::{Duration, Instant};
use serde::Serialize;
use serde_json::{json, Value};
use crate::conversation::Missed;
use crate::discord::CoreEvent;
pub const CAPTIONS_KEPT: usize = 30;
pub const ANSWERS_KEPT: usize = 20;
pub const MISSED_SHOWN: usize = 20;
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct Caption {
pub speaker: String,
pub name: String,
pub text: String,
#[serde(rename = "final")]
pub is_final: bool,
#[serde(skip)]
utt: u32,
}
/// One voice answer's stages in milliseconds (None: that stage could not be seen).
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Default)]
#[serde(rename_all = "camelCase")]
pub struct AnswerTiming {
/// Speech end to its final transcript.
pub final_ms: Option<u64>,
/// The answer asked to its text.
pub brain_ms: Option<u64>,
/// The text to the first sound of the answer.
pub voice_ms: Option<u64>,
/// Speech end to the first sound.
pub total_ms: Option<u64>,
}
struct Pending {
asked: Instant,
/// The asker's latest final: when their speech ended and when the final came.
heard: Option<(Instant, Instant)>,
reply: Option<Instant>,
}
#[derive(Default)]
pub struct Dashboard {
bot_tag: Option<String>,
voice_channel: Option<u64>,
captions: VecDeque<Caption>,
last_final: HashMap<u64, (Instant, Instant)>,
pending: HashMap<String, Pending>,
answers: VecDeque<AnswerTiming>,
}
fn ms(from: Instant, to: Instant) -> u64 {
to.saturating_duration_since(from).as_millis() as u64
}
impl Dashboard {
pub fn on_event(&mut self, ev: &CoreEvent, now: Instant) {
match ev {
CoreEvent::Ready { tag, .. } => self.bot_tag = Some(tag.clone()),
CoreEvent::Joined { channel_id, .. } => self.voice_channel = Some(*channel_id),
CoreEvent::Left { .. } => {
self.voice_channel = None;
self.captions.clear();
self.pending.clear();
}
CoreEvent::Partial(h, text) => self.caption(h.speaker_id, &h.speaker_name, h.utt, text, false),
CoreEvent::Final(f) => {
let h = &f.heard;
self.caption(h.speaker_id, &h.speaker_name, h.utt, &f.text, true);
if !f.text.trim().is_empty() {
let ended = now.checked_sub(Duration::from_millis(f.wait_ms)).unwrap_or(now);
self.last_final.insert(h.speaker_id, (ended, now));
}
}
CoreEvent::AnswerAsked { id, user } => {
let heard = self.last_final.get(user).copied();
self.pending.insert(id.clone(), Pending { asked: now, heard, reply: None });
}
CoreEvent::ReplyText { id, .. } => {
if let Some(p) = self.pending.get_mut(id) {
p.reply = Some(now);
}
}
CoreEvent::SpeakStart(id) => {
if let Some(p) = self.pending.remove(id) {
let t = AnswerTiming {
final_ms: p.heard.map(|(e, f)| ms(e, f)),
brain_ms: p.reply.map(|r| ms(p.asked, r)),
voice_ms: p.reply.map(|r| ms(r, now)),
total_ms: p.heard.map(|(e, _)| ms(e, now)),
};
if self.answers.len() == ANSWERS_KEPT {
self.answers.pop_front();
}
self.answers.push_back(t);
}
}
// An answer that ends without starting (failed, stopped) has nothing to time.
CoreEvent::SpeakEnd(e) => {
self.pending.remove(&e.id);
}
_ => {}
}
}
/// A partial replaces the same utterance's earlier caption; the final replaces the partial
/// (an empty final removes it).
fn caption(&mut self, speaker: u64, name: &str, utt: u32, text: &str, is_final: bool) {
let id = speaker.to_string();
let pos = self.captions.iter().position(|c| c.speaker == id && c.utt == utt && !c.is_final);
if let Some(i) = pos {
self.captions.remove(i);
}
if text.trim().is_empty() {
return;
}
if self.captions.len() == CAPTIONS_KEPT {
self.captions.pop_front();
}
self.captions.push_back(Caption { speaker: id, name: name.into(), text: text.into(), is_final, utt });
}
/// Forget what was said (the services stopped).
pub fn clear(&mut self) {
*self = Self::default();
}
pub fn captions(&self) -> Vec<Caption> {
self.captions.iter().cloned().collect()
}
pub fn answers(&self) -> Vec<AnswerTiming> {
self.answers.iter().copied().collect()
}
/// The median of each stage over the answers kept.
pub fn medians(&self) -> AnswerTiming {
fn median(mut v: Vec<u64>) -> Option<u64> {
v.sort_unstable();
(!v.is_empty()).then(|| v[v.len() / 2])
}
let pick = |f: fn(&AnswerTiming) -> Option<u64>| median(self.answers.iter().filter_map(f).collect());
AnswerTiming { final_ms: pick(|a| a.final_ms), brain_ms: pick(|a| a.brain_ms),
voice_ms: pick(|a| a.voice_ms), total_ms: pick(|a| a.total_ms) }
}
pub fn view(&self, missed: &[Missed], now: Instant) -> Value {
json!({
"discord": self.bot_tag,
"voiceChannel": self.voice_channel.map(|c| c.to_string()),
"captions": self.captions(),
"latency": {"answers": self.answers(), "median": self.medians()},
"missed": missed_view(missed, now),
})
}
}
/// Newest first, at most `MISSED_SHOWN`: who, why, how long ago and how long the request was.
pub fn missed_view(missed: &[Missed], now: Instant) -> Value {
Value::Array(missed.iter().rev().take(MISSED_SHOWN).map(|m| json!({
"speaker": m.speaker.to_string(),
"reason": m.reason,
"secondsAgo": now.saturating_duration_since(m.at).as_secs(),
"chars": m.chars,
})).collect())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::discord::{Heard, HeardFinal};
use crate::playback::PlaybackEnd;
use crate::policy::MissReason;
const A: u64 = 11;
fn heard(utt: u32) -> Heard {
Heard { speaker_id: A, speaker_name: "민수".into(), utt }
}
fn fin(utt: u32, text: &str, wait_ms: u64) -> CoreEvent {
CoreEvent::Final(HeardFinal { heard: heard(utt), text: text.into(), end_reason: "silence".into(),
audio_ms: 900, wait_ms, stt_ms: 200 })
}
fn at(t: Instant, ms: u64) -> Instant {
t + Duration::from_millis(ms)
}
#[test]
fn a_partial_is_replaced_by_the_next_and_then_by_the_final() {
let (mut d, t) = (Dashboard::default(), Instant::now());
d.on_event(&CoreEvent::Partial(heard(1), "자비스 오늘".into()), t);
d.on_event(&CoreEvent::Partial(heard(1), "자비스 오늘 날씨".into()), t);
assert_eq!(d.captions().iter().map(|c| (c.text.as_str(), c.is_final)).collect::<Vec<_>>(),
vec![("자비스 오늘 날씨", false)]);
d.on_event(&fin(1, "자비스 오늘 날씨 어때?", 700), t);
d.on_event(&CoreEvent::Partial(heard(2), "그리고".into()), t);
assert_eq!(d.captions().iter().map(|c| (c.text.as_str(), c.is_final)).collect::<Vec<_>>(),
vec![("자비스 오늘 날씨 어때?", true), ("그리고", false)]);
// An empty final removes its partial.
d.on_event(&fin(2, "", 300), t);
assert_eq!(d.captions().len(), 1);
assert_eq!(d.captions()[0].name, "민수");
}
#[test]
fn captions_are_capped_and_cleared_when_jarvis_leaves() {
let (mut d, t) = (Dashboard::default(), Instant::now());
d.on_event(&CoreEvent::Joined { guild_id: 1, channel_id: 5 }, t);
for i in 0..(CAPTIONS_KEPT as u32 + 5) {
d.on_event(&fin(i, &format!("말 {i}"), 100), t);
}
let caps = d.captions();
assert_eq!(caps.len(), CAPTIONS_KEPT);
assert_eq!(caps.last().unwrap().text, format!("말 {}", CAPTIONS_KEPT + 4));
d.on_event(&CoreEvent::Left { guild_id: 1, channel_id: 5, reason: "command".into() }, t);
assert!(d.captions().is_empty());
assert_eq!(d.view(&[], t)["voiceChannel"], Value::Null);
}
#[test]
fn an_answer_is_timed_from_its_askers_speech_end_to_its_first_sound() {
let (mut d, t) = (Dashboard::default(), Instant::now());
d.on_event(&fin(1, "자비스 몇 시야", 700), at(t, 1000)); // speech ended at 300
d.on_event(&CoreEvent::AnswerAsked { id: "r1".into(), user: A }, at(t, 1010));
d.on_event(&CoreEvent::ReplyText { id: "r1".into(), text: "열 시입니다.".into(), failed: false }, at(t, 1700));
d.on_event(&CoreEvent::SpeakStart("r1".into()), at(t, 1900));
assert_eq!(d.answers(), vec![AnswerTiming { final_ms: Some(700), brain_ms: Some(690), voice_ms: Some(200),
total_ms: Some(1600) }]);
// An answer stopped before it started is not timed; the earcon (another id) is not either.
d.on_event(&CoreEvent::AnswerAsked { id: "r3".into(), user: A }, at(t, 3000));
d.on_event(&CoreEvent::SpeakStart("r2".into()), at(t, 3000));
d.on_event(&CoreEvent::SpeakEnd(PlaybackEnd { id: "r3".into(), played: 0, cancelled: true, failed: false }),
at(t, 3100));
d.on_event(&CoreEvent::SpeakStart("r3".into()), at(t, 3200));
assert_eq!(d.answers().len(), 1);
}
#[test]
fn the_medians_cover_the_last_answers_kept() {
let (mut d, t) = (Dashboard::default(), Instant::now());
for (i, brain) in [500u64, 900, 700].into_iter().enumerate() {
let base = i as u64 * 10_000;
d.on_event(&fin(i as u32, "자비스", 600), at(t, base + 1000));
d.on_event(&CoreEvent::AnswerAsked { id: format!("r{i}"), user: A }, at(t, base + 1000));
d.on_event(&CoreEvent::ReplyText { id: format!("r{i}"), text: "네".into(), failed: false },
at(t, base + 1000 + brain));
d.on_event(&CoreEvent::SpeakStart(format!("r{i}")), at(t, base + 1000 + brain + 100));
}
let m = d.medians();
assert_eq!((m.final_ms, m.brain_ms, m.voice_ms, m.total_ms), (Some(600), Some(700), Some(100), Some(1400)));
for i in 0..ANSWERS_KEPT as u64 {
d.on_event(&CoreEvent::AnswerAsked { id: format!("x{i}"), user: 99 }, at(t, 100_000 + i));
d.on_event(&CoreEvent::SpeakStart(format!("x{i}")), at(t, 100_000 + i));
}
assert_eq!(d.answers().len(), ANSWERS_KEPT);
assert_eq!(d.medians().final_ms, None, "no final seen for the asker of the answers kept");
}
#[test]
fn missed_requests_show_newest_first_without_text() {
let t = Instant::now();
let missed: Vec<Missed> = (0..25).map(|i| Missed { speaker: i, reason: MissReason::Busy,
at: t, chars: 7 }).collect();
let v = missed_view(&missed, at(t, 3000));
let a = v.as_array().unwrap();
assert_eq!(a.len(), MISSED_SHOWN);
assert_eq!(a[0], json!({"speaker": "24", "reason": "busy", "secondsAgo": 3, "chars": 7}));
}
}

View File

@@ -76,6 +76,8 @@ pub enum CoreEvent {
/// The voice worker connection ended: finals of utterances in progress will not come, and the
/// restarted worker numbers utterances afresh.
WorkerLost,
/// `answer(user, ..)` queued speech `id` (the dashboard times answers by it).
AnswerAsked { id: String, user: u64 },
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -318,6 +320,22 @@ impl DiscordCore {
// --- voice ------------------------------------------------------------------------------------
/// The people in Jarvis's voice channel that it hears (ID and display name), as reported to the
/// worker; empty outside a voice channel.
pub fn voice_members(&self) -> Vec<(u64, String)> {
let inner = &self.inner;
let Some((guild, channel)) = inner.session.lock().unwrap().as_ref().map(|s| (s.guild_id, s.channel_id)) else {
return vec![];
};
let users: Vec<u64> = inner.cache.get()
.and_then(|c| c.guild(GuildId::new(guild)).map(|g| g.voice_states.values()
.filter(|v| v.channel_id.map(|c| c.get()) == Some(channel))
.map(|v| v.user_id.get())
.collect()))
.unwrap_or_default();
users.into_iter().filter(|u| should_hear(inner, *u)).map(|u| (u, name_of(inner, u))).collect()
}
pub fn voice_channel_id(&self) -> Option<u64> {
self.inner.session.lock().unwrap().as_ref().map(|s| s.channel_id)
}
@@ -410,6 +428,7 @@ impl DiscordCore {
pub fn answer(&self, user: u64, text: &str) -> Option<String> {
self.voice_channel_id()?;
let id = with_playback(&self.inner, |p| p.adopt());
let _ = self.inner.events.send(CoreEvent::AnswerAsked { id: id.clone(), user });
self.inner.worker.reply(&id, user, text, "voice", true);
Some(id)
}

View File

@@ -145,7 +145,10 @@ and the late "left" update cancelled the new join (seen on real Discord).
`SpeakStart` and `SpeakEnd`; `Stopped` when all speech is stopped (`stop()`, the slash command or the
session ending), sent before the `SpeakEnd`s it causes; `Joined` and `Left` (with a reason: `command`,
`moved`, `disconnected`, `shutdown`); `Addressed` and `Ranked` with the brain's answers to `ask_addressed` and `ask_rank`. `WorkerLost` when the voice worker connection ends: the finals of utterances in progress will not
come, and the restarted worker numbers utterances afresh.
come, and the restarted worker numbers utterances afresh. `AnswerAsked { id, user }` when
`answer(user, ..)` queued speech `id` (the dashboard times answers by it, `admin.spec.md`).
`voice_members()` lists the people in Jarvis's voice channel that it hears (ID and display name),
the same set reported to the worker.
- **Speaking**: `say(text, more)` queues a TTS request and returns its id (`None` when not in a voice
channel); `continue_text(id, text, more)` adds text to an open request. Each sentence the worker
sends becomes one clip (a 48 kHz stereo WAV played through songbird without resampling); clips play

View File

@@ -3,7 +3,9 @@
pub mod log;
pub mod calls;
pub mod chat;
pub mod control;
pub mod conversation;
pub mod dashboard;
pub mod datadir;
pub mod discord;
pub mod earcon;
@@ -17,5 +19,6 @@ pub mod services;
pub mod settings;
pub mod settings_edit;
pub mod speakers;
pub mod system;
pub mod worker_client;
pub mod worker_process;

View File

@@ -1,9 +1,9 @@
//! Jarvis app (headless until the Tauri GUI arrives in M5): jarvis-app <data folder>
//! The data folder can also come from JARVIS_DATA_DIR; the bot token from JARVIS_DISCORD_TOKEN.
//! The data folder can also come from JARVIS_DATA_DIR; the bot token from the stored secrets, else
//! JARVIS_DISCORD_TOKEN. The services run through the same controller the dashboard uses.
use jarvis_app::discord::CoreEvent;
use jarvis_app::services::start_services;
use jarvis_app::{debug_log, error_log};
use jarvis_app::control::{Controller, ServicesHost};
use jarvis_app::error_log;
#[tokio::main]
async fn main() {
@@ -12,32 +12,19 @@ async fn main() {
error_log!("app", "usage: jarvis-app <data folder> (or set JARVIS_DATA_DIR)");
std::process::exit(2);
};
let data_dir = std::path::Path::new(&data_dir);
let data_dir = std::path::PathBuf::from(data_dir);
// The log hub: daily files in <data folder>/logs, stored secrets masked in exports.
let logs_dir = data_dir.join("logs");
let _ = std::fs::create_dir_all(&logs_dir);
let hub = jarvis_app::logs::LogHub::new(Some(&logs_dir));
hub.set_secrets(jarvis_app::secrets::Secrets::new(&data_dir.join("config").join("secrets.dat")).values());
jarvis_app::logs::install(hub);
let mut services = match start_services(data_dir).await {
Ok(s) => s,
Err(e) => {
error_log!("app", "could not start: {e}");
std::process::exit(1);
}
};
loop {
tokio::select! {
ev = services.events.recv() => match ev {
// The conversation loop answers what is heard; here only the shape is logged, never the text.
Some(CoreEvent::Final(f)) => debug_log!("app", "heard {} utt={} chars={}", f.heard.speaker_id, f.heard.utt, f.text.chars().count()),
Some(_) => {}
None => break,
},
_ = shutdown_signal() => break,
}
let controller = Controller::new(ServicesHost { data_dir });
if controller.start().await.is_err() {
std::process::exit(1);
}
services.stop().await;
shutdown_signal().await;
let _ = controller.stop().await;
}
async fn shutdown_signal() {

View File

@@ -56,7 +56,8 @@ pub struct Request {
pub arrived_at: Instant,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
#[serde(rename_all = "snake_case")]
pub enum MissReason {
/// `first`: an answer was running or another caller finished first.
Busy,

View File

@@ -17,6 +17,8 @@ pub struct Services {
pub events: mpsc::UnboundedReceiver<CoreEvent>,
pub worker: WorkerClient,
pub worker_process: WorkerProcess,
/// The conversation's missed requests, for the dashboard.
pub missed: conversation::MissedLog,
}
impl Services {
@@ -66,6 +68,7 @@ pub async fn start_services(data_dir: &Path) -> Result<Services, Box<dyn std::er
// Call detection and the response policy answer what is heard; every event still reaches `events`.
let (out, events) = mpsc::unbounded_channel();
let conv = Conversation::new(core.clone(), calls, response).with_chat(chat_calls);
let missed = conv.missed_handle();
tokio::spawn(conversation::run(conv, core_events, out));
if let Err(e) = core.start().await {
worker.close();
@@ -73,5 +76,5 @@ pub async fn start_services(data_dir: &Path) -> Result<Services, Box<dyn std::er
return Err(Box::new(e));
}
crate::info_log!("app", "services started");
Ok(Services { core, events, worker, worker_process })
Ok(Services { core, events, worker, worker_process, missed })
}

98
app/src/system.rs Normal file
View File

@@ -0,0 +1,98 @@
//! Memory and GPU use for the dashboard (`app/src/admin.spec.md`, "Dashboard data").
use std::process::Stdio;
use std::time::Duration;
use serde::Serialize;
#[derive(Debug, Clone, PartialEq, Serialize)]
#[serde(rename_all = "camelCase")]
pub struct Gpu {
pub name: String,
pub use_percent: u32,
pub vram_used_mb: u64,
pub vram_total_mb: u64,
}
#[derive(Debug, Clone, PartialEq, Serialize, Default)]
#[serde(rename_all = "camelCase")]
pub struct Usage {
pub ram_used_mb: Option<u64>,
pub ram_total_mb: Option<u64>,
pub gpus: Vec<Gpu>,
}
/// One line per GPU of `nvidia-smi --query-gpu=name,utilization.gpu,memory.used,memory.total
/// --format=csv,noheader,nounits`; lines that do not parse are skipped.
pub fn parse_nvidia_smi(out: &str) -> Vec<Gpu> {
out.lines().filter_map(|l| {
let f: Vec<&str> = l.split(',').map(str::trim).collect();
let [name, util, used, total] = f.as_slice() else { return None };
Some(Gpu { name: (*name).into(), use_percent: util.parse().ok()?, vram_used_mb: used.parse().ok()?,
vram_total_mb: total.parse().ok()? })
}).collect()
}
/// The GPUs `nvidia-smi` reports; empty without one (or when it does not answer within 2 s).
pub async fn gpus() -> Vec<Gpu> {
let mut cmd = tokio::process::Command::new("nvidia-smi");
cmd.args(["--query-gpu=name,utilization.gpu,memory.used,memory.total", "--format=csv,noheader,nounits"])
.stdin(Stdio::null()).stdout(Stdio::piped()).stderr(Stdio::null()).kill_on_drop(true);
#[cfg(windows)]
cmd.creation_flags(0x0800_0000); // CREATE_NO_WINDOW
match tokio::time::timeout(Duration::from_secs(2), cmd.output()).await {
Ok(Ok(o)) if o.status.success() => parse_nvidia_smi(&String::from_utf8_lossy(&o.stdout)),
_ => vec![],
}
}
pub async fn usage() -> Usage {
let (used, total) = memory_mb();
Usage { ram_used_mb: used, ram_total_mb: total, gpus: gpus().await }
}
#[cfg(target_os = "linux")]
fn memory_mb() -> (Option<u64>, Option<u64>) {
let Ok(s) = std::fs::read_to_string("/proc/meminfo") else { return (None, None) };
let kb = |key: &str| s.lines().find(|l| l.starts_with(key))
.and_then(|l| l.split_whitespace().nth(1)?.parse::<u64>().ok());
let (total, avail) = (kb("MemTotal:"), kb("MemAvailable:"));
(total.zip(avail).map(|(t, a)| (t - a) / 1024), total.map(|t| t / 1024))
}
#[cfg(windows)]
fn memory_mb() -> (Option<u64>, Option<u64>) {
use windows_sys::Win32::System::SystemInformation::{GlobalMemoryStatusEx, MEMORYSTATUSEX};
// SAFETY: the struct is sized and zeroed as the API asks.
let mut m: MEMORYSTATUSEX = unsafe { std::mem::zeroed() };
m.dwLength = std::mem::size_of::<MEMORYSTATUSEX>() as u32;
if unsafe { GlobalMemoryStatusEx(&mut m) } == 0 {
return (None, None);
}
let mb = 1024 * 1024;
(Some((m.ullTotalPhys - m.ullAvailPhys) / mb), Some(m.ullTotalPhys / mb))
}
#[cfg(not(any(target_os = "linux", windows)))]
fn memory_mb() -> (Option<u64>, Option<u64>) {
(None, None)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn nvidia_smi_lines_become_gpus_and_junk_is_skipped() {
let out = "NVIDIA GeForce RTX 5050, 37, 5120, 8151\nNVIDIA GeForce RTX 4090, [N/A], 1, 24564\n\n";
assert_eq!(parse_nvidia_smi(out), vec![Gpu { name: "NVIDIA GeForce RTX 5050".into(), use_percent: 37,
vram_used_mb: 5120, vram_total_mb: 8151 }]);
}
#[tokio::test]
async fn usage_reads_this_machines_memory() {
let u = usage().await;
if cfg!(any(target_os = "linux", windows)) {
let (used, total) = (u.ram_used_mb.unwrap(), u.ram_total_mb.unwrap());
assert!(used > 0 && used <= total);
}
}
}

View File

@@ -184,6 +184,11 @@ impl WorkerClient {
tokio::spawn(async move { run(inner).await });
}
/// The worker's `hello_ok` while connected (STT and TTS readiness), else None.
pub fn hello(&self) -> Option<HelloOk> {
self.inner.ready.borrow().clone()
}
pub fn is_ready(&self) -> bool {
self.inner.ready.borrow().is_some()
}

File diff suppressed because one or more lines are too long