feat(app): M5 step 1, the admin backend under the management app

- datadir: the data folder layout, the location file (JARVIS_DATA_DIR wins),
  free space for the wizard.
- settings_edit: the fields the five settings screens edit; apply writes only
  non-defaults, keeps unknown keys and writes nothing unless the app's own
  check passes. A memory section reaches the worker's hello.
- secrets: config/secrets.dat, DPAPI for the Windows user; the services take
  the bot token from it before JARVIS_DISCORD_TOKEN.
- logs: the log hub behind the log screen (tabs, daily files kept 14 days,
  the recent error report, a zip under 10 MB, secrets and token-shaped text
  masked); the log macros and the worker's output feed it.

Spec: app/src/admin.spec.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
javis-bot
2026-11-12 13:35:59 +09:00
parent 07c701be54
commit 390f6f0b86
18 changed files with 1457 additions and 19 deletions

View File

@@ -32,6 +32,7 @@ Any code change must either adhere to our spec files perfectly or you should ask
| `src/jarvis/memory/recall_gate.spec.md` | Deterministic skip-enrichment heuristic when the hot window covers a follow-up | Fail-open; language-agnostic via `\w{3,}` + `re.UNICODE`; planner intent always wins |
| `src/voice_worker/voice_worker.spec.md` | v2 voice worker: multi-speaker streaming STT (shared Whisper, batched) and sentence TTS over a local WebSocket | Loopback only with an app token; no transcript text in logs; finals before partials; language-agnostic |
| `app/src/conversation.spec.md` | v2 call detection and response policies in the Rust app: wake words with near matches, call modes, designated people, voice stop, the four response policies with settling, interruption and missed requests | Pure types tested with fake times; language-agnostic matching on normalised letters; nothing dropped silently |
| `app/src/admin.spec.md` | v2 M5 admin backend for the management app: data folder and its location file, settings editing for the screens, stored secrets (DPAPI on Windows), log hub (tabs, daily files, error report, zip export) | One folder; edits keep the user's file (only non-defaults, unknown keys kept, all or nothing); secrets never in a log, copy or export |
| `app/src/discord_core.spec.md` | v2 Discord voice core in the Rust app: join/leave, per-speaker receive (own reorder window and Opus decoding after songbird decrypts) to the voice worker, sentence playback and stop, slash command, registered chat channels (read/send), voice worker process | Hears only the channel it is in and reads only registered channels; join notice on every join; chat is data and sent text never pings; no transcript or chat text in logs |
The LLM contexts graph at `docs/llm_contexts.md` maps every LLM call in the app (model, gating, inputs, outputs, limits, flow). Keep it up-to-date at all times: any change that adds, removes, or alters an LLM context (model resolution, timeout, cap, prompt source, gating flag, data-flow edge) must update `docs/llm_contexts.md` in the same PR.

118
app/Cargo.lock generated
View File

@@ -256,6 +256,15 @@ version = "0.9.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2459377285ad874054d797f3ccebf984978aa39129f6eafde5cdc8315b612f8"
[[package]]
name = "constant_time_eq"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9eb6feb175df65d0a3ccd3e73484e940d0ac664e8034f0519fd49a93db3071d9"
dependencies = [
"rustversion",
]
[[package]]
name = "convert_case"
version = "0.4.0"
@@ -457,7 +466,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "978747c1d849a7d2ee5e8adc0159961c48fb7e5db2f06af6723b80123bb53856"
dependencies = [
"cfg-if",
"hashbrown",
"hashbrown 0.14.5",
"lock_api",
"once_cell",
"parking_lot_core 0.9.12",
@@ -472,7 +481,7 @@ checksum = "e6361d5c062261c78a176addb82d4c821ae42bed6089de0e12603cd25de2059c"
dependencies = [
"cfg-if",
"crossbeam-utils",
"hashbrown",
"hashbrown 0.14.5",
"lock_api",
"once_cell",
"parking_lot_core 0.9.12",
@@ -713,6 +722,12 @@ dependencies = [
"syn 2.0.119",
]
[[package]]
name = "equivalent"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "00d174d5400e5e8fd687ad1049e2f578285fa914201b1af7e8b112a4546bd826"
[[package]]
name = "errno"
version = "0.3.14"
@@ -819,6 +834,16 @@ dependencies = [
"hashmap_derive",
]
[[package]]
name = "fs4"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7e72ed92b67c146290f88e9c89d60ca163ea417a446f61ffd7b72df3e7f1dfd5"
dependencies = [
"rustix",
"windows-sys 0.61.2",
]
[[package]]
name = "futures"
version = "0.3.34"
@@ -999,6 +1024,12 @@ version = "0.14.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5274423e17b7c9fc20b6e7e208532f9b19825d82dfd615708b70edd83df41f1"
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "hashmap_derive"
version = "0.1.0"
@@ -1408,6 +1439,16 @@ dependencies = [
"icu_properties",
]
[[package]]
name = "indexmap"
version = "2.14.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855"
dependencies = [
"equivalent",
"hashbrown 0.17.1",
]
[[package]]
name = "inout"
version = "0.1.4"
@@ -1447,6 +1488,8 @@ version = "0.1.0"
dependencies = [
"async-trait",
"base64 0.22.1",
"chrono",
"fs4",
"futures-util",
"opus2",
"rand 0.9.5",
@@ -1458,6 +1501,8 @@ dependencies = [
"tokio",
"tokio-tungstenite 0.26.2",
"unicode-normalization",
"windows-sys 0.61.2",
"zip",
]
[[package]]
@@ -1720,6 +1765,12 @@ dependencies = [
"pkg-config",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "litemap"
version = "0.8.3"
@@ -1829,6 +1880,18 @@ dependencies = [
"getrandom 0.2.17",
]
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags 2.13.2",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "no-std-net"
version = "0.6.0"
@@ -2701,6 +2764,19 @@ dependencies = [
"transpose",
]
[[package]]
name = "rustix"
version = "1.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e5e76742e5294153d4dde1207dbec5bdd81eef6fabdc0d2118babadabfd858aa"
dependencies = [
"bitflags 2.13.2",
"errno",
"libc",
"linux-raw-sys",
"windows-sys 0.61.2",
]
[[package]]
name = "rustls"
version = "0.22.4"
@@ -3899,6 +3975,12 @@ dependencies = [
"time",
]
[[package]]
name = "typed-path"
version = "0.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ce898823d8cd0f028662d2f349694b1af0a5cc9be83f1d213a00d3147445e1e"
[[package]]
name = "typemap_rev"
version = "0.3.0"
@@ -4511,6 +4593,26 @@ dependencies = [
"syn 3.0.10",
]
[[package]]
name = "zip"
version = "9.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3476fd497ac8af5b1ec7b51acab7bef380f7e8dd8e0487595bafd839417a302"
dependencies = [
"constant_time_eq",
"crc32fast",
"flate2",
"getrandom 0.4.3",
"indexmap",
"libc",
"memchr",
"nix",
"typed-path",
"windows-sys 0.61.2",
"zeroize",
"zopfli",
]
[[package]]
name = "zlib-rs"
version = "0.6.8"
@@ -4522,3 +4624,15 @@ name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zopfli"
version = "0.8.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aaf7fc5d30c28483d93805c4a5e12b05bbb52407fa67c5f8bd552374cd01fb11"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]

View File

@@ -28,9 +28,15 @@ rand = "0.9"
base64 = "0.22"
async-trait = "0.1"
opus2 = "0.4"
fs4 = "1.1"
chrono = { version = "0.4", default-features = false, features = ["clock", "std"] }
zip = { version = "9", default-features = false, features = ["deflate"] }
unicode-normalization = "0.1"
[profile.release]
lto = "thin"
codegen-units = 4
strip = "symbols"
[target.'cfg(windows)'.dependencies]
windows-sys = { version = "0.61", features = ["Win32_Foundation", "Win32_Security_Cryptography", "Win32_System_SystemInformation"] }

96
app/src/admin.spec.md Normal file
View File

@@ -0,0 +1,96 @@
# Admin backend spec (v2 M5)
What the management app's screens (`docs/v2/desktop-app.md`) stand on: the data folder, editing the
settings, the stored secrets and the logs. Plain Rust in the app crate, with no window: the Tauri
shell (M5, later step) only calls these, so every rule here is tested on any OS.
## Principles
- **One folder.** Everything the app keeps lives in the data folder the user chose
(`desktop-app.md`, "데이터 폴더 구조"); outside it there is only the file that remembers where it is.
- **The file stays the user's.** Settings edits change only the keys they name, write only values
that differ from the default, keep keys the app does not know, and never leave a half-written file.
- **Secrets never reach a log, a copy or an export.**
- **Logs carry no conversation text** (already the rule for every log line, `discord_core.spec.md`,
`voice_worker.spec.md`); exports still mask secrets.
## Data folder (`app/src/datadir.rs`)
The location file is `%APPDATA%\JarvisBot\location.json` on Windows and
`$XDG_CONFIG_HOME/jarvis-bot/location.json` (else `~/.config/jarvis-bot/location.json`) elsewhere:
`{"dataDir": "<absolute path>"}`. `JARVIS_DATA_DIR` set in the environment wins over it (tests,
development). No location (no file, no variable) means the first-run wizard has not finished.
`DataDir::create(path)` makes the layout below inside an existing or new folder and returns it; it
never deletes anything. `DataDir::choose(path)` creates it and writes the location file (atomically).
```
config/ runtime/ models/llm models/stt models/tts models/vad memory/ browser/ collect/
cache/ logs/
```
`free_bytes(path)` reports the free space of the disk holding the folder (or its nearest existing
parent); the wizard compares it with `RECOMMENDED_FREE_BYTES` (200 GB) and warns below it. A path
that is not absolute, or is a file, is refused with a message.
## Settings editing (`app/src/settings_edit.rs`)
`config/settings.json` keeps the format of `discord_core.spec.md` and `conversation.spec.md`.
- `fields()` describes every key the screens edit: its dotted path (`call.mode`), screen
(`bot`, `channels`, `chat`, `memory`, `model`), kind (bool, integer with a range, number with a
range, text, text list, choice with its values, Discord ID, Discord ID list, ID to number map,
chat channel list, free object), its default and a Korean label. The screens are built from this
list, so a key is added in one place.
- `view(dir)` returns every field's current value: the file's value, or the default when the file
does not set it. A file that does not parse is reported, not overwritten.
- `apply(dir, changes)` takes `{path: value}`. Each value is checked against its field; a value equal
to the default removes the key (and a section left empty), any other value is set. Keys the app
does not know are kept as they were. The whole result must pass the app's own settings check
(`parse_settings`); only then is it written (temporary file, then rename). A refused change names
the key and changes nothing.
- `memory` (new section): the worker's memory settings (`voice_worker.spec.md`, "Memory settings":
`chat`, `others`, `off`, `since_days`, `recent_minutes`, `recent_records`, `matches`,
`budget_tokens`), sent in the worker `hello` as `memory`. Default `{}` (the worker's defaults).
## Secrets (`app/src/secrets.rs`)
`config/secrets.dat` holds a JSON object of named secrets (`discordToken`). On Windows the bytes are
encrypted with DPAPI for the current Windows user (`CryptProtectData`, no prompt), so the file is
useless copied to another account or PC. Elsewhere (development only) the file is plain JSON with
owner-only permissions (0600). Writes are atomic. `get`, `set`, `remove`; `masked(value)` shows only
the last 4 characters for the screens (`••••abcd`).
The app's token comes from the secrets (`discordToken`), else from `JARVIS_DISCORD_TOKEN` (tests,
headless runs).
## Logs (`app/src/logs.rs`)
`debug_log!` and `error_log!` keep printing to stderr and also feed the log hub once the app
installs one (`logs::install`). Each line has a time, a level (`error`, `warn`, `info`, `debug`), a
service tab and its text. The tab comes from the line's category: `app`, `discord` (`voice`,
`receive`, `play`, `discord`), `chat`, `conversation`, `worker` (the voice worker's own lines; its
STT, LLM and TTS lines go to `stt`, `llm`, `tts` by their worker category), `browser`, `capture`,
`game` (the last three from M6-M7). Unknown categories go to `app`.
- Memory: the last 5,000 lines for the log screen (`query`: tab, lowest level, text to find, and
the lines after a given sequence number, for live follow).
- Files: `logs/<tab>-<YYYY-MM-DD>.log`, one line per entry; files older than 14 days are removed at
start and once a day.
- `recent_error_report()`: the last error with up to 30 lines before and 10 after it (all tabs, in
order), the app version and a short PC summary (OS, CPU count, memory); `None` without an error.
When it is over 2,000 characters (Discord's message limit) the screen offers a download instead.
- `export_zip(from, to, out)`: the log files of those days in a zip, at most 10 MB (Discord's file
limit): when larger, the oldest lines are left out and a note says so.
- Every copy and export runs through `redact`: each stored secret's value, and anything shaped like
a Discord bot token, becomes `[비밀값]`.
## Tests
`cargo test`: data folder layout, location file and the environment override, a refused relative
path, free space; settings view with defaults, apply writing only non-defaults, removing a key set
back to its default, keeping unknown keys, refusing a bad value or a result `parse_settings` refuses
without touching the file; secrets round trip, file permissions on Unix, masking; log tabs from
categories, query filters and follow, files per day and retention, error report bounds, zip size
cap, redaction of stored secrets and token-shaped text. Windows-only paths (DPAPI, `%APPDATA%`) are
checked on the Windows VM in the Windows build step.

175
app/src/datadir.rs Normal file
View File

@@ -0,0 +1,175 @@
//! The data folder and the file that remembers where it is (`app/src/admin.spec.md`).
use std::path::{Path, PathBuf};
use serde_json::json;
/// The wizard warns below this much free space (desktop-app.md: 200 GB recommended).
pub const RECOMMENDED_FREE_BYTES: u64 = 200 * 1_000_000_000;
/// Folders inside the data folder (desktop-app.md, "데이터 폴더 구조").
pub const LAYOUT: &[&str] = &[
"config", "runtime", "models/llm", "models/stt", "models/tts", "models/vad", "memory", "browser",
"collect", "cache", "logs",
];
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DataDir(PathBuf);
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DataDirError(pub String);
impl std::fmt::Display for DataDirError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
impl std::error::Error for DataDirError {}
fn io(what: &'static str, p: &Path) -> impl FnOnce(std::io::Error) -> DataDirError {
let p = p.display().to_string();
move |e| DataDirError(format!("{what} {p}: {e}"))
}
impl DataDir {
/// Make the layout inside `path` (new or existing); nothing is ever deleted.
pub fn create(path: &Path) -> Result<Self, DataDirError> {
if !path.is_absolute() {
return Err(DataDirError(format!("the data folder must be an absolute path: {}", path.display())));
}
if path.is_file() {
return Err(DataDirError(format!("the data folder is a file: {}", path.display())));
}
for sub in LAYOUT {
let d = path.join(sub);
std::fs::create_dir_all(&d).map_err(io("could not create", &d))?;
}
Ok(Self(path.to_path_buf()))
}
/// Create the folder and remember it in the location file.
pub fn choose(path: &Path, location_file: &Path) -> Result<Self, DataDirError> {
let dir = Self::create(path)?;
let body = json!({"dataDir": path.to_string_lossy()}).to_string();
write_atomic(location_file, body.as_bytes()).map_err(io("could not write", location_file))?;
Ok(dir)
}
/// The folder in use: `JARVIS_DATA_DIR`, else the location file; None before the wizard finished.
pub fn locate(location_file: &Path) -> Option<Self> {
Self::locate_with(std::env::var("JARVIS_DATA_DIR").ok(), location_file)
}
/// `locate` with the environment variable's value given.
pub fn locate_with(env: Option<String>, location_file: &Path) -> Option<Self> {
if let Some(p) = env.filter(|p| !p.is_empty()) {
return Some(Self(PathBuf::from(p)));
}
let raw = std::fs::read_to_string(location_file).ok()?;
let v: serde_json::Value = serde_json::from_str(raw.trim_start_matches('\u{feff}')).ok()?;
let p = PathBuf::from(v.get("dataDir")?.as_str()?);
p.is_absolute().then_some(Self(p))
}
pub fn path(&self) -> &Path {
&self.0
}
pub fn config(&self) -> PathBuf {
self.0.join("config")
}
pub fn settings_file(&self) -> PathBuf {
self.config().join("settings.json")
}
pub fn secrets_file(&self) -> PathBuf {
self.config().join("secrets.dat")
}
pub fn logs(&self) -> PathBuf {
self.0.join("logs")
}
}
/// Where the location file lives on this OS (`%APPDATA%\JarvisBot`, else the XDG config folder).
pub fn default_location_file() -> Option<PathBuf> {
if cfg!(windows) {
let appdata = std::env::var_os("APPDATA")?;
return Some(PathBuf::from(appdata).join("JarvisBot").join("location.json"));
}
let base = std::env::var_os("XDG_CONFIG_HOME").map(PathBuf::from)
.or_else(|| std::env::var_os("HOME").map(|h| PathBuf::from(h).join(".config")))?;
Some(base.join("jarvis-bot").join("location.json"))
}
/// Free bytes on the disk that holds `path` (or its nearest existing parent).
pub fn free_bytes(path: &Path) -> Result<u64, DataDirError> {
let mut p = path;
while !p.exists() {
p = p.parent().ok_or_else(|| DataDirError(format!("no existing parent for {}", path.display())))?;
}
fs4::available_space(p).map_err(io("could not read the free space of", p))
}
/// Write `bytes` to `path` through a temporary file in the same folder and a rename: a crash leaves
/// the old file or the new one, never half of one.
pub fn write_atomic(path: &Path, bytes: &[u8]) -> std::io::Result<()> {
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let tmp = path.with_extension(format!("tmp{}", std::process::id()));
std::fs::write(&tmp, bytes)?;
std::fs::rename(&tmp, path).inspect_err(|_| {
let _ = std::fs::remove_file(&tmp);
})
}
#[cfg(test)]
pub(crate) fn temp_dir(tag: &str) -> PathBuf {
let d = std::env::temp_dir().join(format!("jarvis-{tag}-{}-{}", std::process::id(), rand::random::<u32>()));
std::fs::create_dir_all(&d).unwrap();
d
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn create_makes_the_layout_and_keeps_what_is_there() {
let root = temp_dir("dd").join("data");
std::fs::create_dir_all(root.join("memory")).unwrap();
std::fs::write(root.join("memory").join("keep.db"), b"x").unwrap();
let d = DataDir::create(&root).unwrap();
for sub in LAYOUT {
assert!(root.join(sub).is_dir(), "{sub}");
}
assert_eq!(std::fs::read(root.join("memory").join("keep.db")).unwrap(), b"x");
assert_eq!(d.settings_file(), root.join("config").join("settings.json"));
}
#[test]
fn a_relative_path_or_a_file_is_refused() {
assert!(DataDir::create(Path::new("relative/data")).unwrap_err().0.contains("absolute"));
let f = temp_dir("ddf").join("a-file");
std::fs::write(&f, b"x").unwrap();
assert!(DataDir::create(&f).unwrap_err().0.contains("file"));
}
#[test]
fn choose_remembers_the_folder_and_locate_finds_it() {
let base = temp_dir("ddl");
let loc = base.join("cfg").join("location.json");
assert_eq!(DataDir::locate_with(None, &loc), None, "no wizard yet");
let data = base.join("데이터");
DataDir::choose(&data, &loc).unwrap();
assert_eq!(DataDir::locate_with(None, &loc), Some(DataDir(data)));
// The environment variable wins (tests, development); an empty one does not count.
assert_eq!(DataDir::locate_with(Some("/elsewhere".into()), &loc).unwrap().path(), Path::new("/elsewhere"));
assert!(DataDir::locate_with(Some(String::new()), &loc).is_some());
}
#[test]
fn free_space_is_read_from_the_nearest_existing_parent() {
let base = temp_dir("ddfree");
let n = free_bytes(&base.join("not").join("yet")).unwrap();
assert!(n > 0);
}
}

View File

@@ -47,9 +47,10 @@ app/
## Starting
`start_services(data_dir)` reads `<data_dir>/config/settings.json`, starts the voice worker process,
connects to it once it prints `READY`, and starts the Discord core. Until the setup wizard (M5) stores
it encrypted, the bot token comes from `JARVIS_DISCORD_TOKEN` and the data folder from the first
argument or `JARVIS_DATA_DIR`. The binary stops every service on Ctrl+C or SIGTERM.
connects to it once it prints `READY`, and starts the Discord core. The bot token comes from the
stored secrets (`config/secrets.dat`, `admin.spec.md`), else from `JARVIS_DISCORD_TOKEN`; the binary
takes the data folder from the first argument or `JARVIS_DATA_DIR`, installs the log hub (daily files
in `logs/`) and stops every service on Ctrl+C or SIGTERM.
## Settings
@@ -72,6 +73,7 @@ strings (numbers would lose precision).
| `voiceWorker.python` | `python3` (`python.exe` on Windows) | Python of the worker venv. |
| `voiceWorker.cwd` | the repo's `src/` (development builds) | Folder holding the `voice_worker` package. |
| `voiceWorker.args` | `[]` | Extra worker arguments, e.g. `["--melo-python", "<MeloTTS venv python>"]`. |
| `memory` | `{}` | The worker's memory settings (`voice_worker.spec.md`, "Memory settings"), sent in its `hello`; the settings screens check them (`admin.spec.md`). |
The `call` and `response` sections (call detection and response policies, M4) are in
`conversation.spec.md`.

View File

@@ -4,14 +4,18 @@ pub mod log;
pub mod calls;
pub mod chat;
pub mod conversation;
pub mod datadir;
pub mod discord;
pub mod earcon;
pub mod frames;
pub mod logs;
pub mod playback;
pub mod policy;
pub mod receive;
pub mod secrets;
pub mod services;
pub mod settings;
pub mod settings_edit;
pub mod speakers;
pub mod worker_client;
pub mod worker_process;

View File

@@ -15,20 +15,49 @@ pub fn line(category: &str, message: &str) -> String {
format!("[{category:^10}] {message}")
}
/// Print a line and give it to the log hub when one is installed (`app/src/admin.spec.md`).
pub fn emit(level: crate::logs::Level, category: &str, message: &str) {
match level {
crate::logs::Level::Debug => eprintln!("{}", line(category, message)),
crate::logs::Level::Info => eprintln!("[{category}] {message}"),
crate::logs::Level::Warn => eprintln!("[{category}] WARN {message}"),
crate::logs::Level::Error => eprintln!("[{category}] ERROR {message}"),
}
if let Some(hub) = crate::logs::installed() {
hub.push(level, category, message);
}
}
/// Only when `JARVIS_DEBUG=1`.
#[macro_export]
macro_rules! debug_log {
($cat:expr, $($arg:tt)*) => {
if $crate::log::debug_enabled() {
eprintln!("{}", $crate::log::line($cat, &format!($($arg)*)));
$crate::log::emit($crate::logs::Level::Debug, $cat, &format!($($arg)*));
}
};
}
/// Always logged: events the log screen shows without debug logging (start, stop, joins).
#[macro_export]
macro_rules! info_log {
($cat:expr, $($arg:tt)*) => {
$crate::log::emit($crate::logs::Level::Info, $cat, &format!($($arg)*));
};
}
#[macro_export]
macro_rules! warn_log {
($cat:expr, $($arg:tt)*) => {
$crate::log::emit($crate::logs::Level::Warn, $cat, &format!($($arg)*));
};
}
/// Errors are always printed; the same rule applies: no transcript or chat text in the message.
#[macro_export]
macro_rules! error_log {
($cat:expr, $($arg:tt)*) => {
eprintln!("[{}] ERROR {}", $cat, format!($($arg)*));
$crate::log::emit($crate::logs::Level::Error, $cat, &format!($($arg)*));
};
}

454
app/src/logs.rs Normal file
View File

@@ -0,0 +1,454 @@
//! The log hub behind the log screen (`app/src/admin.spec.md`, "Logs"): every line the app logs,
//! by service tab, kept in memory for the screen and in daily files under `logs/`.
use std::collections::{HashMap, VecDeque};
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, OnceLock};
use chrono::{DateTime, Local, NaiveDate};
/// Lines kept in memory for the screen.
pub const KEPT_LINES: usize = 5_000;
/// Log files older than this are removed.
pub const RETENTION_DAYS: i64 = 14;
/// Discord's message limit: a longer error report is offered as a download.
pub const DISCORD_MESSAGE_CHARS: usize = 2_000;
/// Discord's file limit for the zip export.
pub const ZIP_MAX_BYTES: usize = 10 * 1024 * 1024;
const REPORT_BEFORE: usize = 30;
const REPORT_AFTER: usize = 10;
pub const SECRET_MARK: &str = "[비밀값]";
#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, serde::Serialize)]
#[serde(rename_all = "lowercase")]
pub enum Level {
Debug,
Info,
Warn,
Error,
}
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
pub struct LogLine {
pub seq: u64,
/// Milliseconds since the Unix epoch.
pub at_ms: i64,
pub level: Level,
pub tab: String,
pub text: String,
}
/// The screen's tab for a log category (worker lines: by the worker's own category, see `push_worker`).
pub fn tab_of(category: &str) -> &'static str {
match category {
"voice" | "receive" | "play" | "discord" => "discord",
"chat" => "chat",
"conversation" => "conversation",
"worker" | "voice_worker" => "worker",
"stt" => "stt",
"llm" => "llm",
"tts" => "tts",
"browser" => "browser",
"capture" => "capture",
"game" => "game",
_ => "app",
}
}
#[derive(Default)]
struct State {
lines: VecDeque<LogLine>,
next_seq: u64,
files: HashMap<(String, NaiveDate), std::fs::File>,
secrets: Vec<String>,
last_cleanup: Option<NaiveDate>,
}
pub struct LogHub {
dir: Option<PathBuf>,
state: Mutex<State>,
}
impl LogHub {
/// A hub writing daily files to `dir` (None: memory only).
pub fn new(dir: Option<&Path>) -> Arc<Self> {
let hub = Arc::new(Self { dir: dir.map(Path::to_path_buf), state: Mutex::default() });
hub.cleanup(Local::now().date_naive());
hub
}
/// The stored secrets' values, masked in every copy and export.
pub fn set_secrets(&self, values: Vec<String>) {
self.state.lock().unwrap().secrets = values;
}
pub fn push(&self, level: Level, category: &str, text: &str) {
self.push_at(Local::now(), level, tab_of(category), text);
}
/// A line of the voice worker's output: its `[ category ]` prefix names the tab (stt, llm, tts,
/// else worker); a Python warning or traceback line is a warning.
pub fn push_worker(&self, line: &str) {
let (cat, text) = match line.strip_prefix('[').and_then(|r| r.split_once(']')) {
Some((c, rest)) => (c.trim(), rest.trim_start()),
None => ("worker", line),
};
let tab = match tab_of(cat) {
t @ ("stt" | "llm" | "tts") => t,
_ => "worker",
};
let level = if text.contains("Error") || text.contains("Traceback") { Level::Error }
else if text.contains("Warning") { Level::Warn } else { Level::Info };
self.push_at(Local::now(), level, tab, text);
}
fn push_at(&self, at: DateTime<Local>, level: Level, tab: &str, text: &str) {
let mut s = self.state.lock().unwrap();
let line = LogLine { seq: s.next_seq, at_ms: at.timestamp_millis(), level, tab: tab.into(), text: text.into() };
s.next_seq += 1;
if let Some(dir) = &self.dir {
let day = at.date_naive();
let key = (tab.to_owned(), day);
if !s.files.contains_key(&key) {
s.files.retain(|(_, d), _| *d == day); // yesterday's files are done
let path = dir.join(format!("{tab}-{day}.log"));
if let Ok(f) = std::fs::OpenOptions::new().create(true).append(true).open(path) {
s.files.insert(key.clone(), f);
}
}
if let Some(f) = s.files.get_mut(&key) {
let _ = writeln!(f, "{}", format_line(&line));
}
}
s.lines.push_back(line);
if s.lines.len() > KEPT_LINES {
s.lines.pop_front();
}
let today = at.date_naive();
let due = s.last_cleanup != Some(today);
drop(s);
if due {
self.cleanup(today);
}
}
/// Remove log files older than the retention.
fn cleanup(&self, today: NaiveDate) {
self.state.lock().unwrap().last_cleanup = Some(today);
let Some(dir) = &self.dir else { return };
let Ok(entries) = std::fs::read_dir(dir) else { return };
for e in entries.flatten() {
let name = e.file_name().to_string_lossy().into_owned();
if let Some(day) = file_day(&name) {
if (today - day).num_days() > RETENTION_DAYS {
let _ = std::fs::remove_file(e.path());
}
}
}
}
/// Lines for the screen: one tab (None: all), this level and above, containing `find`, after
/// sequence number `after` (live follow), the last `limit`.
pub fn query(&self, tab: Option<&str>, min_level: Level, find: Option<&str>, after: Option<u64>, limit: usize)
-> Vec<LogLine> {
let s = self.state.lock().unwrap();
let find = find.map(str::to_lowercase);
let mut out: Vec<LogLine> = s.lines.iter()
.filter(|l| tab.is_none_or(|t| l.tab == t) && l.level >= min_level
&& after.is_none_or(|a| l.seq > a)
&& find.as_deref().is_none_or(|f| l.text.to_lowercase().contains(f)))
.cloned().collect();
if out.len() > limit {
out.drain(..out.len() - limit);
}
out
}
/// The last error with the lines around it, the app version and a PC summary, masked; None
/// without an error.
pub fn recent_error_report(&self, app_version: &str) -> Option<String> {
let s = self.state.lock().unwrap();
let i = s.lines.iter().rposition(|l| l.level == Level::Error)?;
let from = i.saturating_sub(REPORT_BEFORE);
let to = (i + 1 + REPORT_AFTER).min(s.lines.len());
let mut text = format!("Jarvis {app_version} | {}\n", pc_summary());
for l in s.lines.range(from..to) {
text.push_str(&format_line(l));
text.push('\n');
}
Some(redact(&text, &s.secrets))
}
/// The log files of days `from..=to` in a zip at `out`, masked, at most `ZIP_MAX_BYTES`: when
/// larger, the oldest lines are left out and NOTE.txt says how many. Returns the lines left out.
pub fn export_zip(&self, from: NaiveDate, to: NaiveDate, out: &Path) -> std::io::Result<usize> {
let secrets = self.state.lock().unwrap().secrets.clone();
let Some(dir) = &self.dir else { return Err(std::io::Error::other("no log folder")) };
let mut files: Vec<(NaiveDate, String, Vec<String>)> = vec![];
for e in std::fs::read_dir(dir)?.flatten() {
let name = e.file_name().to_string_lossy().into_owned();
let Some(day) = file_day(&name) else { continue };
if day < from || day > to {
continue;
}
let text = std::fs::read_to_string(e.path()).unwrap_or_default();
files.push((day, name, redact(&text, &secrets).lines().map(str::to_owned).collect()));
}
files.sort();
let mut dropped = 0usize;
loop {
let bytes = build_zip(&files, dropped)?;
if bytes.len() <= ZIP_MAX_BYTES {
std::fs::write(out, bytes)?;
return Ok(dropped);
}
let total: usize = files.iter().map(|f| f.2.len()).sum();
if dropped >= total {
return Err(std::io::Error::other("the logs do not fit in the zip limit"));
}
// Drop a fifth more of the oldest lines and try again.
dropped = (dropped + total.div_ceil(5)).min(total);
}
}
}
/// The zip with the oldest `dropped` lines (oldest files first) left out.
fn build_zip(files: &[(NaiveDate, String, Vec<String>)], dropped: usize) -> std::io::Result<Vec<u8>> {
let mut zip = zip::ZipWriter::new(std::io::Cursor::new(Vec::new()));
let opts = zip::write::SimpleFileOptions::default().compression_method(zip::CompressionMethod::Deflated);
let mut skip = dropped;
for (_, name, lines) in files {
let cut = skip.min(lines.len());
skip -= cut;
if cut == lines.len() {
continue;
}
zip.start_file(name.as_str(), opts).map_err(std::io::Error::other)?;
for l in &lines[cut..] {
zip.write_all(l.as_bytes())?;
zip.write_all(b"\n")?;
}
}
if dropped > 0 {
zip.start_file("NOTE.txt", opts).map_err(std::io::Error::other)?;
zip.write_all(format!("The oldest {dropped} lines were left out to stay under 10 MB.\n").as_bytes())?;
}
Ok(zip.finish().map_err(std::io::Error::other)?.into_inner())
}
fn file_day(name: &str) -> Option<NaiveDate> {
let stem = name.strip_suffix(".log")?;
let date = stem.get(stem.len().checked_sub(10)?..)?;
NaiveDate::parse_from_str(date, "%Y-%m-%d").ok()
}
fn format_line(l: &LogLine) -> String {
let at = DateTime::from_timestamp_millis(l.at_ms).map(|t| t.with_timezone(&Local).format("%Y-%m-%d %H:%M:%S%.3f").to_string())
.unwrap_or_default();
let level = match l.level { Level::Debug => "DEBUG", Level::Info => "INFO", Level::Warn => "WARN", Level::Error => "ERROR" };
format!("{at} {level:5} [{}] {}", l.tab, l.text)
}
fn token_char(c: char) -> bool {
c.is_ascii_alphanumeric() || c == '_' || c == '-'
}
/// A Discord bot token: three base64url parts joined by dots (id, time, signature).
fn looks_like_token(word: &str) -> bool {
let parts: Vec<&str> = word.split('.').collect();
parts.len() == 3 && parts[0].len() >= 20 && parts[1].len() >= 5 && parts[2].len() >= 25
&& parts.iter().all(|p| p.chars().all(token_char))
}
/// Mask each stored secret and anything shaped like a Discord bot token.
pub fn redact(text: &str, secrets: &[String]) -> String {
let mut out = text.to_owned();
for s in secrets.iter().filter(|s| s.len() >= 6) {
out = out.replace(s.as_str(), SECRET_MARK);
}
let mut result = String::with_capacity(out.len());
let mut word = String::new();
let flush = |word: &mut String, result: &mut String| {
if looks_like_token(word) { result.push_str(SECRET_MARK) } else { result.push_str(word) }
word.clear();
};
for c in out.chars() {
if token_char(c) || c == '.' {
word.push(c);
} else {
flush(&mut word, &mut result);
result.push(c);
}
}
flush(&mut word, &mut result);
result
}
/// OS, CPU threads and memory, for error reports.
pub fn pc_summary() -> String {
let cpus = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(0);
let mem = total_memory_bytes().map(|b| format!("{:.1} GB RAM", b as f64 / 1e9)).unwrap_or_default();
format!("{} {} | {cpus} CPU threads | {mem}", std::env::consts::OS, std::env::consts::ARCH)
}
#[cfg(target_os = "linux")]
fn total_memory_bytes() -> Option<u64> {
let s = std::fs::read_to_string("/proc/meminfo").ok()?;
let kb: u64 = s.lines().find(|l| l.starts_with("MemTotal:"))?.split_whitespace().nth(1)?.parse().ok()?;
Some(kb * 1024)
}
#[cfg(windows)]
fn total_memory_bytes() -> Option<u64> {
use windows_sys::Win32::System::SystemInformation::{GlobalMemoryStatusEx, MEMORYSTATUSEX};
// SAFETY: the struct is sized and zeroed as the API asks.
let mut m: MEMORYSTATUSEX = unsafe { std::mem::zeroed() };
m.dwLength = std::mem::size_of::<MEMORYSTATUSEX>() as u32;
(unsafe { GlobalMemoryStatusEx(&mut m) } != 0).then_some(m.ullTotalPhys)
}
#[cfg(not(any(target_os = "linux", windows)))]
fn total_memory_bytes() -> Option<u64> {
None
}
static HUB: OnceLock<Arc<LogHub>> = OnceLock::new();
/// Make the log macros feed `hub` too (once per process).
pub fn install(hub: Arc<LogHub>) {
let _ = HUB.set(hub);
}
pub fn installed() -> Option<&'static Arc<LogHub>> {
HUB.get()
}
#[cfg(test)]
mod tests {
use super::*;
use crate::datadir::temp_dir;
use chrono::TimeZone;
fn at(y: i32, m: u32, d: u32) -> DateTime<Local> {
Local.with_ymd_and_hms(y, m, d, 12, 0, 0).unwrap()
}
#[test]
fn categories_map_to_tabs_and_worker_lines_to_their_own() {
let hub = LogHub::new(None);
hub.push(Level::Debug, "play", "say id=r1");
hub.push(Level::Error, "nonsense", "x");
hub.push_worker("[ voice_worker ] brain reply ms=600");
hub.push_worker("[ stt ] batch=3");
hub.push_worker("/x.py:9: FutureWarning: deprecated");
let tabs: Vec<_> = hub.query(None, Level::Debug, None, None, 99).into_iter().map(|l| (l.tab, l.level)).collect();
assert_eq!(tabs, vec![("discord".into(), Level::Debug), ("app".into(), Level::Error),
("worker".into(), Level::Info), ("stt".into(), Level::Info), ("worker".into(), Level::Warn)]);
}
#[test]
fn query_filters_by_tab_level_text_and_follows() {
let hub = LogHub::new(None);
for i in 0..10 {
hub.push(if i % 2 == 0 { Level::Debug } else { Level::Error }, "chat", &format!("line {i}"));
}
hub.push(Level::Error, "voice", "joined");
assert_eq!(hub.query(Some("chat"), Level::Error, None, None, 99).len(), 5);
assert_eq!(hub.query(None, Level::Debug, Some("LINE 3"), None, 99)[0].text, "line 3");
let last = hub.query(None, Level::Debug, None, None, 3);
assert_eq!(last.iter().map(|l| l.seq).collect::<Vec<_>>(), vec![8, 9, 10]);
assert_eq!(hub.query(None, Level::Debug, None, Some(9), 99).len(), 1, "only lines after seq 9");
for i in 0..KEPT_LINES {
hub.push(Level::Debug, "app", &i.to_string());
}
assert_eq!(hub.query(None, Level::Debug, None, None, usize::MAX).len(), KEPT_LINES);
}
#[test]
fn lines_go_to_daily_files_and_old_files_are_removed() {
let dir = temp_dir("logs");
std::fs::write(dir.join("chat-2026-01-01.log"), "old\n").unwrap();
std::fs::write(dir.join("notes.txt"), "not a log").unwrap();
let hub = LogHub::new(Some(&dir));
hub.push_at(at(2026, 1, 10), Level::Info, "chat", "kept");
hub.push_at(at(2026, 1, 20), Level::Info, "chat", "next day");
assert!(std::fs::read_to_string(dir.join("chat-2026-01-10.log")).unwrap().contains("kept"));
assert!(std::fs::read_to_string(dir.join("chat-2026-01-20.log")).unwrap().contains("next day"));
// The 20th is more than 14 days after the 1st.
assert!(!dir.join("chat-2026-01-01.log").exists());
assert!(dir.join("notes.txt").exists());
}
#[test]
fn the_error_report_holds_the_last_error_and_its_neighbours_masked() {
let hub = LogHub::new(None);
assert_eq!(hub.recent_error_report("0.1.0"), None);
hub.set_secrets(vec!["s3cr3t-value".into()]);
for i in 0..50 {
hub.push(Level::Debug, "app", &format!("before {i}"));
}
hub.push(Level::Error, "discord", "login failed with s3cr3t-value");
for i in 0..20 {
hub.push(Level::Debug, "app", &format!("after {i}"));
}
let r = hub.recent_error_report("0.1.0").unwrap();
assert!(r.starts_with("Jarvis 0.1.0 | "));
assert!(r.contains("before 49") && r.contains("before 20") && !r.contains("before 19"));
assert!(r.contains("after 9") && !r.contains("after 10"));
assert!(r.contains("login failed with [비밀값]") && !r.contains("s3cr3t"));
}
#[test]
fn redaction_masks_secrets_and_token_shaped_text() {
let token = format!("{}.{}.{}", "M".repeat(24), "GabcdE", "x".repeat(30));
let out = redact(&format!("token={token}, ok id=123456789012345678 a.b.c"), &[]);
assert_eq!(out, "token=[비밀값], ok id=123456789012345678 a.b.c");
}
#[test]
fn the_zip_stays_under_the_limit_by_leaving_out_the_oldest_lines() {
let dir = temp_dir("logzip");
let hub = LogHub::new(Some(&dir));
hub.set_secrets(vec!["hidden-token".into()]);
hub.push_at(at(2026, 3, 1), Level::Info, "app", "day one hidden-token");
hub.push_at(at(2026, 3, 2), Level::Info, "app", "day two");
hub.push_at(at(2026, 3, 5), Level::Info, "app", "outside the range");
let out = dir.join("small.zip");
assert_eq!(hub.export_zip(NaiveDate::from_ymd_opt(2026, 3, 1).unwrap(),
NaiveDate::from_ymd_opt(2026, 3, 2).unwrap(), &out).unwrap(), 0);
let mut z = zip::ZipArchive::new(std::fs::File::open(&out).unwrap()).unwrap();
let mut names: Vec<String> = (0..z.len()).map(|i| z.by_index(i).unwrap().name().unwrap().to_string()).collect();
names.sort();
assert_eq!(names, vec!["app-2026-03-01.log", "app-2026-03-02.log"]);
let mut first = String::new();
std::io::Read::read_to_string(&mut z.by_name("app-2026-03-01.log").unwrap(), &mut first).unwrap();
assert!(first.contains("day one [비밀값]"));
// Incompressible lines past 10 MB: the oldest go, the newest stay.
let big = temp_dir("logbig");
let hub = LogHub::new(Some(&big));
let mut seed = 1u64;
for day in 1..=3 {
for _ in 0..40_000 {
let mut s = String::new();
for _ in 0..140 {
seed = seed.wrapping_mul(6364136223846793005).wrapping_add(1442695040888963407);
s.push(char::from(b'!' + (seed >> 58) as u8));
}
hub.push_at(at(2026, 4, day), Level::Info, "app", &s);
}
}
hub.push_at(at(2026, 4, 3), Level::Info, "app", "the newest line");
let out = big.join("big.zip");
let dropped = hub.export_zip(NaiveDate::from_ymd_opt(2026, 4, 1).unwrap(),
NaiveDate::from_ymd_opt(2026, 4, 3).unwrap(), &out).unwrap();
assert!(dropped > 0);
assert!(std::fs::metadata(&out).unwrap().len() as usize <= ZIP_MAX_BYTES);
let mut z = zip::ZipArchive::new(std::fs::File::open(&out).unwrap()).unwrap();
let mut last = String::new();
std::io::Read::read_to_string(&mut z.by_name("app-2026-04-03.log").unwrap(), &mut last).unwrap();
assert!(last.contains("the newest line"));
let mut note = String::new();
std::io::Read::read_to_string(&mut z.by_name("NOTE.txt").unwrap(), &mut note).unwrap();
assert!(note.contains(&dropped.to_string()));
}
}

View File

@@ -12,7 +12,14 @@ async fn main() {
error_log!("app", "usage: jarvis-app <data folder> (or set JARVIS_DATA_DIR)");
std::process::exit(2);
};
let mut services = match start_services(std::path::Path::new(&data_dir)).await {
let data_dir = std::path::Path::new(&data_dir);
// The log hub: daily files in <data folder>/logs, stored secrets masked in exports.
let logs_dir = data_dir.join("logs");
let _ = std::fs::create_dir_all(&logs_dir);
let hub = jarvis_app::logs::LogHub::new(Some(&logs_dir));
hub.set_secrets(jarvis_app::secrets::Secrets::new(&data_dir.join("config").join("secrets.dat")).values());
jarvis_app::logs::install(hub);
let mut services = match start_services(data_dir).await {
Ok(s) => s,
Err(e) => {
error_log!("app", "could not start: {e}");

181
app/src/secrets.rs Normal file
View File

@@ -0,0 +1,181 @@
//! Stored secrets, `config/secrets.dat` (`app/src/admin.spec.md`): DPAPI-encrypted for the Windows
//! user; plain JSON with owner-only permissions elsewhere (development only).
use std::path::{Path, PathBuf};
use serde_json::{Map, Value};
use crate::datadir::write_atomic;
/// The Discord bot token's name in the store.
pub const DISCORD_TOKEN: &str = "discordToken";
#[derive(Debug)]
pub struct SecretsError(pub String);
impl std::fmt::Display for SecretsError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
impl std::error::Error for SecretsError {}
pub struct Secrets {
file: PathBuf,
}
impl Secrets {
pub fn new(file: &Path) -> Self {
Self { file: file.to_path_buf() }
}
fn load(&self) -> Result<Map<String, Value>, SecretsError> {
let bytes = match std::fs::read(&self.file) {
Ok(b) => b,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Map::new()),
Err(e) => return Err(SecretsError(format!("could not read the secrets: {e}"))),
};
let plain = unprotect(&bytes)?;
match serde_json::from_slice(&plain) {
Ok(Value::Object(m)) => Ok(m),
_ => Err(SecretsError("the secrets file is damaged".into())),
}
}
fn save(&self, m: &Map<String, Value>) -> Result<(), SecretsError> {
let plain = serde_json::to_vec(m).expect("a map serialises");
let bytes = protect(&plain)?;
write_atomic(&self.file, &bytes).map_err(|e| SecretsError(format!("could not write the secrets: {e}")))?;
restrict(&self.file);
Ok(())
}
pub fn get(&self, name: &str) -> Result<Option<String>, SecretsError> {
Ok(self.load()?.get(name).and_then(Value::as_str).map(str::to_owned))
}
pub fn set(&self, name: &str, value: &str) -> Result<(), SecretsError> {
let mut m = self.load()?;
m.insert(name.into(), Value::String(value.into()));
self.save(&m)
}
pub fn remove(&self, name: &str) -> Result<(), SecretsError> {
let mut m = self.load()?;
if m.remove(name).is_some() {
self.save(&m)?;
}
Ok(())
}
/// Every stored value, for log redaction.
pub fn values(&self) -> Vec<String> {
self.load().map(|m| m.values().filter_map(Value::as_str).map(str::to_owned).collect()).unwrap_or_default()
}
}
/// Only the last 4 characters, for the screens: `••••abcd`.
pub fn masked(value: &str) -> String {
let chars: Vec<char> = value.chars().collect();
let tail: String = chars[chars.len().saturating_sub(4)..].iter().collect();
format!("••••{tail}")
}
#[cfg(unix)]
fn restrict(path: &Path) {
use std::os::unix::fs::PermissionsExt;
let _ = std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600));
}
#[cfg(not(unix))]
fn restrict(_: &Path) {}
#[cfg(not(windows))]
fn protect(plain: &[u8]) -> Result<Vec<u8>, SecretsError> {
Ok(plain.to_vec())
}
#[cfg(not(windows))]
fn unprotect(bytes: &[u8]) -> Result<Vec<u8>, SecretsError> {
Ok(bytes.to_vec())
}
#[cfg(windows)]
fn dpapi(input: &[u8], encrypt: bool) -> Result<Vec<u8>, SecretsError> {
use windows_sys::Win32::Foundation::LocalFree;
use windows_sys::Win32::Security::Cryptography::{
CryptProtectData, CryptUnprotectData, CRYPTPROTECT_UI_FORBIDDEN, CRYPT_INTEGER_BLOB,
};
let data_in = CRYPT_INTEGER_BLOB { cbData: input.len() as u32, pbData: input.as_ptr() as *mut u8 };
let mut out = CRYPT_INTEGER_BLOB { cbData: 0, pbData: std::ptr::null_mut() };
// SAFETY: the input blob points at `input` for the call; the output is freed with LocalFree.
let ok = unsafe {
if encrypt {
CryptProtectData(&data_in, std::ptr::null(), std::ptr::null(), std::ptr::null(), std::ptr::null(),
CRYPTPROTECT_UI_FORBIDDEN, &mut out)
} else {
CryptUnprotectData(&data_in, std::ptr::null_mut(), std::ptr::null(), std::ptr::null(), std::ptr::null(),
CRYPTPROTECT_UI_FORBIDDEN, &mut out)
}
};
if ok == 0 {
return Err(SecretsError(if encrypt { "could not encrypt the secrets".into() } else {
"could not decrypt the secrets (another Windows user or PC?)".into() }));
}
// SAFETY: DPAPI returned `cbData` bytes at `pbData`.
let v = unsafe { std::slice::from_raw_parts(out.pbData, out.cbData as usize).to_vec() };
unsafe { LocalFree(out.pbData as _) };
Ok(v)
}
#[cfg(windows)]
fn protect(plain: &[u8]) -> Result<Vec<u8>, SecretsError> {
dpapi(plain, true)
}
#[cfg(windows)]
fn unprotect(bytes: &[u8]) -> Result<Vec<u8>, SecretsError> {
dpapi(bytes, false)
}
#[cfg(test)]
mod tests {
use super::*;
use crate::datadir::temp_dir;
#[test]
fn secrets_round_trip_and_are_removed() {
let s = Secrets::new(&temp_dir("sec").join("config").join("secrets.dat"));
assert_eq!(s.get(DISCORD_TOKEN).unwrap(), None);
s.set(DISCORD_TOKEN, "abc.def.ghij").unwrap();
s.set("other", "x").unwrap();
assert_eq!(s.get(DISCORD_TOKEN).unwrap().as_deref(), Some("abc.def.ghij"));
let mut v = s.values();
v.sort();
assert_eq!(v, vec!["abc.def.ghij", "x"]);
s.remove(DISCORD_TOKEN).unwrap();
assert_eq!(s.get(DISCORD_TOKEN).unwrap(), None);
assert_eq!(s.get("other").unwrap().as_deref(), Some("x"));
}
#[cfg(unix)]
#[test]
fn the_file_is_readable_by_its_owner_only() {
use std::os::unix::fs::PermissionsExt;
let f = temp_dir("secp").join("secrets.dat");
Secrets::new(&f).set(DISCORD_TOKEN, "t").unwrap();
assert_eq!(std::fs::metadata(&f).unwrap().permissions().mode() & 0o777, 0o600);
}
#[test]
fn a_damaged_file_is_reported_not_overwritten() {
let f = temp_dir("secd").join("secrets.dat");
std::fs::write(&f, b"not json").unwrap();
let s = Secrets::new(&f);
assert!(s.get(DISCORD_TOKEN).is_err());
assert!(s.set(DISCORD_TOKEN, "t").is_err());
assert_eq!(std::fs::read(&f).unwrap(), b"not json");
}
#[test]
fn masking_shows_only_the_last_four() {
assert_eq!(masked("MTIz.abcd.wxyz"), "••••wxyz");
assert_eq!(masked("ab"), "••••ab");
}
}

View File

@@ -11,7 +11,6 @@ use crate::discord::{CoreEvent, DiscordCore};
use crate::settings::load_settings;
use crate::worker_client::{WorkerClient, WorkerClientOptions};
use crate::worker_process::{ProcessEvent, WorkerProcess, WorkerProcessOptions};
use crate::debug_log;
pub struct Services {
pub core: DiscordCore,
@@ -30,8 +29,13 @@ impl Services {
pub async fn start_services(data_dir: &Path) -> Result<Services, Box<dyn std::error::Error + Send + Sync>> {
let settings = load_settings(&data_dir.join("config").join("settings.json"))?;
// Until the setup wizard stores it encrypted (M5), the token comes from the environment only.
let token = std::env::var("JARVIS_DISCORD_TOKEN").map_err(|_| "JARVIS_DISCORD_TOKEN is not set")?;
// The stored token (the setup wizard keeps it encrypted, admin.spec.md), else the environment.
let stored = crate::secrets::Secrets::new(&data_dir.join("config").join("secrets.dat"))
.get(crate::secrets::DISCORD_TOKEN)?;
let token = match stored {
Some(t) => t,
None => std::env::var("JARVIS_DISCORD_TOKEN").map_err(|_| "no bot token: set it in the app or JARVIS_DISCORD_TOKEN")?,
};
let w = &settings.voice_worker;
let worker_process = WorkerProcess::new(WorkerProcessOptions::new(&w.python, &w.cwd, w.args.clone()))?;
@@ -40,6 +44,7 @@ pub async fn start_services(data_dir: &Path) -> Result<Services, Box<dyn std::er
let mut opts = WorkerClientOptions::new(worker_process.url(), worker_process.token.clone());
opts.stt = settings.voice.stt.clone();
opts.stt.insert("hotwords".into(), Value::from(settings.voice.wake_words.clone()));
opts.memory = settings.memory.clone();
let (worker, worker_events) = WorkerClient::new(opts);
// The worker loads its models before it listens: connect once it says READY (after restarts the
// client keeps retrying on its own).
@@ -67,6 +72,6 @@ pub async fn start_services(data_dir: &Path) -> Result<Services, Box<dyn std::er
worker_process.stop().await;
return Err(Box::new(e));
}
debug_log!("app", "services started");
crate::info_log!("app", "services started");
Ok(Services { core, events, worker, worker_process })
}

View File

@@ -81,6 +81,9 @@ pub struct Settings {
/// Wake words for chat; `None` uses `voice.wakeWords`.
pub chat_wake_words: Option<Vec<String>>,
pub response: PolicySettings,
/// The voice worker's memory settings (voice_worker.spec.md, "Memory settings"), sent in its
/// `hello`; the worker validates them, the settings screens check them first (admin.spec.md).
pub memory: Map<String, Value>,
}
#[derive(Debug, Clone, PartialEq, Eq)]
@@ -122,6 +125,7 @@ impl Default for Settings {
call: CallSettings::default(),
chat_wake_words: None,
response: PolicySettings::default(),
memory: Map::new(),
}
}
}
@@ -390,6 +394,11 @@ pub fn parse_settings(raw: &Value) -> Result<Settings, SettingsError> {
call,
chat_wake_words,
response,
memory: match raw.get("memory") {
None => Map::new(),
Some(Value::Object(m)) => m.clone(),
Some(_) => return err("memory must be an object"),
},
})
}

325
app/src/settings_edit.rs Normal file
View File

@@ -0,0 +1,325 @@
//! Editing `config/settings.json` for the settings screens (`app/src/admin.spec.md`).
//!
//! The screens are built from `fields()`. `apply` changes only the keys it is given, writes only
//! values that differ from the default, keeps keys the app does not know, and writes nothing unless
//! the whole result passes the app's own check (`parse_settings`).
use std::path::Path;
use serde_json::{json, Map, Value};
use crate::datadir::write_atomic;
use crate::settings::{parse_settings, DEFAULT_JOIN_NOTICE};
#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Serialize)]
#[serde(rename_all = "camelCase")]
pub enum Screen {
Bot,
Channels,
Chat,
Memory,
Model,
}
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
#[serde(tag = "type", rename_all = "camelCase")]
pub enum Kind {
Bool,
Int { min: i64, max: i64 },
Number { min: f64, max: f64 },
/// `nullable`: null is allowed (it means "not set").
NullableNumber { min_exclusive: f64 },
Text { max: usize, non_empty: bool },
/// `nullable`: null means "use another setting" (chat wake words: the voice ones).
TextList { max_items: usize, nullable: bool },
Choice { values: &'static [&'static str] },
/// A language code (2-8 letters), or null for "detect it".
LanguageCode,
/// A Discord ID as a JSON string, or null.
Id,
IdList,
/// Discord user ID -> integer (response priorities).
IdNumberMap,
/// The registered chat channels (their own rules are checked by `parse_settings`).
ChatChannels,
}
#[derive(Debug, Clone, PartialEq, serde::Serialize)]
pub struct Field {
pub path: &'static str,
pub screen: Screen,
#[serde(flatten)]
pub kind: Kind,
pub default: Value,
pub label: &'static str,
}
fn f(path: &'static str, screen: Screen, kind: Kind, default: Value, label: &'static str) -> Field {
Field { path, screen, kind, default, label }
}
/// Every key the screens edit, in screen order.
pub fn fields() -> Vec<Field> {
use Kind::*;
use Screen::*;
vec![
f("discord.commandName", Bot, Text { max: 32, non_empty: true }, json!("자비스"), "슬래시 명령 이름"),
f("discord.ephemeralReplies", Bot, Bool, json!(true), "명령 응답은 부른 사람만 보기"),
f("discord.guildId", Channels, Id, Value::Null, "서버"),
f("discord.voiceChannelId", Channels, Id, Value::Null, "자동으로 들어갈 음성 채널"),
f("voice.maxSpeakers", Channels, Int { min: 1, max: 10 }, json!(10), "동시에 받아 적을 최대 인원"),
f("voice.listenToBots", Channels, Bool, json!(false), "다른 봇의 말도 받아 적기"),
f("voice.wakeWords", Channels, TextList { max_items: 20, nullable: false }, json!(["자비스"]), "호출어"),
f("call.mode", Channels, Choice { values: &["wakeWord", "addressed", "everything"] }, json!("wakeWord"),
"호출 방식"),
f("call.similarity", Channels, Number { min: 0.0, max: 0.5 }, json!(0.2), "비슷한 발음 허용 정도"),
f("call.designated", Channels, IdList, json!([]), "지정한 사람"),
f("call.designatedOnly", Channels, Bool, json!(false), "지정한 사람에게만 응답"),
f("call.designatedNoWakeWord", Channels, Bool, json!(false), "지정한 사람은 호출어 없이 대답"),
f("call.stopWords", Channels, TextList { max_items: 20, nullable: false },
json!(["멈춰", "그만", "정지", "stop"]), "정지 명령"),
f("call.followUpMs", Channels, Int { min: 0, max: 5000 }, json!(1500), "호출어 뒤 이어 말하기 대기(ms)"),
f("call.earcon", Channels, Bool, json!(true), "부르면 효과음"),
f("response.policy", Channels, Choice { values: &["first", "inOrder", "latest", "important"] },
json!("first"), "동시 대화 응답 정책"),
f("response.queueMax", Channels, Int { min: 1, max: 20 }, json!(5), "기다리는 요청 최대 개수"),
f("response.collectMs", Channels, Int { min: 0, max: 3000 }, json!(700), "중요도 비교 대기(ms)"),
f("response.settleMs", Channels, Int { min: 0, max: 3000 }, json!(800), "먼저 끝난 말 전사 대기(ms)"),
f("response.priorities", Channels, IdNumberMap, json!({}), "사람별 우선순위"),
f("voice.joinNotice", Channels, Text { max: 2000, non_empty: true }, json!(DEFAULT_JOIN_NOTICE),
"입장 시 전사 고지 문구"),
f("chatChannels", Chat, ChatChannels, json!([]), "채팅 채널"),
f("call.chatWakeWords", Chat, TextList { max_items: 20, nullable: true }, Value::Null,
"채팅 호출어(비우면 음성과 같음)"),
// The worker's memory settings and their defaults (voice_worker.spec.md, "Memory settings").
f("memory.chat", Memory, Bool, json!(true), "등록한 채팅 채널 참고"),
f("memory.others", Memory, Bool, json!(true), "같은 음성 채널의 다른 사람 말 참고"),
f("memory.off", Memory, Bool, json!(false), "지난 대화 참고 끄기"),
f("memory.since_days", Memory, NullableNumber { min_exclusive: 0.0 }, Value::Null,
"참고할 기간(일, 비우면 전체)"),
f("memory.recent_minutes", Memory, Number { min: 0.0, max: 1440.0 }, json!(30), "최근 대화 원문 범위(분)"),
f("memory.recent_records", Memory, Int { min: 0, max: 200 }, json!(20), "최근 대화 원문 개수"),
f("memory.matches", Memory, Int { min: 0, max: 20 }, json!(5), "과거 기록 가져올 개수"),
f("memory.budget_tokens", Memory, Number { min: 0.0, max: 100000.0 }, json!(4000), "참고 분량 상한(토큰)"),
// The worker's STT settings and their defaults (voice_worker.spec.md, "STT").
f("voice.stt.end_silence_ms", Model, Int { min: 100, max: 2000 }, json!(300), "말 끝 판정 시간(ms)"),
f("voice.stt.partials", Model, Bool, json!(true), "중간 전사"),
f("voice.stt.partial_interval_ms", Model, Int { min: 100, max: 5000 }, json!(400), "중간 전사 간격(ms)"),
f("voice.stt.final_beam", Model, Int { min: 1, max: 5 }, json!(2), "최종 전사 정확도(빔)"),
f("voice.stt.language", Model, LanguageCode, json!("ko"), "받아 적을 언어(비우면 자동)"),
]
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct EditError(pub String);
impl std::fmt::Display for EditError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(&self.0)
}
}
impl std::error::Error for EditError {}
fn load_raw(file: &Path) -> Result<Map<String, Value>, EditError> {
let text = match std::fs::read_to_string(file) {
Ok(t) => t,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Map::new()),
Err(e) => return Err(EditError(format!("could not read {}: {e}", file.display()))),
};
match serde_json::from_str(text.trim_start_matches('\u{feff}')) {
Ok(Value::Object(m)) => Ok(m),
Ok(_) => Err(EditError(format!("{} is not a JSON object", file.display()))),
Err(e) => Err(EditError(format!("{} is not valid JSON: {e}", file.display()))),
}
}
fn get_path<'a>(raw: &'a Map<String, Value>, path: &str) -> Option<&'a Value> {
let mut parts = path.split('.');
let mut cur = raw.get(parts.next()?)?;
for p in parts {
cur = cur.as_object()?.get(p)?;
}
Some(cur)
}
fn set_path(raw: &mut Map<String, Value>, path: &str, value: Value) -> Result<(), EditError> {
let parts: Vec<&str> = path.split('.').collect();
let mut cur = raw;
for p in &parts[..parts.len() - 1] {
let entry = cur.entry(p.to_string()).or_insert_with(|| Value::Object(Map::new()));
cur = entry.as_object_mut().ok_or_else(|| EditError(format!("{p} in the file is not an object")))?;
}
cur.insert(parts[parts.len() - 1].into(), value);
Ok(())
}
/// Remove `path`, then every parent object it leaves empty.
fn remove_path(raw: &mut Map<String, Value>, path: &str) {
fn go(m: &mut Map<String, Value>, parts: &[&str]) {
if parts.len() == 1 {
m.remove(parts[0]);
return;
}
if let Some(Value::Object(child)) = m.get_mut(parts[0]) {
go(child, &parts[1..]);
if child.is_empty() {
m.remove(parts[0]);
}
}
}
go(raw, &path.split('.').collect::<Vec<_>>());
}
fn is_id(v: &Value) -> bool {
v.as_str().is_some_and(|s| (17..=20).contains(&s.len()) && s.bytes().all(|b| b.is_ascii_digit()))
}
fn check(field: &Field, v: &Value) -> Result<(), String> {
let ok = match &field.kind {
Kind::Bool => v.is_boolean(),
Kind::Int { min, max } => v.as_i64().is_some_and(|n| (*min..=*max).contains(&n)) && !v.is_f64(),
Kind::Number { min, max } => v.as_f64().is_some_and(|n| (*min..=*max).contains(&n)),
Kind::NullableNumber { min_exclusive } => v.is_null() || v.as_f64().is_some_and(|n| n > *min_exclusive),
Kind::Text { max, non_empty } => v.as_str().is_some_and(|s| {
let n = s.trim().chars().count();
n <= *max && (!non_empty || n > 0)
}),
Kind::TextList { max_items, nullable } => (*nullable && v.is_null())
|| v.as_array().is_some_and(|a| a.len() <= *max_items && a.iter().all(|w| w.as_str().is_some_and(|s| {
let n = s.trim().chars().count();
(1..=64).contains(&n)
}))),
Kind::Choice { values } => v.as_str().is_some_and(|s| values.contains(&s)),
Kind::LanguageCode => v.is_null()
|| v.as_str().is_some_and(|s| (2..=8).contains(&s.len()) && s.chars().all(|c| c.is_ascii_alphabetic() || c == '-')),
Kind::Id => v.is_null() || is_id(v),
Kind::IdList => v.as_array().is_some_and(|a| a.iter().all(is_id)),
Kind::IdNumberMap => v.as_object().is_some_and(|m| m.iter()
.all(|(k, n)| is_id(&Value::String(k.clone())) && n.as_i64().is_some())),
Kind::ChatChannels => v.is_array(),
};
if ok { Ok(()) } else { Err(format!("{} has a value it cannot take", field.path)) }
}
/// Every field's value now: the file's, else the default.
pub fn view(settings_file: &Path) -> Result<Map<String, Value>, EditError> {
let raw = load_raw(settings_file)?;
Ok(fields().into_iter()
.map(|f| (f.path.to_owned(), get_path(&raw, f.path).cloned().unwrap_or(f.default)))
.collect())
}
/// Apply `{path: value}` to the file; all or nothing.
pub fn apply(settings_file: &Path, changes: &Map<String, Value>) -> Result<(), EditError> {
let mut raw = load_raw(settings_file)?;
let all = fields();
for (path, value) in changes {
let field = all.iter().find(|f| f.path == path)
.ok_or_else(|| EditError(format!("{path} is not a setting the screens edit")))?;
check(field, value).map_err(EditError)?;
if *value == field.default {
remove_path(&mut raw, path);
} else {
set_path(&mut raw, path, value.clone())?;
}
}
let doc = Value::Object(raw);
parse_settings(&doc).map_err(|e| EditError(e.0))?;
let mut text = serde_json::to_string_pretty(&doc).expect("JSON serialises");
text.push('\n');
write_atomic(settings_file, text.as_bytes())
.map_err(|e| EditError(format!("could not write {}: {e}", settings_file.display())))?;
crate::debug_log!("admin", "settings changed: {}", changes.keys().cloned().collect::<Vec<_>>().join(", "));
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::datadir::temp_dir;
use crate::settings::Settings;
fn file(tag: &str) -> std::path::PathBuf {
temp_dir(tag).join("settings.json")
}
fn changes(v: Value) -> Map<String, Value> {
v.as_object().unwrap().clone()
}
fn read(p: &Path) -> Value {
serde_json::from_str(&std::fs::read_to_string(p).unwrap()).unwrap()
}
#[test]
fn every_default_is_the_apps_own_default() {
// Writing a field's default out explicitly must give the same settings as leaving it out.
for f in fields() {
let mut raw = Map::new();
set_path(&mut raw, f.path, f.default.clone()).unwrap();
let parsed = parse_settings(&Value::Object(raw)).unwrap_or_else(|e| panic!("{}: {e}", f.path));
let mut d = Settings::default();
// memory and stt pass to the worker as given; their defaults are the worker's.
d.memory = parsed.memory.clone();
d.voice.stt = parsed.voice.stt.clone();
assert_eq!(parsed, d, "{}", f.path);
}
}
#[test]
fn view_shows_the_file_over_the_defaults() {
let p = file("sev");
std::fs::write(&p, r#"{"call": {"mode": "addressed"}, "unknown": 1}"#).unwrap();
let v = view(&p).unwrap();
assert_eq!(v["call.mode"], json!("addressed"));
assert_eq!(v["voice.maxSpeakers"], json!(10));
assert_eq!(v.len(), fields().len());
}
#[test]
fn apply_writes_only_non_defaults_and_keeps_unknown_keys() {
let p = file("sea");
std::fs::write(&p, r#"{"voiceWorker": {"python": "/venv/python"}, "mine": {"x": 1}}"#).unwrap();
apply(&p, &changes(json!({"call.mode": "everything", "voice.maxSpeakers": 10,
"memory.recent_records": 5, "voice.stt.end_silence_ms": 400}))).unwrap();
assert_eq!(read(&p), json!({"voiceWorker": {"python": "/venv/python"}, "mine": {"x": 1},
"call": {"mode": "everything"}, "memory": {"recent_records": 5}, "voice": {"stt": {"end_silence_ms": 400}}}));
// Back to the default: the key goes, and so do sections it leaves empty.
apply(&p, &changes(json!({"call.mode": "wakeWord", "voice.stt.end_silence_ms": 300}))).unwrap();
assert_eq!(read(&p), json!({"voiceWorker": {"python": "/venv/python"}, "mine": {"x": 1},
"memory": {"recent_records": 5}}));
}
#[test]
fn a_bad_value_changes_nothing() {
let p = file("seb");
std::fs::write(&p, r#"{"call": {"mode": "addressed"}}"#).unwrap();
let before = std::fs::read(&p).unwrap();
for (c, key) in [
(json!({"voice.maxSpeakers": 11}), "voice.maxSpeakers"),
(json!({"voice.maxSpeakers": 2.5}), "voice.maxSpeakers"),
(json!({"call.mode": "sometimes"}), "call.mode"),
(json!({"discord.guildId": 123}), "discord.guildId"),
(json!({"voice.joinNotice": " "}), "voice.joinNotice"),
(json!({"memory.since_days": 0}), "memory.since_days"),
(json!({"voice.stt.language": ""}), "voice.stt.language"),
(json!({"nope": 1}), "nope"),
// Passes the field check, refused by the app's own check: a bad command name.
(json!({"discord.commandName": "Big Name"}), "discord.commandName"),
// A chat channel without a valid ID.
(json!({"chatChannels": [{"id": "1"}]}), "chatChannels[0].id"),
] {
let e = apply(&p, &changes(c.clone()).into_iter().chain([("call.earcon".into(), json!(false))]).collect())
.unwrap_err();
assert!(e.0.contains(key), "{c} -> {e}");
assert_eq!(std::fs::read(&p).unwrap(), before, "{c}");
}
}
#[test]
fn a_file_that_does_not_parse_is_reported_not_overwritten() {
let p = file("sep");
std::fs::write(&p, "{ not json").unwrap();
assert!(view(&p).is_err());
assert!(apply(&p, &changes(json!({"call.earcon": false}))).is_err());
assert_eq!(std::fs::read_to_string(&p).unwrap(), "{ not json");
}
}

View File

@@ -35,6 +35,8 @@ pub struct WorkerClientOptions {
pub audio: PcmFormat,
pub tts: PcmFormat,
pub stt: Map<String, Value>,
/// The worker's memory settings, sent in `hello` (voice_worker.spec.md).
pub memory: Map<String, Value>,
/// First reconnect delay; doubles up to `max_backoff`.
pub backoff: Duration,
pub max_backoff: Duration,
@@ -49,6 +51,7 @@ impl WorkerClientOptions {
audio: DISCORD_PCM,
tts: DISCORD_PCM,
stt: Map::new(),
memory: Map::new(),
backoff: Duration::from_millis(500),
max_backoff: Duration::from_secs(5),
hello_timeout: Duration::from_secs(5),
@@ -436,9 +439,12 @@ async fn connect_once(inner: &Arc<Inner>) -> bool {
let (mut write, mut read) = ws.split();
let stt = inner.state.lock().unwrap().stt.clone();
let o = &inner.opts;
let hello = json!({"type": "hello", "token": o.token,
let mut hello = json!({"type": "hello", "token": o.token,
"audio": {"rate": o.audio.rate, "channels": o.audio.channels},
"tts": {"rate": o.tts.rate, "channels": o.tts.channels}, "stt": stt});
if !o.memory.is_empty() {
hello["memory"] = Value::Object(o.memory.clone());
}
if write.send(Message::text(hello.to_string())).await.is_err() {
return false;
}

View File

@@ -14,7 +14,7 @@ use tokio::io::{AsyncBufReadExt, BufReader};
use tokio::process::{Child, Command};
use tokio::sync::{broadcast, Notify};
use crate::{debug_log, error_log};
use crate::error_log;
#[derive(Debug, Clone)]
pub struct WorkerProcessOptions {
@@ -124,7 +124,7 @@ impl WorkerProcess {
}
};
shared.lock().unwrap().child_pid = child.id();
debug_log!("worker", "voice worker started pid={:?} port={port}", child.id());
crate::info_log!("worker", "voice worker started pid={:?} port={port}", child.id());
let mut ready_rx = events.subscribe();
// The worker's own logs hold timings and counts only (voice_worker.spec.md).
if let Some(out) = child.stdout.take() {
@@ -133,10 +133,10 @@ impl WorkerProcess {
let mut lines = BufReader::new(out).lines();
while let Ok(Some(line)) = lines.next_line().await {
if line.starts_with("READY ") {
debug_log!("worker", "voice worker READY after {} ms", started.elapsed().as_millis());
crate::info_log!("worker", "voice worker READY after {} ms", started.elapsed().as_millis());
let _ = events.send(ProcessEvent::Ready);
} else {
debug_log!("worker", "{line}");
worker_line(&line);
}
}
});
@@ -145,7 +145,7 @@ impl WorkerProcess {
tokio::spawn(async move {
let mut lines = BufReader::new(err).lines();
while let Ok(Some(line)) = lines.next_line().await {
debug_log!("worker", "{line}");
worker_line(&line);
}
});
}
@@ -222,3 +222,14 @@ fn spawn(opts: &WorkerProcessOptions, token: &str, port: u16) -> std::io::Result
}
cmd.spawn()
}
/// A line the voice worker printed: on stderr with debug logging, and always to the log hub (its
/// own category picks the tab; warnings and tracebacks reach the log screen without debug logging).
fn worker_line(line: &str) {
if crate::log::debug_enabled() {
eprintln!("{}", crate::log::line("worker", line));
}
if let Some(hub) = crate::logs::installed() {
hub.push_worker(line);
}
}

View File

@@ -154,6 +154,19 @@ async fn hello_carries_token_formats_and_stt_then_audio_flows() {
c.close();
}
#[tokio::test]
async fn hello_carries_the_memory_settings_when_there_are_any() {
let w = FakeWorker::start(0).await;
let mut o = WorkerClientOptions::new(w.url(), TOKEN);
o.memory = json!({"others": false, "recent_records": 5}).as_object().cloned().unwrap();
let (c, _rx) = WorkerClient::new(o);
c.start();
c.wait_ready(Duration::from_secs(2)).await.expect("ready");
let Got::Json(hello) = &w.got()[0] else { panic!("hello first") };
assert_eq!(hello["memory"], json!({"others": false, "recent_records": 5}));
c.close();
}
#[tokio::test]
async fn events_pass_on_and_only_tts_audio_of_open_requests() {
let w = FakeWorker::start(0).await;

File diff suppressed because one or more lines are too long