fix(app): 대화 screen pages back without limit, shows edits and drops stale answers
- history_query takes item cursors (before / from) ordered by time, source and id, so paging never skips or repeats an item and has no 2,000 item ceiling - live refresh re-reads the window from the oldest item shown and compares whole items, so an edited message is redrawn - answers to requests made before a filter change, and live answers older than the newest one asked, are dropped Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -232,11 +232,14 @@ records and "Traces"), so it works while the services are stopped. Nothing it re
|
||||
`thought` (a trace: the question, the answer or none, how long it took and its steps). Jarvis's
|
||||
chat messages are known by the bot's user id (the first part of the stored token is that id in
|
||||
base64) and by the author of any chat answer to someone.
|
||||
- `history_query` `{said, answered, thought, via, person, channel, find, since, until, limit}`: the
|
||||
- `history_query` `{said, answered, thought, via, person, channel, find, since, until, before, from, limit}`: the
|
||||
three kinds (each on unless false), where (`voice`, `chat`, `task`; a task has traces only), a
|
||||
person (their records, answers to them and traces of their questions), a channel, text the record
|
||||
or the trace's question, answer or steps contains (records folded like the store: NFKC, lower
|
||||
case), Unix seconds from (inclusive) and to (exclusive), and `limit` (1-2000, default 200). Answers
|
||||
case), Unix seconds from (inclusive) and to (exclusive), the cursors `before` (only items older
|
||||
than it) and `from` (it and newer), each an item's `{at, key}`, and `limit` (1-2000, default 200).
|
||||
Items are ordered by time, then records before traces, then id, so paging by cursor never skips or
|
||||
repeats an item, even at equal times. Answers
|
||||
`{items, more, people, channels}`: the newest `limit` items in time order, whether older ones
|
||||
were left out, the people who spoke (their latest name) and the channels with records (with the
|
||||
name given a chat channel in the settings). Without the file: no items and `missing: true`.
|
||||
@@ -684,7 +687,7 @@ back as Korean messages for the screen. The Tauri shell passes the screens' `inv
|
||||
| `service_start` / `service_stop` / `service_restart` / `dashboard` | the controller and its snapshot; start and restart run in the background, stop also cancels a start |
|
||||
| `discord_channel_info` `{id}` / `channel_sync_status` | a channel's Discord name and topic / each synced chat channel's last result (above) |
|
||||
| `notes_list` / `note_update` `{id, text}` / `note_delete` `{id}` | everyone's personal notes (above) / change one's text / delete one |
|
||||
| `history_query` `{said, answered, thought, via, person, channel, find, since, until, limit}` | the 대화 screen's records and answer traces (above, "Conversations") |
|
||||
| `history_query` `{said, answered, thought, via, person, channel, find, since, until, before, from, limit}` | the 대화 screen's records and answer traces (above, "Conversations") |
|
||||
| `activity_list` `{limit}` | what Jarvis did (above, "Activity"), newest first |
|
||||
| `schedules_list` / `schedule_delete` `{id}` | every person's schedules (above), soonest first, with IDs as strings / delete one |
|
||||
| `collect_results` `{target, limit}` | `{targets, items}`: every target folder with results, and the collected items newest first (`target` null: every target; `limit` 1-500, default 100) |
|
||||
@@ -736,9 +739,12 @@ Plain HTML, CSS and an ES module, no build step, dark theme after `DESIGN.md`. A
|
||||
notes it was given, the tools offered, the plan, the model's thinking, each tool call (name,
|
||||
arguments, success, failure or skipped and why, its result) and the answer. Filters: the three
|
||||
kinds as chips, where (음성, 채팅, 정기 작업), person, channel, one day and text; "실시간" (on by
|
||||
default) refreshes every 3 seconds and redraws only when something changed, keeping the cards the
|
||||
user opened open and the scroll where it was unless it was at the end; "이전 기록 더 보기" adds 200
|
||||
older items; "생각 모두 펼치기/접기"; a count of each kind shown.
|
||||
default) re-reads every item from the oldest one shown on (`from`, up to 2,000; older pages beyond
|
||||
that stay as loaded) every 3 seconds and redraws only when an item changed, an edited message
|
||||
included, keeping the cards the user opened open and the scroll where it was unless it was at the
|
||||
end; "이전 기록 더 보기" adds the 200 items before the oldest shown (`before`), with no end to how far
|
||||
back it goes; a filter change drops every answer to a request made before it, and a live answer
|
||||
older than the newest one asked is dropped; "생각 모두 펼치기/접기"; a count of each kind shown.
|
||||
- **한 일**: the activity (time, kind, target, who asked, outcome and why it failed), newest 200,
|
||||
with a refresh button.
|
||||
- **업데이트**: the version here, the state, "다시 확인", "받기" (with its percentage) and "설치하고
|
||||
|
||||
@@ -405,6 +405,8 @@ impl AdminApi {
|
||||
said: flag("said"), answered: flag("answered"), thought: flag("thought"),
|
||||
via: text("via"), person: text("person"), channel: text("channel"), find: text("find"),
|
||||
since: args.get("since").and_then(Value::as_f64), until: args.get("until").and_then(Value::as_f64),
|
||||
before: args.get("before").and_then(crate::history_view::Cursor::of),
|
||||
from: args.get("from").and_then(crate::history_view::Cursor::of),
|
||||
limit: args.get("limit").and_then(Value::as_u64).map_or(crate::history_view::DEFAULT_LIMIT, |n| n as usize),
|
||||
};
|
||||
let bot = self.secrets()?.get(DISCORD_TOKEN).ok().flatten().and_then(|t| crate::history_view::bot_id_of_token(&t));
|
||||
|
||||
@@ -32,13 +32,61 @@ pub struct Query {
|
||||
/// Unix seconds, inclusive / exclusive.
|
||||
pub since: Option<f64>,
|
||||
pub until: Option<f64>,
|
||||
/// Only items older than this one (paging back) / this one and newer (the window shown).
|
||||
pub before: Option<Cursor>,
|
||||
pub from: Option<Cursor>,
|
||||
pub limit: usize,
|
||||
}
|
||||
|
||||
/// An item's place in the order: time, then records before traces, then id. Items are ordered by
|
||||
/// it, so paging by it never skips or repeats one, even when two share a time.
|
||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||
pub struct Cursor {
|
||||
pub at: f64,
|
||||
/// 0: a record (`h<id>`), 1: a trace (`t<id>`).
|
||||
pub src: u8,
|
||||
pub id: i64,
|
||||
}
|
||||
|
||||
impl Cursor {
|
||||
/// From `{at, key}`, as the screen sends an item back.
|
||||
pub fn of(v: &Value) -> Option<Self> {
|
||||
let at = v.get("at")?.as_f64()?;
|
||||
let key = v.get("key")?.as_str()?;
|
||||
let src = match key.get(..1)? { "h" => 0, "t" => 1, _ => return None };
|
||||
Some(Self { at, src, id: key[1..].parse().ok()? })
|
||||
}
|
||||
|
||||
/// SQL for "a row of table `src` comes before this cursor", over columns `{p}ts` and `{p}id`.
|
||||
fn before_sql(&self, src: u8, p: &str) -> (String, Vec<Sql>) {
|
||||
use std::cmp::Ordering::*;
|
||||
match src.cmp(&self.src) {
|
||||
Less => (format!("{p}ts <= ?"), vec![Sql::Real(self.at)]),
|
||||
Greater => (format!("{p}ts < ?"), vec![Sql::Real(self.at)]),
|
||||
Equal => (format!("({p}ts < ? OR ({p}ts = ? AND {p}id < ?))"),
|
||||
vec![Sql::Real(self.at), Sql::Real(self.at), Sql::Integer(self.id)]),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// The cursor conditions of `q` for table `src`, appended to `sql` and `args`.
|
||||
fn cursor_sql(q: &Query, src: u8, p: &str, sql: &mut String, args: &mut Vec<Sql>) {
|
||||
if let Some(c) = &q.before {
|
||||
let (cond, a) = c.before_sql(src, p);
|
||||
*sql += &format!(" AND {cond}");
|
||||
args.extend(a);
|
||||
}
|
||||
if let Some(c) = &q.from {
|
||||
let (cond, a) = c.before_sql(src, p);
|
||||
*sql += &format!(" AND NOT {cond}");
|
||||
args.extend(a);
|
||||
}
|
||||
}
|
||||
|
||||
impl Default for Query {
|
||||
fn default() -> Self {
|
||||
Self { said: true, answered: true, thought: true, via: None, person: None, channel: None, find: None,
|
||||
since: None, until: None, limit: DEFAULT_LIMIT }
|
||||
since: None, until: None, before: None, from: None, limit: DEFAULT_LIMIT }
|
||||
}
|
||||
}
|
||||
|
||||
@@ -98,7 +146,7 @@ pub fn query(path: &Path, bot: Option<&str>, q: &Query) -> Result<Value, String>
|
||||
};
|
||||
let name_of = |id: &str| names.get(id).cloned().unwrap_or_default();
|
||||
|
||||
let mut items: Vec<(f64, Value)> = Vec::new();
|
||||
let mut items: Vec<((f64, u8, i64), Value)> = Vec::new();
|
||||
let mut more = false;
|
||||
if q.said || q.answered {
|
||||
let mut sql = format!("SELECT h.id, h.ts, h.kind, h.channel_id, h.author_id, h.author_name, h.to_id, h.text, \
|
||||
@@ -129,13 +177,14 @@ pub fn query(path: &Path, bot: Option<&str>, q: &Query) -> Result<Value, String>
|
||||
}
|
||||
if let Some(t) = q.since { sql += " AND h.ts >= ?"; args.push(Sql::Real(t)); }
|
||||
if let Some(t) = q.until { sql += " AND h.ts < ?"; args.push(Sql::Real(t)); }
|
||||
cursor_sql(q, 0, "h.", &mut sql, &mut args);
|
||||
sql += &format!(" ORDER BY h.ts DESC, h.id DESC LIMIT {}", limit + 1);
|
||||
let mut st = c.prepare(&sql).map_err(err)?;
|
||||
let rows = st.query_map(params_from_iter(args), |r| {
|
||||
let (id, ts, kind): (i64, f64, String) = (r.get(0)?, r.get(1)?, r.get(2)?);
|
||||
let (author, author_name, to): (String, String, Option<String>) = (r.get(4)?, r.get(5)?, r.get(6)?);
|
||||
let answer: bool = r.get(8)?;
|
||||
Ok((ts, json!({
|
||||
Ok(((ts, 0, id), json!({
|
||||
"key": format!("h{id}"), "type": if answer { "answered" } else { "said" }, "at": ts,
|
||||
"via": if kind == "chat" { "chat" } else { "voice" }, "channel": r.get::<_, String>(3)?,
|
||||
"who": {"id": author, "name": author_name},
|
||||
@@ -159,13 +208,14 @@ pub fn query(path: &Path, bot: Option<&str>, q: &Query) -> Result<Value, String>
|
||||
}
|
||||
if let Some(t) = q.since { sql += " AND ts >= ?"; args.push(Sql::Real(t)); }
|
||||
if let Some(t) = q.until { sql += " AND ts < ?"; args.push(Sql::Real(t)); }
|
||||
cursor_sql(q, 1, "", &mut sql, &mut args);
|
||||
let sql = format!("{sql} ORDER BY ts DESC, id DESC LIMIT {}", limit + 1);
|
||||
let mut st = c.prepare(&sql).map_err(err)?;
|
||||
let rows = st.query_map(params_from_iter(args), |r| {
|
||||
let (id, ts): (i64, f64) = (r.get(0)?, r.get(1)?);
|
||||
let user: String = r.get(4)?;
|
||||
let steps: String = r.get(8)?;
|
||||
Ok((ts, json!({
|
||||
Ok(((ts, 1, id), json!({
|
||||
"key": format!("t{id}"), "type": "thought", "at": ts, "via": r.get::<_, String>(2)?,
|
||||
"channel": r.get::<_, Option<String>>(3)?,
|
||||
"who": {"id": user.clone(), "name": name_of(&user)},
|
||||
@@ -177,7 +227,7 @@ pub fn query(path: &Path, bot: Option<&str>, q: &Query) -> Result<Value, String>
|
||||
more |= got.len() > limit;
|
||||
items.extend(got.into_iter().take(limit));
|
||||
}
|
||||
items.sort_by(|a, b| b.0.total_cmp(&a.0));
|
||||
items.sort_by(|a, b| b.0.0.total_cmp(&a.0.0).then(b.0.1.cmp(&a.0.1)).then(b.0.2.cmp(&a.0.2)));
|
||||
more |= items.len() > limit;
|
||||
items.truncate(limit);
|
||||
items.reverse();
|
||||
@@ -305,6 +355,35 @@ mod tests {
|
||||
assert_eq!(v["more"], true);
|
||||
}
|
||||
|
||||
/// Every item, walking back page by page with the cursor of the oldest one shown.
|
||||
#[test]
|
||||
fn pages_walk_back_through_everything_past_any_limit_even_at_equal_times() {
|
||||
let (path, c) = db("pages");
|
||||
for i in 0..7 {
|
||||
// Records and traces at the same moments, so a page edge falls inside equal times.
|
||||
rec(&c, (i / 2) as f64, "voice", "vc", "1", "Alice", None, &format!("말 {i}"));
|
||||
trace(&c, (i / 2) as f64, "voice", Some("vc"), "1", &format!("생각 {i}"), "", "[]");
|
||||
}
|
||||
let all = query(&path, None, &Query { limit: MAX_LIMIT, ..Query::default() }).unwrap()["items"].clone();
|
||||
let all: Vec<Value> = all.as_array().unwrap().clone();
|
||||
assert_eq!(all.len(), 14);
|
||||
let cursor = |i: &Value| Cursor::of(&json!({"at": i["at"], "key": i["key"]})).unwrap();
|
||||
let mut got: Vec<Value> = Vec::new();
|
||||
let mut before = None;
|
||||
loop {
|
||||
let v = query(&path, None, &Query { limit: 3, before, ..Query::default() }).unwrap();
|
||||
let page = v["items"].as_array().unwrap().clone();
|
||||
before = page.first().map(cursor);
|
||||
got.splice(0..0, page);
|
||||
if v["more"] == false { break; }
|
||||
}
|
||||
assert_eq!(got, all);
|
||||
// From a cursor on: that item and everything newer, whatever the limit was before.
|
||||
let from = query(&path, None, &Query { limit: MAX_LIMIT, from: Some(cursor(&all[5])), ..Query::default() }).unwrap();
|
||||
assert_eq!(from["items"].as_array().unwrap()[..], all[5..]);
|
||||
assert!(Cursor::of(&json!({"at": 1.0, "key": "x1"})).is_none());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_bot_id_comes_from_its_token() {
|
||||
assert_eq!(bot_id_of_token("MTIzNDU2Nzg5MDEyMzQ1Njc4.GxYzAb.abcdef").as_deref(), Some("123456789012345678"));
|
||||
|
||||
107
app/ui/app.js
107
app/ui/app.js
@@ -446,10 +446,16 @@ function conversationScreen() {
|
||||
const find = h("input", { type: "text", placeholder: "찾기 (말, 답, 생각)", style: "max-width:220px" });
|
||||
const live = h("input", { type: "checkbox", class: "switch", checked: true });
|
||||
const open = new Set(); // traces the user opened stay open across refreshes
|
||||
let expandAll = false, limit = 200, sig = null;
|
||||
// The window shown: every item from the cursor `from` on, re-read live (at most PAGE_MAX), plus
|
||||
// `older` pages that no longer fit it. Each filter change starts a new epoch; answers from an
|
||||
// earlier epoch, and live answers older than the newest one asked, are dropped.
|
||||
const PAGE = 200, PAGE_MAX = 2000;
|
||||
const cursorOf = (i) => ({ at: i.at, key: i.key });
|
||||
let expandAll = false, sig = null, shown = [], older = [], from = null, hasOlder = false, missing = false;
|
||||
let epoch = 0, liveSeq = 0, first = true;
|
||||
const box = h("div", { class: "talk" });
|
||||
const summary = h("div", { class: "small muted" });
|
||||
const more = h("button", { class: "btn", hidden: true, onclick: () => { limit += 200; load(true); } }, "이전 기록 더 보기");
|
||||
const more = h("button", { class: "btn", hidden: true, onclick: () => loadOlder() }, "이전 기록 더 보기");
|
||||
|
||||
const fill = (sel, list, label) => {
|
||||
const key = JSON.stringify(list);
|
||||
@@ -486,37 +492,78 @@ function conversationScreen() {
|
||||
h("div", { class: "text" }, i.text));
|
||||
}
|
||||
|
||||
async function load(keepTop = false) {
|
||||
function filters() {
|
||||
const since = day.value ? new Date(day.value + "T00:00:00").getTime() / 1000 : null;
|
||||
return { said: said.checked, answered: answered.checked, thought: thought.checked,
|
||||
via: via.value || null, person: person.value || null, channel: channel.value || null, find: find.value || null,
|
||||
since, until: since == null ? null : since + 86400 };
|
||||
}
|
||||
|
||||
function render(keepTop = false) {
|
||||
const items = older.concat(shown);
|
||||
// The whole items, so an edited message or a changed answer is redrawn too.
|
||||
const s = JSON.stringify([items, expandAll, hasOlder, missing]);
|
||||
if (s === sig) return;
|
||||
sig = s;
|
||||
const stick = box.scrollTop + box.clientHeight >= box.scrollHeight - 30, fromBottom = box.scrollHeight - box.scrollTop;
|
||||
const kids = [];
|
||||
let lastDay = null;
|
||||
for (const i of items) {
|
||||
const d = dayOf(i.at);
|
||||
if (d !== lastDay) { kids.push(h("div", { class: "day" }, d)); lastDay = d; }
|
||||
kids.push(item(i));
|
||||
}
|
||||
box.replaceChildren(...(kids.length ? kids : [h("div", { class: "muted" },
|
||||
missing ? "아직 대화 기록이 없습니다. 서비스를 켜고 자비스와 이야기하면 여기에 나옵니다." : "조건에 맞는 기록이 없습니다.")]));
|
||||
more.hidden = !hasOlder;
|
||||
const n = (t) => items.filter((i) => i.type === t).length;
|
||||
summary.textContent = `사람 말 ${n("said")} · 자비스 말 ${n("answered")} · 생각 ${n("thought")}` + (hasOlder ? " (이전 기록 더 있음)" : "");
|
||||
if (keepTop) box.scrollTop = box.scrollHeight - fromBottom;
|
||||
else if (stick || first) box.scrollTop = box.scrollHeight;
|
||||
first = false;
|
||||
}
|
||||
|
||||
function lists(r) {
|
||||
load.channels = r.channels;
|
||||
fill(person, r.people, (p) => `${p.name || p.id}`);
|
||||
fill(channel, r.channels, channelLabel);
|
||||
}
|
||||
|
||||
/** The window from `from` on (the newest page at first), re-read on every live tick. */
|
||||
async function load() {
|
||||
const ep = epoch, my = ++liveSeq;
|
||||
try {
|
||||
const since = day.value ? new Date(day.value + "T00:00:00").getTime() / 1000 : null;
|
||||
const r = await api("history_query", { said: said.checked, answered: answered.checked, thought: thought.checked,
|
||||
via: via.value || null, person: person.value || null, channel: channel.value || null, find: find.value || null,
|
||||
since, until: since == null ? null : since + 86400, limit });
|
||||
load.channels = r.channels;
|
||||
fill(person, r.people, (p) => `${p.name || p.id}`);
|
||||
fill(channel, r.channels, channelLabel);
|
||||
const s = JSON.stringify([r.items.map((i) => i.key + ":" + (i.text || i.answer || "").length), expandAll, r.more]);
|
||||
if (s === sig) return;
|
||||
sig = s;
|
||||
const stick = box.scrollTop + box.clientHeight >= box.scrollHeight - 30, fromBottom = box.scrollHeight - box.scrollTop;
|
||||
const kids = [];
|
||||
let lastDay = null;
|
||||
for (const i of r.items) {
|
||||
const d = dayOf(i.at);
|
||||
if (d !== lastDay) { kids.push(h("div", { class: "day" }, d)); lastDay = d; }
|
||||
kids.push(item(i));
|
||||
}
|
||||
box.replaceChildren(...(kids.length ? kids : [h("div", { class: "muted" },
|
||||
r.missing ? "아직 대화 기록이 없습니다. 서비스를 켜고 자비스와 이야기하면 여기에 나옵니다." : "조건에 맞는 기록이 없습니다.")]));
|
||||
more.hidden = !r.more;
|
||||
const n = (t) => r.items.filter((i) => i.type === t).length;
|
||||
summary.textContent = `사람 말 ${n("said")} · 자비스 말 ${n("answered")} · 생각 ${n("thought")}` + (r.more ? ` (최근 ${limit}개까지)` : "");
|
||||
if (keepTop) box.scrollTop = box.scrollHeight - fromBottom;
|
||||
else if (stick || !load.done) box.scrollTop = box.scrollHeight;
|
||||
load.done = true;
|
||||
const r = await api("history_query", { ...filters(), ...(from ? { from, limit: PAGE_MAX } : { limit: PAGE }) });
|
||||
if (ep !== epoch || my !== liveSeq) return;
|
||||
lists(r);
|
||||
missing = !!r.missing;
|
||||
if (!from) hasOlder = r.more;
|
||||
else if (r.more) { older = []; hasOlder = true; } // more new ones than a window holds: page back from them
|
||||
shown = r.items;
|
||||
if (shown.length) from = cursorOf(shown[0]);
|
||||
render();
|
||||
} catch (e) { fail(e); }
|
||||
}
|
||||
const reload = () => { limit = 200; sig = null; load.done = false; load(); };
|
||||
|
||||
/** The page before the oldest item shown; it joins the live window while that stays under PAGE_MAX. */
|
||||
async function loadOlder() {
|
||||
const oldest = older[0] || shown[0];
|
||||
if (!oldest) return;
|
||||
const ep = epoch;
|
||||
try {
|
||||
const r = await api("history_query", { ...filters(), before: cursorOf(oldest), limit: PAGE });
|
||||
if (ep !== epoch) return;
|
||||
++liveSeq; // a live answer asked with the old window would drop this page
|
||||
hasOlder = r.more;
|
||||
if (!older.length && shown.length + r.items.length <= PAGE_MAX) {
|
||||
shown = r.items.concat(shown);
|
||||
if (shown.length) from = cursorOf(shown[0]);
|
||||
} else older = r.items.concat(older);
|
||||
render(true);
|
||||
} catch (e) { fail(e); }
|
||||
}
|
||||
|
||||
const reload = () => { epoch++; sig = null; shown = []; older = []; from = null; hasOlder = false; first = true; load(); };
|
||||
for (const el of [said, answered, thought, via, person, channel, day]) el.addEventListener("change", reload);
|
||||
find.addEventListener("input", () => { clearTimeout(find.t); find.t = setTimeout(reload, 300); });
|
||||
const view = h("div", {}, h("h1", {}, "대화"),
|
||||
|
||||
Reference in New Issue
Block a user