fix(browser): the bot opens its pages in windows of their own behind, so it never takes the focus
On the shared screen a tab brought to the front makes Chrome bring its window forward and take the keyboard focus from the game or program the user is in (seen on the game PC). The botBrowser tool now opens each person's page in a window of its own, behind and at the place and size of the bot browser's window (Target.createTarget with newWindow and background), and closes it again if the connection to it fails. While the browser runs on the virtual monitor, the app looks for new Chrome windows every 200 ms and settles each like the first (tool window, fitted to the work area, without activating it), the newest in front of the older ones, so the shared screen shows the page the bot opened last. Checked on the game PC: the bot's window loading a public page comes in front of the first, both out of the taskbar and Alt+Tab, the focus staying in Discord also with the user idle for hours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -444,7 +444,11 @@ the browser runs headless with a note.
|
||||
area (Windows may give the monitor a taskbar of its own) and goes behind all other windows, all
|
||||
without activating it. It keeps drawing when covered (`--disable-backgrounding-occluded-windows`,
|
||||
`--disable-renderer-backgrounding`, `--disable-background-timer-throttling`). The window holds
|
||||
the first tab, which the bot never closes; the bot's tabs open in it, the newest in front.
|
||||
the first page, which the bot never closes. The bot's pages open in windows of their own
|
||||
(bot_browser.spec.md), behind and at this window's place, because a tab brought to the front
|
||||
takes the keyboard focus. While the browser runs, the app looks for new windows every 200 ms
|
||||
and settles each the same way, the newest in front of the older ones, so the shared screen shows
|
||||
the page the bot opened last.
|
||||
- If the monitor or the window cannot be made, the browser runs headless and the dashboard's
|
||||
`browser` note says why ("가상 모니터 없이 실행: ...").
|
||||
|
||||
@@ -811,8 +815,9 @@ started elevated, the same program is the elevated half; `tests/bot_browser.rs`
|
||||
driver installed through the elevated start (Windows' SignPath question answered once) with the
|
||||
settings folder's permissions, a declined install leaving no device, package, registry key or
|
||||
settings folder and the next install giving one working device, the monitor off after install, on at the rightmost monitor's corner
|
||||
while the bot browser runs and off after, one Chrome window there filling the work area and holding
|
||||
the bot's tab, the focus staying in the program the user types into, no taskbar button, no Chrome
|
||||
while the bot browser runs and off after, one Chrome window there filling the work area, then the bot's own
|
||||
window (opened as the tool opens it, loading a public page) in front of it, the focus staying in the program the user types into
|
||||
also with the user idle past Windows' focus lock, no taskbar or Alt+Tab entry for either, no Chrome
|
||||
left after the stop, and the uninstall leaving no device, driver package, registry key or settings
|
||||
folder. Game detection (`tests/game_watch.rs`): a program started from a Steam game folder (also under
|
||||
a path with non-ASCII letters) or from a Battle.net game's install folder (a test uninstall entry)
|
||||
@@ -848,4 +853,4 @@ after OK, and starts normally once released; `/S /RUN` (fresh, and over a runnin
|
||||
and the finish page's "Jarvis 실행" start the app's window; a silent install while the uninstaller's
|
||||
window is open, and a silent uninstall while the installer's is, stop with code 5 and change
|
||||
nothing. On Linux the same window test runs on an X display (one window at the given place and
|
||||
size, the bot's tab in it).
|
||||
size, the bot's own window at the same place).
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
|
||||
use std::net::{IpAddr, SocketAddr};
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::sync::Mutex;
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::Duration;
|
||||
|
||||
use tokio::io::{AsyncReadExt, AsyncWriteExt};
|
||||
@@ -16,6 +17,8 @@ const HEAD_TIMEOUT: Duration = Duration::from_secs(30);
|
||||
const MAX_HEAD: usize = 64 * 1024;
|
||||
const READY_TIMEOUT: Duration = Duration::from_secs(30);
|
||||
const CLOSE_WAIT: Duration = Duration::from_secs(5);
|
||||
/// How often the windows the bot opens on the virtual monitor are looked for.
|
||||
const WATCH_EVERY: Duration = Duration::from_millis(200);
|
||||
|
||||
/// Whether an address is on the public internet: not this PC, the local network or a special range.
|
||||
pub fn is_public(ip: &IpAddr) -> bool {
|
||||
@@ -255,6 +258,14 @@ pub struct BotBrowser {
|
||||
child: Mutex<Option<tokio::process::Child>>,
|
||||
proxy: EgressProxy,
|
||||
pub cdp_port: u16,
|
||||
/// While set (with a screen, on Windows), a thread settles the windows the bot opens.
|
||||
watching: Arc<AtomicBool>,
|
||||
}
|
||||
|
||||
impl Drop for BotBrowser {
|
||||
fn drop(&mut self) {
|
||||
self.watching.store(false, Ordering::Relaxed);
|
||||
}
|
||||
}
|
||||
|
||||
impl BotBrowser {
|
||||
@@ -293,12 +304,25 @@ impl BotBrowser {
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(200)).await;
|
||||
};
|
||||
let b = Self { child: Mutex::new(Some(child)), proxy, cdp_port };
|
||||
let b = Self { child: Mutex::new(Some(child)), proxy, cdp_port, watching: Arc::new(AtomicBool::new(false)) };
|
||||
if let Some(r) = screen {
|
||||
if let Err(e) = b.open_window(r).await {
|
||||
b.stop().await;
|
||||
return Err(e);
|
||||
}
|
||||
if cfg!(windows) {
|
||||
// The bot opens a window per person (bot_browser.spec.md), each settled like the first.
|
||||
b.watching.store(true, Ordering::Relaxed);
|
||||
let (on, pid) = (b.watching.clone(), b.pid());
|
||||
std::thread::spawn(move || {
|
||||
while on.load(Ordering::Relaxed) {
|
||||
if crate::vdisplay::settle_windows(pid) > 0 {
|
||||
crate::debug_log!("browser", "a new bot browser window settled on the virtual monitor");
|
||||
}
|
||||
std::thread::sleep(WATCH_EVERY);
|
||||
}
|
||||
});
|
||||
}
|
||||
}
|
||||
crate::info_log!("browser", "bot browser ready (CDP on 127.0.0.1:{cdp_port}{})",
|
||||
if screen.is_some() { ", on the virtual monitor" } else { "" });
|
||||
@@ -307,14 +331,15 @@ impl BotBrowser {
|
||||
|
||||
/// The window on `r`: a background page in a new window (CDP), then out of the window switcher
|
||||
/// and fitted to the monitor's work area.
|
||||
/// It holds the first tab, which the bot never closes, so the window stays.
|
||||
/// It holds the first page, which the bot never closes, so the window stays; the bot's own
|
||||
/// windows open where it is.
|
||||
async fn open_window(&self, r: crate::vdisplay::Rect) -> Result<(), String> {
|
||||
let ws = version(self.cdp_port).await.and_then(|v| v["webSocketDebuggerUrl"].as_str().map(str::to_owned))
|
||||
.ok_or("Chrome의 CDP 주소를 읽지 못했습니다")?;
|
||||
cdp_call(&ws, serde_json::json!({"id": 1, "method": "Target.createTarget", "params": {"url": "about:blank",
|
||||
"newWindow": true, "background": true, "left": r.x, "top": r.y, "width": r.w, "height": r.h}})).await
|
||||
.map_err(|e| format!("Chrome 창을 열지 못했습니다: {e}"))?;
|
||||
let pid = self.child.lock().unwrap().as_ref().and_then(|c| c.id()).unwrap_or(0);
|
||||
let pid = self.pid();
|
||||
for _ in 0..25 {
|
||||
if cfg!(not(windows)) || crate::vdisplay::settle_windows(pid) > 0 {
|
||||
return Ok(());
|
||||
@@ -324,6 +349,11 @@ impl BotBrowser {
|
||||
Err("Chrome 창을 찾지 못했습니다".into())
|
||||
}
|
||||
|
||||
/// Chrome's main process (0 once it ended), the one owning its windows.
|
||||
pub fn pid(&self) -> u32 {
|
||||
self.child.lock().unwrap().as_ref().and_then(|c| c.id()).unwrap_or(0)
|
||||
}
|
||||
|
||||
pub fn cdp_url(&self) -> String {
|
||||
format!("http://127.0.0.1:{}", self.cdp_port)
|
||||
}
|
||||
@@ -335,6 +365,7 @@ impl BotBrowser {
|
||||
|
||||
/// `Browser.close`, then up to 5 s for Chrome to end, else it is killed; the proxy ends too.
|
||||
pub async fn stop(&self) {
|
||||
self.watching.store(false, Ordering::Relaxed);
|
||||
let child = self.child.lock().unwrap().take();
|
||||
if let Some(mut child) = child {
|
||||
if let Some(ws) = version(self.cdp_port).await.and_then(|v| v["webSocketDebuggerUrl"].as_str().map(str::to_owned)) {
|
||||
|
||||
@@ -330,26 +330,26 @@ mod win {
|
||||
1
|
||||
}
|
||||
|
||||
/// Make process `pid`'s top-level Chrome windows tool windows (not in Alt+Tab, Win+Tab or the
|
||||
/// taskbar), fill their monitor's work area (Windows may give the virtual monitor a taskbar of
|
||||
/// its own, which would cover the page's bottom) and put them behind all others, all without
|
||||
/// activating them. Returns how many it made tool windows.
|
||||
/// Make process `pid`'s new top-level Chrome windows tool windows (not in Alt+Tab, Win+Tab or
|
||||
/// the taskbar) and fill their monitor's work area (Windows may give the virtual monitor a
|
||||
/// taskbar of its own, which would cover the page's bottom), all without activating them. The
|
||||
/// bot's windows stay behind all others, the newest in front of the older ones. Returns how many
|
||||
/// windows were new.
|
||||
pub fn settle_windows(pid: u32) -> usize {
|
||||
let mut found = Found { pid, windows: vec![] };
|
||||
// SAFETY: `collect` only reads windows and writes into `found`, which lives through the call.
|
||||
unsafe { EnumWindows(Some(collect), &mut found as *mut Found as LPARAM) };
|
||||
let mut changed = 0;
|
||||
for h in found.windows {
|
||||
// SAFETY: a plain query on a window handle EnumWindows gave.
|
||||
let (old, new): (Vec<HWND>, Vec<HWND>) = found.windows.into_iter()
|
||||
.partition(|&h| unsafe { GetWindowLongPtrW(h, GWL_EXSTYLE) } as u32 & WS_EX_TOOLWINDOW != 0);
|
||||
for &h in &new {
|
||||
// SAFETY: style changes on a window of our own child process; hiding and showing it
|
||||
// again is what makes the taskbar notice the new style.
|
||||
unsafe {
|
||||
let ex = GetWindowLongPtrW(h, GWL_EXSTYLE) as u32;
|
||||
if ex & WS_EX_TOOLWINDOW == 0 {
|
||||
ShowWindow(h, SW_HIDE);
|
||||
SetWindowLongPtrW(h, GWL_EXSTYLE, ((ex | WS_EX_TOOLWINDOW) & !WS_EX_APPWINDOW) as isize);
|
||||
ShowWindow(h, SW_SHOWNOACTIVATE);
|
||||
changed += 1;
|
||||
}
|
||||
ShowWindow(h, SW_HIDE);
|
||||
SetWindowLongPtrW(h, GWL_EXSTYLE, ((ex | WS_EX_TOOLWINDOW) & !WS_EX_APPWINDOW) as isize);
|
||||
ShowWindow(h, SW_SHOWNOACTIVATE);
|
||||
let mut mi = MONITORINFO { cbSize: std::mem::size_of::<MONITORINFO>() as u32, ..Default::default() };
|
||||
if GetMonitorInfoW(MonitorFromWindow(h, MONITOR_DEFAULTTONEAREST), &mut mi) != 0 {
|
||||
let w = mi.rcWork;
|
||||
@@ -359,7 +359,14 @@ mod win {
|
||||
}
|
||||
}
|
||||
}
|
||||
changed
|
||||
if !new.is_empty() {
|
||||
// Top first, each to the bottom: they keep their order, all behind the new ones.
|
||||
for &h in &old {
|
||||
// SAFETY: a z-order change of our own child process's window, without activating it.
|
||||
unsafe { SetWindowPos(h, HWND_BOTTOM, 0, 0, 0, 0, SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE) };
|
||||
}
|
||||
}
|
||||
new.len()
|
||||
}
|
||||
|
||||
/// The root devices with the driver's hardware ID, in any class (a device whose driver install
|
||||
|
||||
Reference in New Issue
Block a user