feat(v2): pcControl keys go behind a window when they can, else in front for a moment
User decision 2026-11-19 (mixed way). agent-desktop only presses keys in the window in front, so the app now picks one way before any key is sent: - behind: one key without modifiers that types or moves is posted to the window's one shown text control (Edit, RichEdit, RichEditD2DPT, Scintilla); the person's screen and focus stay. Windows does not tell a background thread's focused control (win-ci VM: GetFocus is null even attached), so only windows with exactly one text control get keys behind. - in front for a moment: combinations, other keys and every other window; the window that was in front is brought back after (also after a failed press). A window already in front gets the keys there. - never in front while game mode is on or the window in front is full screen. - never both ways, so a key is never pressed twice. Specs: admin.spec.md "PC control" (Keys), pc-features.md, pc_control.spec.md. Tests: six new fake-desktop tests in pc.rs; tests/pc_keys_windows.rs on a real desktop (win-ci VM: letters behind Notepad with Character Map kept in front, ctrl+a in front and back, the classic Calculator in front and back). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -65,7 +65,7 @@ One tool, `action` picks the operation (read as its first known word):
|
||||
| `read` | `window` | the window's interactive elements: ref, role, name, value, state | view |
|
||||
| `click` | `ref` | presses the element (button, menu item, list item, tab, check box) | click |
|
||||
| `type` | `ref`, `text` | puts the text into the field (replacing what was there) | type |
|
||||
| `keys` | `window`, `keys` | presses keys in the window (`ctrl+s`, `enter`, `tab`) | type |
|
||||
| `keys` | `window`, `keys` | presses keys in the window (`ctrl+s`, `enter`, `tab`): behind it when it takes them, else in front for a moment (`admin.spec.md`, "PC control") | type |
|
||||
| `run` | `program` | starts a program by name (`notepad`, `calc`, a Start menu name) and reads its window | run |
|
||||
| `close` | `window` | closes the program's window | click |
|
||||
|
||||
|
||||
Reference in New Issue
Block a user