Add Discord-native hybrid front-end for Jarvis (bot + bridge)
Some checks failed
Release / semantic-release (push) Successful in 59s
tests / Unit tests (Linux, Python 3.11) (push) Successful in 13m45s
Release / build-linux (push) Failing after 7m47s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled

Transform isair/jarvis into a Discord-controlled voice assistant running on
the Ubuntu VNC desktop, keeping the mature ~39k-line Python brain intact.

- bot/ (Node + bun, discord.js): /자비스 slash commands (ephemeral),
  voice channel join + voice receive/playback, pluggable VNC screen broadcast
  (selfbot live / noVNC / screenshot)
- bridge/ (Python, Flask): wraps jarvis STT + run_reply_engine + Piper TTS
  behind a thin localhost HTTP API
- .env.example, scripts/ (start_bridge/start_bot/dev), README rewrite,
  docs/language-comparison.md and docs/vnc-xfce-setup.md

Language decision: hybrid (Python brain + Node/bun Discord layer) because
Discord blocks bot video; native screen broadcast only works via a Node
selfbot library.
This commit is contained in:
javis-bot
2026-06-09 14:51:05 +09:00
parent a5bf8d1826
commit c4abf63f38
308 changed files with 94135 additions and 1 deletions

8
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,8 @@
# Support Jarvis development
# Choose the platforms that work best for you - you don't need to use all of them
# GitHub Sponsors (recommended) - no fees, integrated with GitHub
github: [isair]
# Ko-fi for one-time donations - simple "buy me a coffee" style
ko_fi: isair

65
.github/copilot_instructions.md vendored Normal file
View File

@@ -0,0 +1,65 @@
# Code quality standards
Write code that is clear, maintainable, and easy to understand.
Prioritize readability and simplicity over cleverness.
The best code is the least amount of code possible.
Always document complex logic and follow established style guides to ensure consistency across the codebase.
No need to keep old parameters or logic for backwards compatibility.
Every new piece of code should have tests that cover its functionality.
Do not add comments or documentation mentioning something is different than before. Comments and documentation should always be about the current state of the code.
# Testing guidelines
Tests should focus on observable outcomes and behaviors, not internal implementation details.
Treat the system as a black box: verify that inputs produce the correct outputs and side effects, regardless of how the result is achieved.
Write tests that are reliable, isolated, and easy to understand.
# Python guidelines
Follow Python best practices: use idiomatic constructs, leverage built-in modules, and write code that is explicit and readable.
Prefer list comprehensions and generator expressions for concise data processing.
Use type hints to improve code clarity and maintainability.
# Project specific rules
Data privacy comes first, always.
All user-facing command line output should make use of emojis. Especially an initial emoji to start off the lines that depict what the line is about. Output should make use of indentation spacing to establish a visual hierarchy and aim to make output as easy to sift through as possible.
## Utilities
Any important point in our logical flows should have debug logs using the `debug_log` method from `src/jarvis/debug.py`. Avoid excessive logging to keep the logs easily readable and actionable.
## Architecture decisions
For any spec files, and architectural decisions mentioned below, any code change must either adhere to them perfectly or you should ask the user to confirm changes, which should also propagate to the specs themselves.
### Listening flow
Check [here](/src/jarvis/listening/listening.spec.md) for the full listening flow specification.
### Reply flow
Check [here](/src/jarvis/reply/reply.spec.md) for the full reply flow specification.
### Language-agnostic design
Avoid hardcoded language patterns as this assistant needs to support an arbitrary amount of different languages.
### Tool-profile separation
Tools define when/how to be used. Profiles define what to do after tools execute. Keep these concerns separate in `tools.py` and `profiles.py`.
### Tool response flow
Tools return raw data without LLM processing. Profiles handle all response formatting and personality through the daemon's LLM loop. This ensures consistent response style across all profiles.

489
.github/workflows/release.yml vendored Normal file
View File

@@ -0,0 +1,489 @@
name: Release
on:
push:
branches:
- main
- develop
concurrency:
group: ${{ github.workflow }}-${{ github.ref_name }}
cancel-in-progress: true
jobs:
# Semantic versioning analysis (main only)
semantic-release:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
outputs:
new_release_published: ${{ steps.semantic.outputs.new_release_published }}
new_release_version: ${{ steps.semantic.outputs.new_release_version }}
new_release_git_tag: ${{ steps.semantic.outputs.new_release_git_tag }}
permissions:
contents: write
steps:
- name: 📥 Checkout code
uses: actions/checkout@v5
with:
fetch-depth: 0
- name: 🐍 Set up Node.js
uses: actions/setup-node@v6
with:
node-version: '20'
- name: 📦 Install semantic-release
run: |
npm install -g semantic-release@22 \
@semantic-release/github@9 \
conventional-changelog-conventionalcommits@7
- name: 🏷️ Semantic Release
id: semantic
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
# Run semantic-release and capture output
npx semantic-release --debug > release_output.log 2>&1 || true
# Check if a release was created
if grep -q "Published release" release_output.log; then
echo "new_release_published=true" >> $GITHUB_OUTPUT
# Extract version from the log
VERSION=$(grep "Published release" release_output.log | sed -n 's/.*Published release \([0-9]\+\.[0-9]\+\.[0-9]\+\).*/\1/p')
echo "new_release_version=$VERSION" >> $GITHUB_OUTPUT
echo "new_release_git_tag=v$VERSION" >> $GITHUB_OUTPUT
echo "✅ Released version $VERSION"
else
echo "new_release_published=false" >> $GITHUB_OUTPUT
echo " No release created (no releasable changes found)"
fi
# Show the full log for debugging
cat release_output.log
# Build desktop apps for all platforms
build-windows:
runs-on: windows-latest
needs: [semantic-release]
if: always() && (needs.semantic-release.result == 'success' || needs.semantic-release.result == 'skipped')
steps:
- name: 📥 Checkout code
uses: actions/checkout@v5
- name: 🐍 Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: requirements.txt
- name: 📝 Generate version file
id: version
shell: pwsh
run: |
if ("${{ github.ref }}" -eq "refs/heads/main" -and "${{ needs.semantic-release.outputs.new_release_published }}" -eq "true") {
$version = "${{ needs.semantic-release.outputs.new_release_version }}"
$channel = "stable"
} else {
$version = "dev-$($env:GITHUB_SHA.Substring(0,7))"
$channel = "develop"
}
@"
# Auto-generated at build time
VERSION = "$version"
RELEASE_CHANNEL = "$channel"
"@ | Out-File -FilePath src/jarvis/_version.py -Encoding utf8
Write-Host "Generated version file with VERSION=$version, RELEASE_CHANNEL=$channel"
echo "app_version=$version" >> $env:GITHUB_OUTPUT
- name: 📦 Install dependencies
run: |
python -m pip install --upgrade pip
# Install requirements but skip heavy optional packages (PyTorch, etc.)
# Filter out chatterbox-tts, mlx-whisper, and nvidia-* (CUDA libs are
# downloaded by the installer on-demand, not bundled in the build)
Get-Content requirements.txt | Where-Object { $_ -notmatch '^(chatterbox-tts|mlx-whisper|nvidia-)' } | Set-Content requirements-desktop.txt
pip install -r requirements-desktop.txt
pip install pyinstaller
- name: 🎨 Generate icons
run: |
python src/desktop_app/desktop_assets/generate_icons.py
- name: 🔨 Build executable (onedir)
run: |
pyinstaller jarvis_desktop.spec
- name: 🛠️ Install Inno Setup
run: |
choco install innosetup -y
- name: 📦 Build Windows installer
run: |
& "C:\Program Files (x86)\Inno Setup 6\ISCC.exe" /DMyAppVersion="${{ steps.version.outputs.app_version }}" installer\windows\jarvis_setup.iss
- name: 📦 Package installer as Jarvis-Windows-x64.zip
run: |
# Rename installer to Jarvis.exe for backwards compatibility with old updaters
Copy-Item dist\Jarvis-Setup-x64.exe dist\Jarvis.exe
cd dist
Compress-Archive -Path Jarvis.exe -DestinationPath Jarvis-Windows-x64.zip
- name: 📤 Upload Windows artifact
uses: actions/upload-artifact@v7
with:
name: Jarvis-Windows
path: dist/Jarvis-Windows-x64.zip
build-macos:
runs-on: ${{ matrix.os }}
needs: [semantic-release]
if: always() && (needs.semantic-release.result == 'success' || needs.semantic-release.result == 'skipped')
strategy:
fail-fast: false
matrix:
include:
- os: macos-latest # Apple Silicon (arm64)
arch: arm64
- os: macos-15-intel # Intel (x64)
arch: x64
steps:
- name: 📥 Checkout code
uses: actions/checkout@v5
- name: 🐍 Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: requirements.txt
- name: 📝 Generate version file
run: |
if [ "${{ github.ref }}" = "refs/heads/main" ] && [ "${{ needs.semantic-release.outputs.new_release_published }}" = "true" ]; then
VERSION="${{ needs.semantic-release.outputs.new_release_version }}"
CHANNEL="stable"
else
VERSION="dev-${GITHUB_SHA:0:7}"
CHANNEL="develop"
fi
cat > src/jarvis/_version.py << EOF
# Auto-generated at build time
VERSION = "$VERSION"
RELEASE_CHANNEL = "$CHANNEL"
EOF
echo "Generated version file with VERSION=$VERSION, RELEASE_CHANNEL=$CHANNEL"
- name: 📦 Install dependencies
run: |
python -m pip install --upgrade pip
# Install requirements but skip heavy optional packages (PyTorch/Chatterbox)
# MLX Whisper is only included on arm64 - it requires Apple Silicon
if [ "${{ matrix.arch }}" = "arm64" ]; then
grep -v -E '^chatterbox-tts' requirements.txt > requirements-desktop.txt
else
grep -v -E '^(chatterbox-tts|mlx-whisper)' requirements.txt > requirements-desktop.txt
fi
pip install -r requirements-desktop.txt
pip install pyinstaller
- name: 🎨 Generate icons
run: |
python src/desktop_app/desktop_assets/generate_icons.py
- name: 🔨 Build application
run: |
pyinstaller jarvis_desktop.spec
# Note: Ad-hoc code signing is intentionally skipped
# codesign --force --deep breaks Qt WebEngine's symlink structure
# causing crashes when QWebEngineView is shown.
# See: https://github.com/pyinstaller/pyinstaller/issues/6612
# Users can bypass Gatekeeper by right-clicking and selecting "Open"
- name: 📦 Package macOS build
run: |
cd dist
# `ditto -c -k --keepParent` preserves the symlinks, xattrs, and
# permissions that Qt/Qt WebEngine frameworks rely on. Plain
# `zip -r` follows symlinks, producing a zip that extracts into a
# bundle macOS refuses to launch ("Jarvis.app can't be opened").
ditto -c -k --keepParent Jarvis.app Jarvis-macOS-${{ matrix.arch }}.zip
- name: 📤 Upload macOS artifact
uses: actions/upload-artifact@v7
with:
name: Jarvis-macOS-${{ matrix.arch }}
path: dist/Jarvis-macOS-${{ matrix.arch }}.zip
build-linux:
runs-on: ubuntu-latest
needs: [semantic-release]
if: always() && (needs.semantic-release.result == 'success' || needs.semantic-release.result == 'skipped')
steps:
- name: 🧹 Free up disk space
run: |
# Remove unnecessary large packages to free up disk space
sudo rm -rf /usr/share/dotnet
sudo rm -rf /usr/local/lib/android
sudo rm -rf /opt/ghc
sudo rm -rf /opt/hostedtoolcache/CodeQL
sudo docker image prune --all --force
df -h
- name: 📥 Checkout code
uses: actions/checkout@v5
- name: 🐍 Set up Python
uses: actions/setup-python@v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: requirements.txt
- name: 📦 Install system dependencies
run: |
sudo apt-get update
sudo apt-get install -y libxcb-cursor0 libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-shape0 portaudio19-dev binutils
- name: 📝 Generate version file
run: |
if [ "${{ github.ref }}" = "refs/heads/main" ] && [ "${{ needs.semantic-release.outputs.new_release_published }}" = "true" ]; then
VERSION="${{ needs.semantic-release.outputs.new_release_version }}"
CHANNEL="stable"
else
VERSION="dev-${GITHUB_SHA:0:7}"
CHANNEL="develop"
fi
cat > src/jarvis/_version.py << EOF
# Auto-generated at build time
VERSION = "$VERSION"
RELEASE_CHANNEL = "$CHANNEL"
EOF
echo "Generated version file with VERSION=$VERSION, RELEASE_CHANNEL=$CHANNEL"
- name: 📦 Install Python dependencies
run: |
python -m pip install --upgrade pip
# Install requirements but skip heavy optional packages (PyTorch, etc.)
grep -v -E '^(chatterbox-tts|mlx-whisper)' requirements.txt > requirements-desktop.txt
pip install -r requirements-desktop.txt
pip install pyinstaller
- name: 🎨 Generate icons
run: |
python src/desktop_app/desktop_assets/generate_icons.py
- name: 🔨 Build executable
run: |
pyinstaller jarvis_desktop.spec
- name: 📦 Package Linux build
run: |
cd dist
# Package the Jarvis directory (not a single file anymore)
tar -czf Jarvis-Linux-x64.tar.gz Jarvis/
- name: 📤 Upload Linux artifact
uses: actions/upload-artifact@v7
with:
name: Jarvis-Linux
path: dist/Jarvis-Linux-x64.tar.gz
# Create versioned release (main only, if semantic-release published)
release-main:
needs: [semantic-release, build-windows, build-macos, build-linux]
runs-on: ubuntu-latest
# Run even if some builds failed - upload whatever succeeded
if: always() && needs.semantic-release.result == 'success' && needs.semantic-release.outputs.new_release_published == 'true'
permissions:
contents: write
steps:
- name: 📥 Download all artifacts
uses: actions/download-artifact@v8
with:
path: artifacts
- name: 📋 List available artifacts
run: |
echo "Available artifacts:"
find artifacts -type f \( -name "*.zip" -o -name "*.tar.gz" \) | sort
- name: 📎 Attach binaries to release
uses: softprops/action-gh-release@v3
with:
tag_name: ${{ needs.semantic-release.outputs.new_release_git_tag }}
# Use glob to upload only artifacts that exist
files: |
artifacts/**/*.zip
artifacts/**/*.tar.gz
fail_on_unmatched_files: false
append_body: true
body: |
---
### ⚡ Prerequisites
- [Ollama](https://ollama.com/download) (all platforms)
### 📦 Downloads
| Platform | File | Notes |
|----------|------|-------|
| **Windows** | `Jarvis-Windows-x64.zip` | Extract → Run `Jarvis.exe` |
| **macOS (Apple Silicon)** | `Jarvis-macOS-arm64.zip` | Extract → Move to Applications → Right-click → Open |
| **macOS (Intel)** | `Jarvis-macOS-x64.zip` | Extract → Move to Applications → Right-click → Open |
| **Linux** | `Jarvis-Linux-x64.tar.gz` | `tar -xzf` → Run `./Jarvis/Jarvis` |
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
# Create/update latest pre-release (develop only)
release-develop:
needs: [build-windows, build-macos, build-linux]
runs-on: ubuntu-latest
# Run even if some builds failed - upload whatever succeeded
if: always() && github.ref == 'refs/heads/develop'
permissions:
contents: write
steps:
- name: 📥 Checkout code
uses: actions/checkout@v5
with:
fetch-depth: 0 # Full history for changelog generation
fetch-tags: true # Ensure all tags are fetched
- name: 📝 Generate changelog from main
id: changelog
run: |
# Get the latest tag on main (most recent stable release)
LATEST_TAG=$(git describe --tags --abbrev=0 origin/main 2>/dev/null || echo "")
if [ -z "$LATEST_TAG" ]; then
echo "No tags found, using full develop history"
COMPARE_REF="origin/main"
SINCE_TEXT="main branch"
else
COMPARE_REF="$LATEST_TAG"
SINCE_TEXT="$LATEST_TAG"
fi
echo "Generating changelog comparing to: $COMPARE_REF"
# Generate changelog grouped by type
{
echo "CHANGELOG<<CHANGELOG_EOF"
echo ""
echo "## 📋 Changelog (since $SINCE_TEXT)"
echo ""
# Features
FEATURES=$(git log "$COMPARE_REF"..HEAD --pretty=format:"* %s ([%h](https://github.com/${{ github.repository }}/commit/%H))" --grep="^feat" --regexp-ignore-case 2>/dev/null || true)
if [ -n "$FEATURES" ]; then
echo "### ✨ Features"
echo ""
echo "$FEATURES"
echo ""
fi
# Bug fixes
FIXES=$(git log "$COMPARE_REF"..HEAD --pretty=format:"* %s ([%h](https://github.com/${{ github.repository }}/commit/%H))" --grep="^fix" --regexp-ignore-case 2>/dev/null || true)
if [ -n "$FIXES" ]; then
echo "### 🐛 Bug Fixes"
echo ""
echo "$FIXES"
echo ""
fi
# Refactoring
REFACTOR=$(git log "$COMPARE_REF"..HEAD --pretty=format:"* %s ([%h](https://github.com/${{ github.repository }}/commit/%H))" --grep="^refactor" --regexp-ignore-case 2>/dev/null || true)
if [ -n "$REFACTOR" ]; then
echo "### ♻️ Code Refactoring"
echo ""
echo "$REFACTOR"
echo ""
fi
# Documentation
DOCS=$(git log "$COMPARE_REF"..HEAD --pretty=format:"* %s ([%h](https://github.com/${{ github.repository }}/commit/%H))" --grep="^docs" --regexp-ignore-case 2>/dev/null || true)
if [ -n "$DOCS" ]; then
echo "### 📝 Documentation"
echo ""
echo "$DOCS"
echo ""
fi
# Other changes (chore, style, test, etc.)
# Get all commits, then exclude the ones we already captured
OTHER=$(git log "$COMPARE_REF"..HEAD --pretty=format:"%s|%h|%H" 2>/dev/null | grep -v -i -E "^(feat|fix|refactor|docs)" | while IFS='|' read -r subject short full; do
if [ -n "$subject" ]; then
echo "* $subject ([$short](https://github.com/${{ github.repository }}/commit/$full))"
fi
done || true)
if [ -n "$OTHER" ]; then
echo "### 🔧 Other Changes"
echo ""
echo "$OTHER"
echo ""
fi
echo "CHANGELOG_EOF"
} >> $GITHUB_OUTPUT
- name: 📥 Download all artifacts
uses: actions/download-artifact@v8
with:
path: artifacts
- name: 📋 List available artifacts
run: |
echo "Available artifacts:"
find artifacts -type f \( -name "*.zip" -o -name "*.tar.gz" \) | sort
- name: 📝 Create/Update Latest Release
uses: softprops/action-gh-release@v3
with:
tag_name: latest
name: Latest Development Build
# Use glob to upload only artifacts that exist
files: |
artifacts/**/*.zip
artifacts/**/*.tar.gz
fail_on_unmatched_files: false
draft: false
prerelease: true
body: |
🚀 **Latest development build from develop branch**
This is an automated build from the latest commit on develop.
These builds may be unstable. For stable releases, use versioned releases.
---
${{ steps.changelog.outputs.CHANGELOG }}
---
### ⚡ Prerequisites
- [Ollama](https://ollama.com/download) (all platforms)
### 📦 Downloads
| Platform | File | Notes |
|----------|------|-------|
| **Windows** | `Jarvis-Windows-x64.zip` | Extract → Run `Jarvis.exe` |
| **macOS (Apple Silicon)** | `Jarvis-macOS-arm64.zip` | Extract → Move to Applications → Right-click → Open |
| **macOS (Intel)** | `Jarvis-macOS-x64.zip` | Extract → Move to Applications → Right-click → Open |
| **Linux** | `Jarvis-Linux-x64.tar.gz` | `tar -xzf` → Run `./Jarvis/Jarvis` |
**Branch**: develop
**Commit**: ${{ github.sha }}
**Date**: ${{ github.event.head_commit.timestamp }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

28
.github/workflows/tests.yml vendored Normal file
View File

@@ -0,0 +1,28 @@
name: tests
on:
pull_request:
push:
branches: [ main, develop ]
jobs:
unit:
name: Unit tests (Linux, Python 3.11)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/setup-python@v6
with:
python-version: '3.11'
cache: pip
cache-dependency-path: requirements.txt
- name: Install system dependencies
run: sudo apt-get update && sudo apt-get install -y portaudio19-dev libegl1 libxkbcommon0
- name: Install deps
run: |
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
- name: Run unit tests
run: |
python -m pytest -q -m unit