docs: update README, CLAUDE.md, and skills for Groq Whisper
- Add GROQ_API_KEY to README env vars and authentication steps - Add Voice Transcription section to CLAUDE.md - Update setup skill with Groq voice transcription setup step - Rewrite add-voice-transcription skill for Groq-first approach
This commit is contained in:
@@ -1,144 +1,90 @@
|
|||||||
---
|
---
|
||||||
name: add-voice-transcription
|
name: add-voice-transcription
|
||||||
description: Add voice message transcription to NanoClaw using OpenAI's Whisper API. Automatically transcribes WhatsApp voice notes so the agent can read and respond to them.
|
description: Add voice message transcription to NanoClaw using Groq Whisper (fast, free) with OpenAI fallback. Works on Discord and WhatsApp.
|
||||||
---
|
---
|
||||||
|
|
||||||
# Add Voice Transcription
|
# Add Voice Transcription
|
||||||
|
|
||||||
This skill adds automatic voice message transcription to NanoClaw's WhatsApp channel using OpenAI's Whisper API. When a voice note arrives, it is downloaded, transcribed, and delivered to the agent as `[Voice: <transcript>]`.
|
This skill adds automatic voice message transcription to NanoClaw. When a voice note arrives, it is downloaded, transcribed, and delivered to the agent as `[Voice message transcription]: <text>`.
|
||||||
|
|
||||||
|
**Provider priority:** Groq Whisper (fast, free) > OpenAI Whisper (fallback).
|
||||||
|
|
||||||
## Phase 1: Pre-flight
|
## Phase 1: Pre-flight
|
||||||
|
|
||||||
### Check if already applied
|
### Discord
|
||||||
|
|
||||||
|
Voice transcription is built into `src/channels/discord.ts`. No code changes needed — just configure API keys (Phase 3).
|
||||||
|
|
||||||
|
### WhatsApp
|
||||||
|
|
||||||
Check if `src/transcription.ts` exists. If it does, skip to Phase 3 (Configure). The code changes are already in place.
|
Check if `src/transcription.ts` exists. If it does, skip to Phase 3 (Configure). The code changes are already in place.
|
||||||
|
|
||||||
### Ask the user
|
If not, merge the skill branch:
|
||||||
|
|
||||||
Use `AskUserQuestion` to collect information:
|
|
||||||
|
|
||||||
AskUserQuestion: Do you have an OpenAI API key for Whisper transcription?
|
|
||||||
|
|
||||||
If yes, collect it now. If no, direct them to create one at https://platform.openai.com/api-keys.
|
|
||||||
|
|
||||||
## Phase 2: Apply Code Changes
|
|
||||||
|
|
||||||
**Prerequisite:** WhatsApp must be installed first (`skill/whatsapp` merged). This skill modifies WhatsApp channel files.
|
|
||||||
|
|
||||||
### Ensure WhatsApp fork remote
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git remote -v
|
|
||||||
```
|
|
||||||
|
|
||||||
If `whatsapp` is missing, add it:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git remote add whatsapp https://github.com/qwibitai/nanoclaw-whatsapp.git
|
|
||||||
```
|
|
||||||
|
|
||||||
### Merge the skill branch
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
git remote add whatsapp https://github.com/qwibitai/nanoclaw-whatsapp.git 2>/dev/null
|
||||||
git fetch whatsapp skill/voice-transcription
|
git fetch whatsapp skill/voice-transcription
|
||||||
git merge whatsapp/skill/voice-transcription
|
git merge whatsapp/skill/voice-transcription
|
||||||
```
|
|
||||||
|
|
||||||
This merges in:
|
|
||||||
- `src/transcription.ts` (voice transcription module using OpenAI Whisper)
|
|
||||||
- Voice handling in `src/channels/whatsapp.ts` (isVoiceMessage check, transcribeAudioMessage call)
|
|
||||||
- Transcription tests in `src/channels/whatsapp.test.ts`
|
|
||||||
- `openai` npm dependency in `package.json`
|
|
||||||
- `OPENAI_API_KEY` in `.env.example`
|
|
||||||
|
|
||||||
If the merge reports conflicts, resolve them by reading the conflicted files and understanding the intent of both sides.
|
|
||||||
|
|
||||||
### Validate code changes
|
|
||||||
|
|
||||||
```bash
|
|
||||||
npm install --legacy-peer-deps
|
npm install --legacy-peer-deps
|
||||||
npm run build
|
npm run build
|
||||||
npx vitest run src/channels/whatsapp.test.ts
|
|
||||||
```
|
```
|
||||||
|
|
||||||
All tests must pass and build must be clean before proceeding.
|
## Phase 2: Configure
|
||||||
|
|
||||||
## Phase 3: Configure
|
### Get API key
|
||||||
|
|
||||||
### Get OpenAI API key (if needed)
|
**Groq (recommended — fast + free):**
|
||||||
|
|
||||||
If the user doesn't have an API key:
|
> 1. Go to https://console.groq.com
|
||||||
|
> 2. Sign up (no credit card needed)
|
||||||
> I need you to create an OpenAI API key:
|
> 3. Create an API key (starts with `gsk_`)
|
||||||
>
|
>
|
||||||
|
> Free tier: 2,000 requests/day, 8 hours of audio/day. Uses `whisper-large-v3-turbo` at ~200x real-time speed.
|
||||||
|
|
||||||
|
**OpenAI (fallback):**
|
||||||
|
|
||||||
> 1. Go to https://platform.openai.com/api-keys
|
> 1. Go to https://platform.openai.com/api-keys
|
||||||
> 2. Click "Create new secret key"
|
> 2. Create a key (starts with `sk-`)
|
||||||
> 3. Give it a name (e.g., "NanoClaw Transcription")
|
|
||||||
> 4. Copy the key (starts with `sk-`)
|
|
||||||
>
|
>
|
||||||
> Cost: ~$0.006 per minute of audio (~$0.003 per typical 30-second voice note)
|
> Cost: ~$0.006 per minute of audio. Requires funded account.
|
||||||
|
|
||||||
Wait for the user to provide the key.
|
|
||||||
|
|
||||||
### Add to environment
|
### Add to environment
|
||||||
|
|
||||||
Add to `.env`:
|
Add to `.env`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
OPENAI_API_KEY=<their-key>
|
GROQ_API_KEY=gsk_... # Primary (fast, free)
|
||||||
|
OPENAI_API_KEY=sk-... # Fallback (optional if Groq is set)
|
||||||
```
|
```
|
||||||
|
|
||||||
Sync to container environment:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p data/env && cp .env data/env/env
|
|
||||||
```
|
|
||||||
|
|
||||||
The container reads environment from `data/env/env`, not `.env` directly.
|
|
||||||
|
|
||||||
### Build and restart
|
### Build and restart
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
npm run build
|
npm run build
|
||||||
launchctl kickstart -k gui/$(id -u)/com.nanoclaw # macOS
|
systemctl --user restart nanoclaw nanoclaw-codex # Linux
|
||||||
# Linux: systemctl --user restart nanoclaw
|
# macOS: launchctl kickstart -k gui/$(id -u)/com.nanoclaw
|
||||||
```
|
```
|
||||||
|
|
||||||
## Phase 4: Verify
|
## Phase 3: Verify
|
||||||
|
|
||||||
### Test with a voice note
|
Send a voice note in any registered chat. The agent should receive it as `[Voice message transcription]: <text>`.
|
||||||
|
|
||||||
Tell the user:
|
### Check logs
|
||||||
|
|
||||||
> Send a voice note in any registered WhatsApp chat. The agent should receive it as `[Voice: <transcript>]` and respond to its content.
|
|
||||||
|
|
||||||
### Check logs if needed
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
tail -f logs/nanoclaw.log | grep -i voice
|
tail -f logs/nanoclaw.log | grep -iE "transcri|audio"
|
||||||
```
|
```
|
||||||
|
|
||||||
Look for:
|
Look for:
|
||||||
- `Transcribed voice message` — successful transcription with character count
|
- `Audio transcribed + cached` with `provider: "groq"` and `elapsed` — success
|
||||||
- `OPENAI_API_KEY not set` — key missing from `.env`
|
- `Transcription cache hit` — second service read from cache (no duplicate API call)
|
||||||
- `OpenAI transcription failed` — API error (check key validity, billing)
|
- `no transcription API key` — neither `GROQ_API_KEY` nor `OPENAI_API_KEY` set
|
||||||
- `Failed to download audio message` — media download issue
|
- `groq Whisper 4xx` — check key validity
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
### Voice notes show "[Voice Message - transcription unavailable]"
|
**No transcription:** Check `GROQ_API_KEY` (or `OPENAI_API_KEY`) is set in `.env`. Restart service after changes.
|
||||||
|
|
||||||
1. Check `OPENAI_API_KEY` is set in `.env` AND synced to `data/env/env`
|
**Slow transcription:** Verify `provider: "groq"` in logs. If it shows `openai`, the Groq key may be missing or invalid.
|
||||||
2. Verify key works: `curl -s https://api.openai.com/v1/models -H "Authorization: Bearer $OPENAI_API_KEY" | head -c 200`
|
|
||||||
3. Check OpenAI billing — Whisper requires a funded account
|
|
||||||
|
|
||||||
### Voice notes show "[Voice Message - transcription failed]"
|
**Agent doesn't respond to voice notes:** Verify the chat is registered and the agent is running. Voice transcription only runs for registered groups.
|
||||||
|
|
||||||
Check logs for the specific error. Common causes:
|
|
||||||
- Network timeout — transient, will work on next message
|
|
||||||
- Invalid API key — regenerate at https://platform.openai.com/api-keys
|
|
||||||
- Rate limiting — wait and retry
|
|
||||||
|
|
||||||
### Agent doesn't respond to voice notes
|
|
||||||
|
|
||||||
Verify the chat is registered and the agent is running. Voice transcription only runs for registered groups.
|
|
||||||
|
|||||||
@@ -99,6 +99,12 @@ AskUserQuestion: Do you also want to use Codex (OpenAI) agents?
|
|||||||
|
|
||||||
If yes: Tell user to add `OPENAI_API_KEY=<key>` (or `CODEX_OPENAI_API_KEY=<key>`) to `.env`. The Codex runner will use this key.
|
If yes: Tell user to add `OPENAI_API_KEY=<key>` (or `CODEX_OPENAI_API_KEY=<key>`) to `.env`. The Codex runner will use this key.
|
||||||
|
|
||||||
|
### Voice Transcription (Optional)
|
||||||
|
|
||||||
|
AskUserQuestion: Do you want voice message transcription in Discord/WhatsApp?
|
||||||
|
|
||||||
|
If yes: Tell user to get a free API key at https://console.groq.com (no credit card needed) and add `GROQ_API_KEY=<key>` to `.env`. Uses Groq Whisper (whisper-large-v3-turbo, ~200x faster than OpenAI). Falls back to OpenAI Whisper (`OPENAI_API_KEY`) if Groq key is not set.
|
||||||
|
|
||||||
## 5. Install Skills Marketplace
|
## 5. Install Skills Marketplace
|
||||||
|
|
||||||
Register and install the NanoClaw skills marketplace plugin so all feature skills (channel integrations, add-ons) are available:
|
Register and install the NanoClaw skills marketplace plugin so all feature skills (channel integrations, add-ons) are available:
|
||||||
@@ -188,6 +194,8 @@ Tell user to test: send a message in their registered chat. Show: `tail -f logs/
|
|||||||
|
|
||||||
**Codex agent not responding:** Ensure `OPENAI_API_KEY` is set in `.env` and `codex` CLI is installed globally (`npm install -g @openai/codex`). The runner needs `codex` in PATH.
|
**Codex agent not responding:** Ensure `OPENAI_API_KEY` is set in `.env` and `codex` CLI is installed globally (`npm install -g @openai/codex`). The runner needs `codex` in PATH.
|
||||||
|
|
||||||
|
**Voice transcription not working:** Check `GROQ_API_KEY` (primary) or `OPENAI_API_KEY` (fallback) is set in `.env`. Test: send a voice message in Discord and check `logs/nanoclaw.log` for `Audio transcribed` entries with `provider` and `elapsed` fields.
|
||||||
|
|
||||||
**No response to messages:** Check trigger pattern. Main channel doesn't need prefix. Check DB: `npx tsx setup/index.ts --step verify`. Check `logs/nanoclaw.log`.
|
**No response to messages:** Check trigger pattern. Main channel doesn't need prefix. Check DB: `npx tsx setup/index.ts --step verify`. Check `logs/nanoclaw.log`.
|
||||||
|
|
||||||
**Channel not connecting:** Verify the channel's credentials are set in `.env`. Channels auto-enable when their credentials are present. For WhatsApp: check `store/auth/creds.json` exists. For token-based channels: check token values in `.env`. Restart the service after any `.env` change.
|
**Channel not connecting:** Verify the channel's credentials are set in `.env`. Channels auto-enable when their credentials are present. For WhatsApp: check `store/auth/creds.json` exists. For token-based channels: check token values in `.env`. Restart the service after any `.env` change.
|
||||||
|
|||||||
10
CLAUDE.md
10
CLAUDE.md
@@ -13,7 +13,7 @@ Two systemd services (`nanoclaw`, `nanoclaw-codex`) share the same codebase but
|
|||||||
| `src/index.ts` | Orchestrator: state, message loop, agent invocation |
|
| `src/index.ts` | Orchestrator: state, message loop, agent invocation |
|
||||||
| `src/agent-runner.ts` | Spawns agent processes, manages env/sessions/skills |
|
| `src/agent-runner.ts` | Spawns agent processes, manages env/sessions/skills |
|
||||||
| `src/token-refresh.ts` | OAuth auto-refresh + session directory sync |
|
| `src/token-refresh.ts` | OAuth auto-refresh + session directory sync |
|
||||||
| `src/channels/discord.ts` | Discord channel (8s typing refresh) |
|
| `src/channels/discord.ts` | Discord channel (8s typing refresh, Groq/OpenAI Whisper transcription) |
|
||||||
| `src/ipc.ts` | IPC watcher and task processing |
|
| `src/ipc.ts` | IPC watcher and task processing |
|
||||||
| `src/router.ts` | Message formatting and outbound routing |
|
| `src/router.ts` | Message formatting and outbound routing |
|
||||||
| `src/config.ts` | Trigger pattern, paths, intervals |
|
| `src/config.ts` | Trigger pattern, paths, intervals |
|
||||||
@@ -68,3 +68,11 @@ Codex runner uses `codex app-server` JSON-RPC (not `codex exec`):
|
|||||||
- `turn/steer` for mid-execution message injection (IPC polling during turn)
|
- `turn/steer` for mid-execution message injection (IPC polling during turn)
|
||||||
- `approvalPolicy: "never"` + `sandbox: "danger-full-access"` for bypass
|
- `approvalPolicy: "never"` + `sandbox: "danger-full-access"` for bypass
|
||||||
- Per-group: model (`CODEX_MODEL`), effort (`CODEX_EFFORT`), MCP servers via `config.toml`
|
- Per-group: model (`CODEX_MODEL`), effort (`CODEX_EFFORT`), MCP servers via `config.toml`
|
||||||
|
|
||||||
|
## Voice Transcription
|
||||||
|
|
||||||
|
Audio attachments in Discord are transcribed via Groq Whisper (primary) or OpenAI Whisper (fallback):
|
||||||
|
- `GROQ_API_KEY` — Groq `whisper-large-v3-turbo`, ~200x real-time, free tier (console.groq.com)
|
||||||
|
- `OPENAI_API_KEY` — OpenAI `whisper-1`, fallback if Groq key not set
|
||||||
|
- Shared file cache (`cache/transcriptions/`) deduplicates across both services
|
||||||
|
- `.pending` file coordination prevents duplicate API calls
|
||||||
|
|||||||
@@ -149,6 +149,9 @@ claude login
|
|||||||
|
|
||||||
# Codex — set API key
|
# Codex — set API key
|
||||||
export OPENAI_API_KEY=sk-...
|
export OPENAI_API_KEY=sk-...
|
||||||
|
|
||||||
|
# Groq — for fast voice transcription (free at console.groq.com)
|
||||||
|
export GROQ_API_KEY=gsk_...
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Environment Variables
|
### 3. Environment Variables
|
||||||
@@ -161,6 +164,7 @@ DISCORD_BOT_TOKEN= # Claude Code Discord bot token
|
|||||||
ASSISTANT_NAME=claude # Bot trigger name (@claude)
|
ASSISTANT_NAME=claude # Bot trigger name (@claude)
|
||||||
ANTHROPIC_API_KEY= # Or use OAuth (claude login)
|
ANTHROPIC_API_KEY= # Or use OAuth (claude login)
|
||||||
OPENAI_API_KEY= # For Codex
|
OPENAI_API_KEY= # For Codex
|
||||||
|
GROQ_API_KEY= # For voice transcription (Groq Whisper, fast + free)
|
||||||
```
|
```
|
||||||
|
|
||||||
For dual-service setup, create `.env.codex` for Codex-specific overrides:
|
For dual-service setup, create `.env.codex` for Codex-specific overrides:
|
||||||
|
|||||||
Reference in New Issue
Block a user