fix(prompts): classify every sub-8B model (2b/4b/5b/6b) as SMALL
Some checks failed
Release / semantic-release (push) Successful in 24s
tests / Unit tests (Linux, Python 3.11) (push) Failing after 5m16s
Release / build-linux (push) Failing after 7m21s
Release / build-windows (push) Has been cancelled
Release / build-macos (arm64, macos-latest) (push) Has been cancelled
Release / build-macos (x64, macos-15-intel) (push) Has been cancelled
Release / release-main (push) Has been cancelled
Release / release-develop (push) Has been cancelled

detect_model_size only matched 1b/3b/7b, so a genuinely small model such as
qwen*:4b fell through to LARGE and got the terse, less-guided prompt set it
can't follow — contributing to off-tone, rambling replies. Extend the small
patterns to cover all sub-8B sizes (1b-7b) across :/-/_ separators and sync the
spec table.
This commit is contained in:
javis-bot
2026-06-24 18:31:54 +09:00
parent d5fd218c86
commit 83999a5b0b
3 changed files with 16 additions and 6 deletions

View File

@@ -22,11 +22,15 @@ class ModelSize(Enum):
LARGE = "large" # 8b+ - can infer tool usage from context LARGE = "large" # 8b+ - can infer tool usage from context
# Model size patterns - models matching these are considered SMALL # Model size patterns - models matching these are considered SMALL.
# Covers every sub-8B size (1b-7b): these models need the explicit, repeated
# tool/greeting/instruction constraints and falter on the terse LARGE prompt.
# Without 2b/4b/5b/6b here a genuinely small model (e.g. qwen*:4b) was
# misclassified as LARGE and given the less-guided prompt set.
_SMALL_MODEL_PATTERNS = ( _SMALL_MODEL_PATTERNS = (
":1b", ":3b", ":7b", ":1b", ":2b", ":3b", ":4b", ":5b", ":6b", ":7b",
"-1b", "-3b", "-7b", "-1b", "-2b", "-3b", "-4b", "-5b", "-6b", "-7b",
"_1b", "_3b", "_7b", "_1b", "_2b", "_3b", "_4b", "_5b", "_6b", "_7b",
"gemma4", # Gemma 4 - always small regardless of tag "gemma4", # Gemma 4 - always small regardless of tag
) )

View File

@@ -4,7 +4,7 @@ This module provides model-size-aware prompt generation for the reply engine.
### Problem Statement ### Problem Statement
Small models (1b, 3b, 7b parameters) lack the reasoning capacity to infer when NOT to use tools. When given prompts like "Proactively use available tools," they may incorrectly call tools for simple greetings like "hello" or "ni hao" because they cannot distinguish between: Small models (every sub-8B size, 1b-7b parameters) lack the reasoning capacity to infer when NOT to use tools. When given prompts like "Proactively use available tools," they may incorrectly call tools for simple greetings like "hello" or "ni hao" because they cannot distinguish between:
- Requests that require tools (weather, search, data retrieval) - Requests that require tools (weather, search, data retrieval)
- Simple conversation (greetings, small talk, general knowledge) - Simple conversation (greetings, small talk, general knowledge)
@@ -14,7 +14,7 @@ The module detects model size from the model name and selects appropriate prompt
| Model Size | Detection Pattern | Tool Prompts | | Model Size | Detection Pattern | Tool Prompts |
|------------|-------------------|--------------| |------------|-------------------|--------------|
| SMALL | `:1b`, `:3b`, `:7b`, `gemma4` | Conservative — explicit "DO NOT use tools for greetings" + worked negative examples + repetition | | SMALL | `:1b`-`:7b` (every size 1-7B, all separators), `gemma4` | Conservative — explicit "DO NOT use tools for greetings" + worked negative examples + repetition |
| LARGE | All others (8b+) | Proactive — "use tools confidently" + short anti-confabulation + auto-derive clause | | LARGE | All others (8b+) | Proactive — "use tools confidently" + short anti-confabulation + auto-derive clause |
### Architecture ### Architecture

View File

@@ -21,9 +21,15 @@ class TestModelSizeDetection:
("gemma:7b", True), ("gemma:7b", True),
("phi3:3b", True), ("phi3:3b", True),
("qwen2:7b", True), ("qwen2:7b", True),
# Sub-8B sizes that were previously misclassified as LARGE.
("qwen3.5:4b", True), # the deployed model that produced weak, off-tone replies
("gemma2:2b", True),
("model:5b", True),
("model:6b", True),
# Various separators # Various separators
("model-3b-instruct", True), ("model-3b-instruct", True),
("model_1b_chat", True), ("model_1b_chat", True),
("model-4b-instruct", True),
# Large models (should return LARGE) # Large models (should return LARGE)
("gpt-oss:20b", False), ("gpt-oss:20b", False),
("llama3.1:8b", False), ("llama3.1:8b", False),