fix(reply): stop weak models parroting persona example facts
A 4b model replied to "하이" with "테니스 연습을 Trenches Gym에서..." — it copied the literal "box at Trenches Gym" few-shot example embedded in the persona prompt and mangled boxing into tennis, presenting a prompt example as if it were a real user fact. Remove the copyable proper-noun example and add an explicit guard: use ONLY names/places/activities that literally appear in the memory section, never borrow them from the instructions or example wording.
This commit is contained in:
@@ -120,10 +120,13 @@ _SYSTEM_PROMPT_TEMPLATE: str = (
|
||||
"'tell me a joke', 'chat with me'), never reply with a bare greeting like 'Hey there!', "
|
||||
"'Hi!', 'How can I help you?', or a generic observation about an unrelated topic. "
|
||||
"When the 'Information the user has shared…' section is present, you MUST pick one concrete "
|
||||
"fact from it and build the reply around that fact (e.g. 'You mentioned you box at Trenches "
|
||||
"Gym — how's training going this week?'). Do not talk about things that are not in that "
|
||||
"section. Only when that section is absent may you invent a fresh observation, question, or "
|
||||
"joke. Produce a varied response each time — do not repeat a previous reply verbatim. "
|
||||
"fact from it and build the reply around that fact, opening with a short natural reference to "
|
||||
"it. CRITICAL: use ONLY names, places, activities, and details that literally appear in that "
|
||||
"section — never borrow any name, place, or activity from these instructions or from any "
|
||||
"example wording, and never invent specifics that are not in that section. Do not talk about "
|
||||
"things that are not in that section. Only when that section is absent may you invent a fresh "
|
||||
"observation, question, or joke. Produce a varied response each time — do not repeat a "
|
||||
"previous reply verbatim. "
|
||||
"Banned phrasings: 'I can only tell you what you have shared with me in this conversation', "
|
||||
"'I don't have access to any personal information outside of what you tell me', 'I don't have "
|
||||
"personal details outside of our conversation history', 'I do not store personal details "
|
||||
|
||||
@@ -60,6 +60,16 @@ class TestBuildSystemPrompt:
|
||||
assert "SINGLE sentence" in prompt
|
||||
assert "never add it as " in prompt
|
||||
|
||||
def test_persona_has_no_copyable_proper_noun_examples(self):
|
||||
# A weak model parroted the literal "Trenches Gym" example from the
|
||||
# persona as if it were a real user fact (boxing mangled to tennis).
|
||||
# The persona must not embed copyable personal proper nouns, and must
|
||||
# tell the model to use ONLY facts that literally appear in the memory
|
||||
# section — never borrow names/places from the instructions themselves.
|
||||
prompt = build_system_prompt("Jarvis")
|
||||
assert "Trenches" not in prompt
|
||||
assert "never borrow any name, place, or activity from these instructions" in prompt
|
||||
|
||||
|
||||
class TestOutputLanguageDirective:
|
||||
"""A deployment may lock replies to a single language via OUTPUT_LANGUAGE.
|
||||
|
||||
Reference in New Issue
Block a user