Evidence: Creative writing
What we asked for, and what we handed over. Every answer the tools gave is kept exactly as it came back — listed below, one file each, unedited.
Brief
Creative writing — the brief, the constraints and the scoring
Sent as one message, free tier, fresh chat (Claude: incognito), memory off. No fixture is pasted under it; the constraints are the fixture.
Write me a short story, 700 to 800 words, under a title. First person, present tense, one scene in one room, in real time. Two people speak and nobody else appears. It must begin with exactly this sentence: “The book was due on the ninth of June, 1995.” It must end with exactly this sentence: “I keep the receipt.” A library card appears twice, and means something different the second time. Keep spoken dialogue under 80 words in total. Do not use any of these words: suddenly, realise, realize, breath, heart, silence, shattered, tears, whisper, echo, somehow. No dream, no ghost, no dead narrator, no twist that reframes the story, and no final paragraph that explains what the story meant. Just the story.
The traps
- The two fixed sentences. Models “improve” a given first line and soften a given last line - “The book had been due…”, “I kept the receipt.” Both must be verbatim.
- The banned list. Eleven words that model fiction reaches for by reflex, “suddenly”, “realise”, “heart”, “breath”, “silence” above all. Each use counts.
- Present tense, first person, real time. The drift is into past tense by the second page, and into a flashback that breaks “one scene in one room”.
- Two speakers, nobody else. A third person walking in, a remembered voice quoted, a phone call - all count as someone else appearing.
- The library card twice, meaning changed. A checkable device: it must appear twice, and the second time do something the first did not (a proof of identity, then a proof of who she was; a way in, then a way back). Read, not counted.
- Under 80 words of dialogue. Models write stories as conversations. This forces the story into what is seen and done.
- The twist and the moral. No ghost, no dream, no “I understand now that…” paragraph. The ending is the receipt.
- 700 to 800 words below the title, a window not a floor.
Scoring
- Constraints broken - the headline: first sentence exact, last sentence exact, each banned word used, tense or person drift outside dialogue, a third person, a scene change, a twist or a moral paragraph.
measuredleads with “N broken”. - The card - appears twice, meaning changed. Recorded as yes / once / no change.
- Dialogue - words inside quotation marks, against 80.
- Length - 700 to 800 words below the title.
- The story - does anything happen; is the ending earned; what the prose does with a room and two people. Recorded.
- Time, refusals, output form.
check_story.py runs the sentence, banned-word, tense, person, dialogue, card-count, length and form checks on a raw file’s ## Answer section.
Ground truth
Answer key - creative writing
Must be exact
- First sentence below the title: The book was due on the ninth of June, 1995.
- Last sentence: I keep the receipt.
Must not appear (each use counts)
suddenly · realise · realize · realised · realized · breath · heart · silence · shattered · tears · whisper · echo · somehow (and their inflections: breathe/breathing, whispered, echoes, heartbeat, tearful).
Must hold
- First person, present tense, outside dialogue. “I was”, “she said”, “I had” are drift.
- One scene, one room, real time. A flashback rendered as scene, a cut to another place or day, a phone call, breaks it.
- Two people speak; no third person appears, is quoted, or calls.
- A library card, twice, with a change of meaning between the two.
- Spoken dialogue under 80 words in total.
- 700 to 800 words below the title.
- A title. No preface, no note, no offer, no paragraph after the last sentence.
Must not be
- A dream, a ghost, a dead narrator, an unreliable-narrator reveal, a “twist” that reframes what came before.
- A closing paragraph that states the meaning (“I understand now”, “It was never about the book”).
README
Fixtures for the creative-writing test
brief.md- the message sent (the constraints are the fixture), the traps, and how answers are scored.ground-truth.md- the two fixed sentences, the banned list, what must hold and what must not be.check_story.py- fixed-sentence, banned-word, tense, person, dialogue, card, length and form checks on a raw file:python check_story.py raw/<tool>.md.
Nothing personal or real in the brief; the date and the book are the story’s own.
What came back
- ChatGPTmd
- Claude first attemptmd
- Claudemd
- Copilotmd
- DeepSeekmd
- Geminimd
- Grokmd
- Kimimd
- Perplexitymd
- Qwenmd
How we marked it
- fixtures/check_story.pypy3 KB
Also kept
- The test (2026-09)The test, written up
- results-2026-09.yamlyaml5 KB