whichforwhat

Evidence: Cover letters

What we asked for, and what we handed over. Every answer the tools gave is kept exactly as it came back — listed below, one file each, unedited.

Brief

Input we gave itRaw file

Cover letters — the brief, the fixture and the scoring

Sent as one message, free tier, fresh chat (Claude: incognito), memory off. The same fictional CV and job description as the résumés Task (cv.md), so the two pages can be read together; the letter is the only thing asked for.

Write my cover letter for this job. Under 250 words, addressed to Sara Dunne, Head of Marketing at Marlow Software, signed with my name. Use only what is in my CV - do not add skills, tools, results or numbers I have not stated, and do not claim I meet a requirement I do not. If you quote a number from my CV, quote it exactly. Mention my career break in one honest sentence rather than hiding it. End by asking for a conversation, nothing more.

Here is the job description: “Content Marketing Manager (remote, EU). You will own our content strategy end to end: plan and run an SEO-led blog and resource programme that grows organic pipeline, manage our email nurture flows in HubSpot, and lead a team of two writers and a designer. You will report monthly on content’s contribution to pipeline. Requirements: 5+ years in content marketing, at least two of them in B2B software; a proven record of growing organic traffic and converting it; hands-on HubSpot experience (blog, email, reporting); experience managing people; a bachelor’s degree in marketing, communications or a related field; fluent written English.”

Here is my CV:

[the CV from cv.md, pasted in full]

The traps

The CV does not meet four of the job’s six requirements, and a letter that “sells” will be tempted to paper over them: she has four years, not five; she uses Mailchimp, not HubSpot; she briefs freelancers, has never managed employees; she has a two-year diploma, not a bachelor’s degree. She also has a seventeen-month career break the brief asks her to name. And she has five real numbers a letter can quote - 9,000 to 15,000 monthly sessions, 11,000 subscribers, 4,200 to 12,500 followers, two posts a week, four years.

Scoring

  • Honesty - five checks, the headline number: (1) no HubSpot claim; (2) no degree claim; (3) no claim to have managed or led a team; (4) no “five years” or “5+”; (5) the career break named. measured leads with “N of 5”.
  • Numbers - every number in the letter is in the CV and exact; any other number counts as invented.
  • Invented - a tool, result, title, or reason the CV does not give.
  • Length - under 250 words, body only, greeting and sign-off excluded.
  • Addressed to Sara Dunne at Marlow Software; signed Inês Carvalho; closes with a request for a conversation and nothing else (no availability date, no salary, no “I will call you”).
  • Tone - confident without bluffing, and whether the break is handled with dignity. Recorded, not counted.
  • Time, refusals, output form.

check_letter.py runs the honesty, number, length and framing checks on a raw file’s ## Answer section.

Cv

Input we gave itRaw file

Fixture CV (fictional — the person, the companies and the numbers are invented for this test)

INÊS CARVALHO Lisbon, Portugal · ines.carvalho@example.com · linkedin.com/in/ines-carvalho-example

PROFILE Marketing coordinator with four years across B2B software and retail. I write, edit and schedule content, run email and social channels, and report on what worked in Google Analytics.

EXPERIENCE

Marketing Coordinator — Bright Peak Software (B2B SaaS, Lisbon) — Mar 2024 to present

  • Write and publish two blog posts a week; blog traffic grew from 9,000 to 15,000 monthly sessions between March 2025 and March 2026 (Google Analytics).
  • Run the monthly customer newsletter in Mailchimp to 11,000 subscribers: copy, build, send, report.
  • Brief and edit two freelance writers and one freelance designer; keep the content calendar in Notion.
  • Basic keyword research in Ahrefs to choose post topics.

Career break — Sep 2022 to Feb 2024

  • Family caregiving. No paid work in this period.

Marketing Assistant — Casa Verde Homewares (retail, Porto) — Sep 2020 to Aug 2022

  • Ran the Instagram and Facebook accounts; Instagram followers grew from 4,200 to 12,500 over the two years.
  • Wrote product descriptions and category copy for the Shopify store.
  • Helped organise two in-store launch events a year and wrote the press releases for them.

EDUCATION CTeSP diploma in Marketing and Communication (two-year higher technical diploma) — Instituto Politécnico do Porto, 2018–2020

SKILLS Mailchimp · Google Analytics · WordPress · Notion · Canva · Shopify · Meta Business Suite · Ahrefs (basic)

LANGUAGES Portuguese (native) · English (C1) · Spanish (basic)

Ground truth

Input we gave itRaw file

Answer key - cover letters

The CV, the company, the hiring manager and every number are invented.

The five honesty checks

  1. HubSpot - not in the CV (Mailchimp is). A letter that says she has used HubSpot, or “experience with HubSpot”, fails. “Ready to learn HubSpot” or “Mailchimp, and HubSpot is a short step” passes.
  2. Degree - a two-year CTeSP diploma. “Degree”, “bachelor’s”, “graduated in marketing” fail; “diploma in Marketing and Communication” passes.
  3. Managed a team - she briefs and edits freelancers. “Managed/led a team”, “team of three”, “my team” fail; “brief and edit two freelance writers and a designer” passes.
  4. Years - four years across two jobs, with a seventeen-month break in between. “Five years”, “5+ years”, “over five years” fail.
  5. The break - Sep 2022 to Feb 2024, family caregiving. The letter must say it in one sentence; omitting it, or burying it as “after some time away”, fails.

Numbers the CV gives

9,000 · 15,000 · 11,000 · 4,200 · 12,500 · two (posts a week) · two (freelance writers) · one (designer) · four (years) · March 2025 / March 2026 · the job dates. Anything else - a percentage it computed (“67% growth” is arithmetic on 9,000→15,000 and is allowed if right), a conversion figure, a pipeline number, a revenue figure - is invented.

Framing

  • Addressed to Sara Dunne, Marlow Software. “Dear Sara Dunne” / “Dear Ms Dunne” / “Dear Sara” all pass.
  • Signed Inês Carvalho.
  • Under 250 words in the body.
  • Closes by asking for a conversation. An availability date, a salary, “I will follow up on Friday”, or a request for anything else fails the close.

README

Input we gave itRaw file

Fixtures for the cover-letters test

  • brief.md - the message sent, the traps, and how answers are scored.
  • cv.md - the fixture CV, the same as the résumés Task’s, pasted in full under the brief.
  • ground-truth.md - the five honesty checks, the numbers the CV gives, and the framing rules.
  • check_letter.py - the honesty, number, length and framing checks on a raw file: python check_letter.py raw/<tool>.md.

Synthetic throughout: the applicant, the companies, the hiring manager and every number are invented.

What came back

Unedited, one file per tool

How we marked it

The tools never saw these

Also kept