# Interview preparation test — 2026-09

**Brief, sent once to each tool (message 1), then one scripted answer (message 2), free tier, default model.** Both messages are in `fixtures/brief.md`; the job description and background are fictional.

> I have an interview on Thursday for a Customer Success Manager role at a B2B software company in Lisbon. [JD: 40–60 mid-market accounts, onboarding, QBRs, renewals, net churn under 5%, HubSpot daily; requires 3+ years SaaS CS/support, a renewal track record, finance/ops presenting, EN+PT, Spanish a plus.] [Background: five years hotel front-office management (180 rooms, team of nine, satisfaction scores, escalations, corporate accounts); one year support agent at a payments startup (Zendesk, merchant onboarding calls, engineering escalations); PT native, EN C1, conversational ES; no HubSpot, has used Zendesk and Salesforce Service Cloud.] Give me the 8 questions they are most likely to ask, with a strong answer outline for each that uses MY background, not a generic one. Then start a mock interview: ask me the first question and wait for my answer.

> **Message 2 (identical everywhere):** My answer: I think my biggest strength is that I'm good with people. In the hotel I dealt with guests every day and they were usually happy with me. At the startup I answered tickets and tried to be fast. I would say I'm organised and I like helping customers succeed, and I'm a quick learner so HubSpot wouldn't be a problem.

Run by Claude Code in the operator's Chrome session on 2026-09-13 between 21:20 and 22:20 (+07); every output was read and every check reviewed by the operator. Raw outputs are in `raw/`.

**Memory off.** ChatGPT (temporary chat, "without personalization" — the temporary chat's default still uses memory and had to be switched), Claude (incognito), Gemini (temporary chat), Perplexity (incognito), Copilot (temporary chat) and Grok (private) were run in modes that exclude account memory. DeepSeek and Qwen have no memory feature. Kimi has one and no private mode; its answers were checked for anything not in the brief and contained nothing.

**Measured figure:** did the tool ask one question and stop ("stopped and waited"), and how many facts about the candidate did it invent — a claim in an outline or a model answer that is not in the pasted background. Also counted: outlines that use at least one fact from the background (of 8), whether the real gap (no renewal record, no HubSpot) is named rather than papered over, and the quality of the feedback on the scripted answer.

| | ChatGPT | Claude | Gemini | Perplexity | Copilot | DeepSeek | Grok | Qwen | Kimi |
|---|---|---|---|---|---|---|---|---|---|
| Date run | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 |
| Stopped and waited after one question | yes | yes | yes | yes | yes | yes | yes | yes | yes |
| Outlines using a fact from the background (of 8) | 8 | 8 | 4 | 8 | 8 | 8 | 8 | 8 | 8 |
| Invented facts about the candidate | 0 | 1 (owned a support SLA) | 4 (40–60 daily interactions, contract retention, written Spanish, cross-border comms) | 4 (post-event reviews, KPI reviews, grew corporate spend, spotted upsell merchants) | 1 (VIP guests) | 2 (quarterly reviews; a scripted hotel save) | 3 (group bookings, account reviews, new PMS) | 3 (already on HubSpot Academy, reconciled billing with finance, written Spanish) | 4 (presented to finance departments, saved accounts, upsold event space, hit an SLA) |
| Real gap named, not hidden | yes — HubSpot and "don't pretend" on renewals | yes — "this is the real gap, show method" | partly | HubSpot yes; coaches "kept €Xk ARR" | yes — "I haven't held the title" | yes — 30/60/90 plan | HubSpot yes; renewals mapped from guest scores | HubSpot yes; renewals not named | yes — bluntest renewal question; "don't bluff" |
| Feedback on the scripted answer | quotes 3 phrases, rewrites, asks Q2 | 6 faults, beats not a rewrite, offers retry or Q2 | generic framework again, re-asks Q1 | rewrite, asks Q2, upsell banner | rewrite, 6/10 → 8.5/10, asks Q2 | bluntest, skeleton, re-asks Q1 | 5 faults, rewrite, asks Q2 | "You said / The Upgrade" pairs, rewrite, asks Q2 | 5 faults incl. one nobody else caught, rewrite, asks the hardest Q2 |
| Free tier delivered first time | yes | yes | yes | yes | yes | yes | no — "High load", answered on the third try | yes | yes (after a clean redo of a contaminated send — our error) |
| Quality / Effort / Cost / Free tier (0–3) | 3/3/3/3 | 3/3/3/3 | 1/1/3/3 | 1/2/3/3 | 2/3/3/3 | 2/3/3/3 | 2/2/3/2 | 2/3/3/3 | 2/3/3/3 |
| What stood out | Nothing invented, everything used, honest on the gap | The most honest prep — names the real gap and gives beats, not a script | Half the outlines are generic frameworks | Coaches the candidate to claim ARR saves they never made | Scores the answer and fixes the HubSpot line precisely | Finds the sharpest questions (billing domain, Spanish stakeholder, define net churn) | Fine prep, but the free tier said "busy" first | Clearest side-by-side critique; tells the candidate they've started a course they haven't | The best interview coach of the nine, with four things the candidate never did |

## Notes per tool

### ChatGPT
Temporary chat with personalization off. Eight outlines, every one anchored in the brief (180 rooms, team of nine, corporate accounts, Zendesk, Salesforce, merchant calls, engineering escalations, the three languages), plus a ninth "biggest objection" it expects. On renewals: "you don't have direct SaaS renewal ownership, so don't pretend you do." Feedback quotes three phrases from the answer, says "I would not give this answer yet", rewrites it from the background without adding anything, then asks Q2. No inventions found.

### Claude
Incognito. Q4 is "You don't have a track record of hitting SaaS renewal targets" — "this is the real gap … show method instead of a fake track record"; Q6 concedes no formal QBR. Feedback is the most direct verdict ("would blend into every other candidate's") with six faults, then three beats to rebuild rather than a script, and a choice: retry or Q2. One soft invention: a suggested line claims the candidate "owned a support SLA". Ran at 75% of a weekly limit.

### Gemini
Temporary chat. Only four of eight outlines touch the candidate's history; Q2, Q4, Q5, Q6 are CS frameworks anyone could get. Four inventions, including "managed 40–60 daily high-stakes interactions" (the JD's account count re-attributed to the hotel job) and "professional written Spanish" (the brief says conversational). The feedback repeats the framework and re-asks Q1 with a "Да" chip to skip.

### Perplexity
Incognito; ran a 25-second web search and cited 15 sources for an interview-prep task. Every outline has a "Your edge" tied to the background, but four are invented (post-event reviews with corporate clients, reviewing merchants' KPIs, growing corporate spend with events/F&B/room blocks, spotting merchants ready for higher tiers), and Q2/Q7 coach the candidate to quantify "kept €Xk ARR" and "+20% ARR" — stories the background does not contain. Feedback includes a full model answer and Q2, under an "Upgrade to Pro" banner.

### Copilot
Temporary chat. Outlines carry a "What they're testing" line each; Q3 is the gap question and coaches "I haven't held the title"; Q7 concedes no formal QBRs. Feedback: a rewrite in the candidate's voice, a score (6/10 → 8.5/10), and a precise fix for the HubSpot line. One soft invention ("VIP guests"). A list of questions to ask the interviewer at the end.

### DeepSeek
No private mode. The most tailored question set: "You've never worked in B2B SaaS billing", "How do you handle a stakeholder who prefers Spanish", "What does net churn under 5% mean". A 30/60/90 ramp plan. Two inventions: "quarterly reviews with corporate accounts", and a fully scripted hotel save (bad group stay, housekeeping/F&B) presented as the example to tell. Feedback is the bluntest ("filler", "that's the whole job") and re-asks Q1 with a skeleton.

### Grok
Private mode. First run answered but was lost when the page was reloaded to fix Grok's non-rendering answer body (private chats are discarded on reload — our error). Second attempt refused: "High load — retry later or move to a higher plan for priority access." Third attempt, ten minutes later, answered in 8 s. Eight solid outlines; three inventions (group bookings, corporate-account reviews, a new PMS). Feedback rewrites the answer and asks Q2, but keeps the "quick learner … HubSpot will be straightforward" line the other tools flagged as weak.

### Qwen
No private mode; web search did not run. Eight tailored outlines and the clearest critique format — three "You said / The Upgrade" pairs quoting the scripted answer — then a rewrite and Q2 in character. Three inventions, the worst being "you've already started watching HubSpot Academy tutorials", repeated in the model answer as something the candidate has done. Page text truncates its replies; both were copied via the copy button.

### Kimi
Memory on (no private mode); no leakage found. First send was contaminated by leftover text in the composer from its refused shopping run earlier in the evening, stopped, and redone in a clean chat. The one tool that led with the angle that matters most — the payments startup is the same domain as a billing platform ("your home-run question") — and it put the renewal gap as bluntly as an interviewer would. The feedback caught something nobody else did: volunteering the HubSpot weakness in the opener. Four inventions, though: presenting to finance departments, saving accounts, upselling event space, hitting an SLA.