Job search test — 2026-09

Second round (same day): the candidate set was widened from five to the whole Roster plus two admission candidates; DeepSeek, Grok, Qwen and Kimi were run with the identical brief and checked the same way.

Brief, sent once to each tool, no follow-ups on the first pass, free tier, default model:

I’m a front-end developer with 5 years of React and TypeScript, based in Lisbon, Portugal, open to remote roles anywhere in the EU. Find me 5 current openings for mid-to-senior front-end roles posted in the last two weeks, with a link to each posting. Then draft a short, specific application note (under 120 words) for the one you think fits best, and tell me why you picked it.

The candidate is fictional. Run by Claude Code in the operator’s Chrome session; every output and every link check was reviewed by the operator. Raw outputs are in raw/.

Measured figure: each of the five links was opened on 2026-09-13. An opening counts as wrong if the link is a search or category page rather than a posting, the link is missing, the posting is closed or a different role, or it was posted before 2026-08-30 (the two-week window). Dates were read from the pages’ own structured data.

	ChatGPT	Claude	Gemini	Perplexity	Copilot	DeepSeek	Grok	Qwen	Kimi
Date run	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13
Constraint misses (of 5: five openings, front-end mid/senior, Lisbon or remote EU, ≤ 2 weeks old, a link each)	1	1	1	2	2	3	1	2	1
Openings checked → wrong	5 → 1	5 → 2	5 → 3	5 → 5	5 → 5	5 → 5	5 → 5	5 → 5	5 → 2
What was wrong	Top pick posted 19 days ago	Two openings with no link	Three links are search pages; no dates	All five links are category pages	All five links are category pages; UK and Ukraine roles	No links at all; two openings older than two weeks (said so); one requires residence outside Portugal	Three dead links (410, “Job not found”, expired LinkedIn); two real postings from Dec 2025 and May 2026 — its pick was called “posted around 13 Sep”	All five links are company careers home pages; no search run; roles invented from memory, with an honest disclaimer	One expired link; one official posting created 28 Aug, two days outside the window
Real, open postings among the links	5	3	2	0	0	0 (no links to open)	0 (2 pages render, both expired in their data)	0	4
Note: specific to the posting?	Yes — but written from account memory	Yes — but written from account memory	Yes	Yes, to an unnamed company	Yes	Yes — to KCS iT, with details from the role; signed “[Your Name]”	Yes — to a nine-month-old posting	Yes — to a company, not a posting	Yes — to a live, in-window posting, naming two of its details; no memory contamination
Follow-ups to a usable result	0	1	1	2	2	1	2	2	0
Quality / Effort / Cost / Free tier (0–3)	2/3/3/3	2/2/3/3	1/2/3/3	0/1/3/3	0/1/3/3	1/1/3/3	0/1/3/3	0/1/3/3	2/3/3/3

The memory finding

ChatGPT and Claude both have memory switched on for this account, and both used it: the application notes named the account holder’s real past employers, tooling and background, and ChatGPT signed with the account holder’s first name — none of which was in the brief. For a real job seeker that is arguably the feature working. For a fair test it is a confound, so those details are redacted in the raw files and future persona-based Tasks will be run with memory off. Gemini, Perplexity and Copilot showed no such contamination, and neither did any of the second-round tools.

Notes per tool

ChatGPT

Five LinkedIn postings, all live, titles matching. INSCALE (its pick) posted 2026-08-25; DSV 08-30; Komodo 09-07; Affinity 09-07; Hexa 09-10. It said honestly that none is fully remote.

Claude

Wellhub (Built In, 09-07) and Plain Concepts (Built In, 09-04) live; Jobgether via Arc live, closed: false, posted 09-10. Star and Truv named with no URL. It noticed the Wellhub listing’s “Entry level” tag contradicts a team-lead role.

Gemini

VTEX and Rush Street are live Greenhouse applications. The other three links are WorkingNomads, Indeed and ReactJobs category pages. No dates given.

Perplexity

Three RemoteRocketship and two ReactJobs category pages. Footnote admitted the links are “filtered results pages”. Top pick has no employer name.

Copilot

Three RemoteRocketship and two ReactJobs category pages; two rows share a URL. Two roles “Remote: Ukraine”, one “Remote: United Kingdom”.

DeepSeek

Second round. Named five employers with location and posting age — Redcare (18 Aug) and TrueLogic (May, “reposted”) flagged as old by the answer itself — and gave no URL for any of them. Qonto requires residence in Germany, France, Italy, Serbia or Spain. Note specific to KCS iT.

Grok

Second round. Searched, opened four pages, then linked a freelancermap posting from 9 Dec 2025 as “posted around 13 Sep 2026” (the page’s render date). trabajo.org 410; Ashby “Job not found” (the board has zero open jobs); LinkedIn link redirects with expired_jd_redirect; the Darwin freelancermap posting is from 12 May 2026. Zero inside the window.

Qwen

Admission candidate. Did not search. Five careers home pages (miro, remote.com, elastic, revolut, typeform) with role titles invented from memory, prefaced by an honest disclaimer that it has no live browsing. Retitled the persona “Senior”. Note 92 words, to a company.

Kimi

Admission candidate. Five rounds of searches and six page opens, reasoning shown; dropped a Ukraine role for not being EU. Plain Concepts (Workable, published 09-04, Portugal remote) and Octopus Energy (Lever, 09-11, Barcelona/Valencia) live and in window; Qualysoft (09-10, Budapest) live; journaway (Personio, official) live but created 08-28, two days too old — Kimi quoted an aggregator’s “Sep 2”; IT Labs link expired (410). Its pick is the Portugal-remote posting, and the 73-word note names the role’s 3D/geospatial work and six-month initial term. The only tool whose five were all remote-EU.