Shopping test — 2026-09

Brief, sent once to each tool, no follow-ups on the first pass, free tier, default model:

I’m in Portugal. Find me the best cordless stick vacuum under €300 that I can buy online today from a shop that ships to Portugal, for a 70 m² flat with hard floors and one long-haired cat. Give me 3 options with the current price, a link to the product page, and one line on why each. Then pick one and tell me why. If a well-known model is over budget right now, say so instead of listing it.

Run by Claude Code in the operator’s Chrome session between 17:19 and 18:10 (+07) on 2026-09-13, one message per tool; every output and every price check was reviewed by the operator. Raw outputs are in raw/.

Measured figure: each of the three links opened within the hour. A link counts as right if it is a page for that one product (a shop’s product page, or an aggregator’s single-product page with live offers), the product exists and is buyable today, the stated price is within 10% of the page’s price, and the shop ships to Portugal. Where a shop blocked the automation (Worten and Castro Electrónica block scripts; Castro also blocks the browser), the same product’s live offers on KuantoKusta, which quotes the shop by name, were used. Amazon.es was checked with the delivery country set to Portugal — with the browser’s default location its listings show as unavailable. Fnac.pt was in maintenance for the whole window, so anything linked there is recorded as unverifiable rather than wrong.

	ChatGPT	Claude	Gemini	Perplexity	Copilot	DeepSeek	Grok	Qwen	Kimi
Date run	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13	2026-09-13
Constraint misses (of 5: three options, under €300, ships to PT, a link each, a pick with a reason)	1 (one link missing)	1 (no links)	1 (no links)	1 (one shop is UK)	0	0	0	0	5 (no answer)
Links opened → right	3 → 2 (Samsung has no link of its own)	3 → 0	3 → 0	3 → 1	3 → 2	3 → 2, 1 unverifiable	3 → 3	3 → 1	—
Prices stated → confirmed	3 → 3, all exact	3 → 2	3 → 0 (no shop named)	3 → 1	3 → 3 (Dyson within 10%, but unavailable)	3 → 2, 1 unverifiable	3 → 3, given as ranges	3 → 3, all exact	—
Buyable today	3 of 3	2 of 3 (Bosch page 404)	unknown	1 of 3	2 of 3	2 of 3	3 of 3	1 of 3 — the pick is out of stock	—
Over-budget models handled as asked?	yes — Jet 75E, Dyson V8 with prices	yes — Tineco S15	yes — Dyson V8, Shark	yes — Dyson V12, Samsung, Shark	yes — V8 Absolute	yes — V8 Absolute	yes — V10/V15, Shark	partly — Dyson V8 “€350–500+” is wrong	—
Follow-ups to a buyable result	0	1	2	1	0	0	0	1	—
Output form	table + pick, citation chips	comparison card, no links	plain text, A/B pair, offers follow-ups	table with 77 sources, asks a question	table with a URL column	markdown list with full URLs	markdown list with “link example” URLs	list with “View on…” links	refusal toast
Quality / Effort / Cost / Free tier (0–3)	2/3/3/3	1/2/3/3	1/1/3/3	1/2/3/3	2/3/3/3	2/3/3/3	2/3/3/3	1/2/3/3	0/0/3/1
What stood out	Every price exact, every item in stock, pet-specific reasons; forgot one link	Best reasoning, zero links	Two contradictory shortlists, no shops	Linked a UK store in pounds as “Hoover EU”	The clearest link table; one item unavailable	Real Portuguese shop pages, cheapest set	Every link live and in stock; prices as ranges	Three exact prices for two things you cannot buy	“Too many people are chatting with Kimi”

Notes per tool

ChatGPT

Dreame R20 SE €159.99 at Worten (sold by Worten, free delivery), Xiaomi G20 €161.59 and Samsung Jet 65 Pet €173.89 at Castro Electrónica — all three exact, confirmed via KuantoKusta’s per-shop offers because Castro blocks the automation. The Samsung row’s citation chip points at the Xiaomi page: no link of its own. Named the Jet 75E and Dyson V8 as over budget with prices rather than listing them.

Claude

Twelve searches, eight fetches, three failed (PCDIGA and Worten product pages). Cecotec €179 confirmed on cecotec.pt (cheaper at You Get, €158.90); Rowenta 9.60 €189.90 at PCDIGA confirmed; Bosch Unlimited 6 “€209.90 at PCDIGA” — PCDIGA’s page is a 404, the model sells elsewhere at €182–212. No hyperlink anywhere in the answer; the comparison card is not clickable. Its research log labelled two steps “robot vacuum”.

Gemini

Served an A/B pair and asked which was more helpful. Answer A (scored): three models, no shops, no links; “Proscenic P15 / P11 Ultra” is two models with one price, neither stocked in PT. Answer B: one real Worten link (Bosch Flexxo, €250.12 exact — but sold by marketplace seller Morele, so “buying directly from Worten ensures … local warranty” is wrong), a “Galaxus AT” shop that delivers within Austria, and an unlinked Tradeinn item.

Perplexity

Rowenta 9.60 Animal Care at rowenta.pt €269.99 exact and buyable — the same model is €189.90 at PCDIGA the same hour, so its pick costs €80 more than it needs to. Hoover HF3 linked at hoover-home.com/en_GB, £179.99 in GBP, labelled “Hoover EU – ships to PT”. Dreame T20 linked to a KuantoKusta page with no offers (“Temporariamente indisponível”). Closed by asking a preference question instead of the “buy now” link the brief asked for.

Copilot

Shark IW3612EU via KuantoKusta (aggregator, 15 live PT offers from €279.90 — counted as right); Rowenta X-PERT 7.60 at rowenta.pt €139.99 exact; Dyson V8 Advanced at Worten €221.77 (within 10% of €234.87) but “De momento indisponível online”. Seventeen reasoning actions. A URL column in the table is the most usable link presentation of the nine.

DeepSeek

Xiaomi G20 at Auchan €179.99 exact; Rowenta Dual Force 2-in-1 at Auchan €129.99 exact; Dreame T30 Flex at Fnac.pt €224.05 unverifiable — fnac.pt was in maintenance for the whole window and the model is not on KuantoKusta. Recorded as unverifiable, not wrong. Its pick is €18 dearer than the same Xiaomi at Castro Electrónica.

Grok

Three links, three live listings, all in stock with delivery to Portugal: Dreame R20 €228.72 and Shark Detect Pro IW1611EU €237.69 on Amazon.es (checked with the delivery country set to Portugal — with the browser’s default location both showed as unavailable), Levoit LVAC-200 €149.99 at Worten. Prices given as ranges (“€202–229”) with “check the product pages”, which is honest but is not the current price the brief asked for. Shark’s range (€260–270) is 9% above the live price.

Qwen

Web search ran this time. Three real product pages with three exact prices — Cecotec RX50 Pet Flex €149 (PcComponentes), Dreame R10 €129.99 (Worten), Ultenic U18 €119.99 (PcComponentes) — of which only the Ultenic can be bought today: the Cecotec, its pick, shows “no exact arrival date” and the Dreame is “unavailable online”. “Dyson V8 €350–500+” is wrong (a V8 Advanced was €221.77 at Worten the same hour). First render truncated again; full answer after reload.

Kimi

Refused four times over twenty minutes with “Too many people are chatting with Kimi right now. Subscribe to get into a special priority queue” — the composer emptied each time and the second attempt opened the pricing page. The same free account had answered three briefs earlier in the day. No output to score; the free tier was not available when we needed it.