# Shopping test — 2026-09

**Brief, sent once to each tool, no follow-ups on the first pass, free tier, default model:**

> I'm in Portugal. Find me the best cordless stick vacuum under €300 that I can buy online today from a shop that ships to Portugal, for a 70 m² flat with hard floors and one long-haired cat. Give me 3 options with the current price, a link to the product page, and one line on why each. Then pick one and tell me why. If a well-known model is over budget right now, say so instead of listing it.

Run by Claude Code in the operator's Chrome session between 17:19 and 18:10 (+07) on 2026-09-13, one message per tool; every output and every price check was reviewed by the operator. Raw outputs are in `raw/`.

**Measured figure:** each of the three links opened within the hour. A link counts as right if it is a page for that one product (a shop's product page, or an aggregator's single-product page with live offers), the product exists and is buyable today, the stated price is within 10% of the page's price, and the shop ships to Portugal. Where a shop blocked the automation (Worten and Castro Electrónica block scripts; Castro also blocks the browser), the same product's live offers on KuantoKusta, which quotes the shop by name, were used. Amazon.es was checked with the delivery country set to Portugal — with the browser's default location its listings show as unavailable. Fnac.pt was in maintenance for the whole window, so anything linked there is recorded as unverifiable rather than wrong.

| | ChatGPT | Claude | Gemini | Perplexity | Copilot | DeepSeek | Grok | Qwen | Kimi |
|---|---|---|---|---|---|---|---|---|---|
| Date run | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 | 2026-09-13 |
| Constraint misses (of 5: three options, under €300, ships to PT, a link each, a pick with a reason) | 1 (one link missing) | 1 (no links) | 1 (no links) | 1 (one shop is UK) | 0 | 0 | 0 | 0 | 5 (no answer) |
| Links opened → right | 3 → 2 (Samsung has no link of its own) | 3 → 0 | 3 → 0 | 3 → 1 | 3 → 2 | 3 → 2, 1 unverifiable | 3 → 3 | 3 → 1 | — |
| Prices stated → confirmed | 3 → 3, all exact | 3 → 2 | 3 → 0 (no shop named) | 3 → 1 | 3 → 3 (Dyson within 10%, but unavailable) | 3 → 2, 1 unverifiable | 3 → 3, given as ranges | 3 → 3, all exact | — |
| Buyable today | 3 of 3 | 2 of 3 (Bosch page 404) | unknown | 1 of 3 | 2 of 3 | 2 of 3 | 3 of 3 | 1 of 3 — the pick is out of stock | — |
| Over-budget models handled as asked? | yes — Jet 75E, Dyson V8 with prices | yes — Tineco S15 | yes — Dyson V8, Shark | yes — Dyson V12, Samsung, Shark | yes — V8 Absolute | yes — V8 Absolute | yes — V10/V15, Shark | partly — Dyson V8 "€350–500+" is wrong | — |
| Follow-ups to a buyable result | 0 | 1 | 2 | 1 | 0 | 0 | 0 | 1 | — |
| Output form | table + pick, citation chips | comparison card, no links | plain text, A/B pair, offers follow-ups | table with 77 sources, asks a question | table with a URL column | markdown list with full URLs | markdown list with "link example" URLs | list with "View on…" links | refusal toast |
| Quality / Effort / Cost / Free tier (0–3) | 2/3/3/3 | 1/2/3/3 | 1/1/3/3 | 1/2/3/3 | 2/3/3/3 | 2/3/3/3 | 2/3/3/3 | 1/2/3/3 | 0/0/3/1 |
| What stood out | Every price exact, every item in stock, pet-specific reasons; forgot one link | Best reasoning, zero links | Two contradictory shortlists, no shops | Linked a UK store in pounds as "Hoover EU" | The clearest link table; one item unavailable | Real Portuguese shop pages, cheapest set | Every link live and in stock; prices as ranges | Three exact prices for two things you cannot buy | "Too many people are chatting with Kimi" |

## Notes per tool

### ChatGPT
Dreame R20 SE €159.99 at Worten (sold by Worten, free delivery), Xiaomi G20 €161.59 and Samsung Jet 65 Pet €173.89 at Castro Electrónica — all three exact, confirmed via KuantoKusta's per-shop offers because Castro blocks the automation. The Samsung row's citation chip points at the Xiaomi page: no link of its own. Named the Jet 75E and Dyson V8 as over budget with prices rather than listing them.

### Claude
Twelve searches, eight fetches, three failed (PCDIGA and Worten product pages). Cecotec €179 confirmed on cecotec.pt (cheaper at You Get, €158.90); Rowenta 9.60 €189.90 at PCDIGA confirmed; Bosch Unlimited 6 "€209.90 at PCDIGA" — PCDIGA's page is a 404, the model sells elsewhere at €182–212. No hyperlink anywhere in the answer; the comparison card is not clickable. Its research log labelled two steps "robot vacuum".

### Gemini
Served an A/B pair and asked which was more helpful. Answer A (scored): three models, no shops, no links; "Proscenic P15 / P11 Ultra" is two models with one price, neither stocked in PT. Answer B: one real Worten link (Bosch Flexxo, €250.12 exact — but sold by marketplace seller Morele, so "buying directly from Worten ensures … local warranty" is wrong), a "Galaxus AT" shop that delivers within Austria, and an unlinked Tradeinn item.

### Perplexity
Rowenta 9.60 Animal Care at rowenta.pt €269.99 exact and buyable — the same model is €189.90 at PCDIGA the same hour, so its pick costs €80 more than it needs to. Hoover HF3 linked at hoover-home.com/en_GB, £179.99 in GBP, labelled "Hoover EU – ships to PT". Dreame T20 linked to a KuantoKusta page with no offers ("Temporariamente indisponível"). Closed by asking a preference question instead of the "buy now" link the brief asked for.

### Copilot
Shark IW3612EU via KuantoKusta (aggregator, 15 live PT offers from €279.90 — counted as right); Rowenta X-PERT 7.60 at rowenta.pt €139.99 exact; Dyson V8 Advanced at Worten €221.77 (within 10% of €234.87) but "De momento indisponível online". Seventeen reasoning actions. A URL column in the table is the most usable link presentation of the nine.

### DeepSeek
Xiaomi G20 at Auchan €179.99 exact; Rowenta Dual Force 2-in-1 at Auchan €129.99 exact; Dreame T30 Flex at Fnac.pt €224.05 unverifiable — fnac.pt was in maintenance for the whole window and the model is not on KuantoKusta. Recorded as unverifiable, not wrong. Its pick is €18 dearer than the same Xiaomi at Castro Electrónica.

### Grok
Three links, three live listings, all in stock with delivery to Portugal: Dreame R20 €228.72 and Shark Detect Pro IW1611EU €237.69 on Amazon.es (checked with the delivery country set to Portugal — with the browser's default location both showed as unavailable), Levoit LVAC-200 €149.99 at Worten. Prices given as ranges ("~€202–229") with "check the product pages", which is honest but is not the current price the brief asked for. Shark's range (~€260–270) is 9% above the live price.

### Qwen
Web search ran this time. Three real product pages with three exact prices — Cecotec RX50 Pet Flex €149 (PcComponentes), Dreame R10 €129.99 (Worten), Ultenic U18 €119.99 (PcComponentes) — of which only the Ultenic can be bought today: the Cecotec, its pick, shows "no exact arrival date" and the Dreame is "unavailable online". "Dyson V8 €350–500+" is wrong (a V8 Advanced was €221.77 at Worten the same hour). First render truncated again; full answer after reload.

### Kimi
Refused four times over twenty minutes with "Too many people are chatting with Kimi right now. Subscribe to get into a special priority queue" — the composer emptied each time and the second attempt opened the pricing page. The same free account had answered three briefs earlier in the day. No output to score; the free tier was not available when we needed it.