# Gardening test — 2026-09

**Brief, sent once per photo to each tool, one message, no follow-ups, free tier, default model, photo attached** (`fixtures/brief.md`):

> What plant is this, what is wrong with it, and what exactly should I do this week? It's in a pot on my north-facing terrace in Lisbon and I water it every 3 days.

**Deviation from the planned protocol.** The plan used two photos of the operator's own plants. These are two openly licensed photographs from Wikimedia Commons whose plant and problem were identified by the photographers, one of them a plant pathologist - which gives a ground truth to score against. Because both are outdoor shots, "north-facing windowsill" in the brief became "north-facing terrace". Licences, sources, ground truth and the reference plan for each are in `fixtures/README.md`.

- **Photo A** (`fixtures/plant-a.jpg`): *Clivia miniata* with citrus mealybug, *Planococcus citri* - white cottony masses on the strap leaves. A good answer says a Clivia watered every 3 days is too wet.
- **Photo B** (`fixtures/plant-b.jpg`): a bronze-leaved wax begonia (*Begonia semperflorens* type) with powdery mildew, *Oidium begoniae*. A good answer blames wet foliage and still air rather than the watering interval, and keeps water off the leaves.

Run by Claude Code in the operator's Chrome session on 2026-09-14 between 13:02 and 13:36 (+07) for eight tools, and at 15:55–15:58 for Copilot, whose first image upload raised a one-time consent dialog that the operator approved; every output read against the ground truth and every judgement reviewed by the operator. Each photo was a new chat. Qwen's and Kimi's composers ignore an injected file, so they received the same Commons photograph at 1280 px through an in-page paste or drop; the other seven got the 1400 px (A) and 1200 px (B) files in `fixtures/`. Raw outputs are in `raw/`, with Kimi's Russian answer to B translated there.

**Measured figure: plants identified, of 2.** Genus right counts as identified; the second number is problems diagnosed, of 2. Also scored: whether the plan is specific to the diagnosis rather than generic care; whether the answer uses the north-facing terrace and the 3-day watering; anything in the plan that would harm the plant; follow-ups; output form.

| | ChatGPT | Claude | Gemini | Perplexity | Copilot | DeepSeek | Grok | Qwen | Kimi |
|---|---|---|---|---|---|---|---|---|---|
| Date run | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 | 2026-09-14 |
| Plants identified, of 2 | 2 | 2 | 2 | 1 | 2 | 1 | 2 | 2 | 1 |
| Photo A: plant | Clivia miniata | Clivia miniata | Clivia miniata | **Dracaena** | Clivia miniata, "less certain than the pest" | **Yucca, or Hippeastrum** | Clivia miniata | Clivia miniata | **Agave americana** |
| Photo B: plant | wax begonia, bronze-leaved | begonia, wax / dragon-wing | wax begonia | wax begonia, bronze-leaved | wax begonia, bronze-leaved, "90% likely" | begonia, "likely rex" | wax begonia, bronze-leaved | begonia, "rex or flowering hybrid" | wax begonia |
| Problems diagnosed, of 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 2 |
| Species named | — | — | — | — | — | — | **both: *Planococcus citri*, *Oidium begoniae*** | — | *Oidium begoniae* on B |
| Plan specific to the diagnosis? | both: alcohol swab, soap next day, re-inspect at 7 and 14 days, check the crown; prune to the bin, fungicide, water the soil, no repot, no feed | both: six-step stepper for A; potassium bicarbonate named first for B | both, short | both, the fullest: patch test, sealed bag, fan, "do not" list | both, with a "how serious" line | A: alcohol, then **"a systemic insecticide such as Acaricide (imidacloprid)"** as spray and drench; B: sulphur / neem / soda | both; B adds a milk spray | both | A: sound steps **plus "move it to full sun" and repot in cactus mix** - right for an agave, wrong for a Clivia; B: mist before cutting so spores don't fly |
| Uses the context? | A: 3 days "too frequent", 7–14 days instead; B: "too rigid", separates soil moisture from wet foliage, north terrace fine but move to the open | A: "can be too frequent"; B: "probably fine volume-wise", water the soil in the morning | A: "far too frequent"; B: "likely too frequent", never wet the foliage | A: too frequent in low light; B: "not automatically wrong", check the soil | A: "may be more frequent than needed"; B: "strongly suspect" over-watering | A: "likely too frequent"; B: "might be too much" | A: "may be contributing"; B: "a bit frequent", stagnant air matters more | A: "too frequent", 7–10 days; B: "too frequent" | A: too wet **and too shaded**; B: "not itself to blame" |
| Anything harmful? | no | no | no | no | no | **a mite killer named as the insecticide; imidacloprid is not sold to amateurs in the EU** | no | no | **full sun for a Clivia** |
| Answer language | English | English | English | English | English | English | English | English | A English, **B Russian** |
| Follow-ups needed | 0 | 0 | 0 | 1 (wrong plant on A) | 0 | 1 (wrong plant on A) | 0 | 0 | 2 (wrong plant on A; B in Russian) |
| Output form | prose, day-by-day; B had 5 source chips; ~15 s | prose + 6-step stepper widget; ~30 s | one screen each; ~15 s; B under a **medical** disclaimer banner | text with extension-service sources (16 each), two "Computer" agent offers, three chips; 10 s | prose, today/tomorrow/later; three chips; ~20 s | markdown with a table; search on; ~30 s | markdown after 6 searches; ~30 s | A came as **two answers side by side to rate**; B one answer with citations; ~45 s | prose with visible reasoning; ~35 s |
| Quality / Effort / Cost / Free tier (0–3) | 3/3/3/3 | 2/2/3/3 | 2/2/3/3 | 2/3/3/3 | 2/3/3/3 | 1/2/3/3 | 3/3/3/3 | 2/2/3/3 | 1/2/3/3 |
| What stood out | Both right, and the plan a gardener would write: check the crown, don't repot a stressed plant, a Portuguese sentence for the counter, and the point that mildew is about wet leaves, not wet soil. | Both right; a stepper you can follow at the plant; the one tool that did not blame the watering for the mildew. | Both right on one screen each - and a medical disclaimer on a begonia. | Read a Clivia as a Dracaena, then wrote the best-sourced plans of the nine. | Both right and says how sure it is - "less certain of the plant than the pest", "90% likely". | Read a Clivia as a Yucca and prescribed a mite killer; got the begonia. In English this time. | The only tool to name both species exactly; every step sound. | Both right; served photo A as an A/B vote. | Read a Clivia as an agave and told you to put it in full sun; answered the begonia in Russian. |

## Notes per tool

### ChatGPT

Photo A: Clivia miniata "or a closely related Clivia", mealybugs, and a today / tomorrow / 7 days / 14 days plan - wash, alcohol on a cotton bud with a patch test, soap next day, re-inspect twice because eggs survive, cut heavily infested leaves at the base, look inside the crown. Says 3-day watering is too frequent for a shaded Clivia and gives the finger test with a 7–14-day expectation. Offers the Portuguese product name as a follow-up. Photo B: wax begonia, bronze-leaved; severe powdery mildew; the one answer that explains the disease is about damp foliage and still air while the roots can be dry; prune the worst leaves to the bin, keep the rest, thin, fungicide with a full sentence for a Portuguese garden centre and a warning about sulphur in heat; water the soil in the morning; move to the most open spot; don't repot, don't feed this week; judge by new growth. Nothing wrong in either.

### Claude

Photo A: Clivia miniata with a differential against amaryllis and agapanthus; mealybugs with the honeydew and sooty-mould consequence; six steps in a stepper widget (isolate, alcohol, soap or neem, check the soil surface, repeat weekly for three weeks, cut the watering). Photo B: "wax/dragon-wing type"; powdery mildew; prune to the bin, airflow, water the soil in the morning, potassium bicarbonate named as the most effective option, then neem or baking soda, repeat every 7–10 days. On B it says the 3-day interval "is probably fine volume-wise" and to stretch it only if the top inch stays damp - which matches the reference better than the tools that blamed the schedule.

### Gemini

Photo A: Clivia miniata / Bush Lily; mealybugs; four dated steps (alcohol day 1, wash day 2, adjust day 3, soap or neem day 7 and weekly); "far too frequent" watering; suggests a brighter spot or mild morning sun, which a Clivia tolerates. Photo B: wax begonia; powdery mildew; five steps, correct and brief; "never wet the foliage". The B answer arrived under Gemini's health-advice disclaimer ("if you need medical advice or a diagnosis, consult a specialist"), in Russian.

### Perplexity

Photo A: "a Dracaena, probably Dracaena fragrans" - wrong; a Clivia's strap leaves read as Dracaena. Mealybugs right, "not mildew". The plan is sound regardless of the name - isolate, alcohol with a 24-hour patch test, soap or oil labelled for mealybugs, no feed, no repot, water when the top half is dry, re-treat every 7–10 days for three or four rounds, keep isolated two weeks - and it dates the next step ("September 21"). Photo B: wax begonia, bronze-leaved; powdery mildew; the fullest plan of the nine, from extension handbooks: disinfected scissors, a sealed bag, thin no more than a third, a small fan, potassium bicarbonate or a registered fungicide with a patch test, and a "do not" list (no vinegar or concentrated baking soda, no feeding, no repot, no sudden sun). Sixteen sources each, two "Computer" agent tasks offered.

### Copilot

Ran two and a half hours after the others because attaching the first image opened a one-time consent dialog. Photo A: mealybugs "with reasonable confidence", Clivia miniata "or a closely related strap-leaved ornamental - I'm less certain of the plant ID than of the pest"; today / tomorrow / 5–7 days plan including the pot rim and the crown; "possibly" on the watering; calls the infestation light and localised. Photo B: wax begonia; powdery mildew, "fungal rather than mealybugs"; prune leaves over 50% covered, debris, airflow, sulphur or potassium bicarbonate, water the soil only, finger test, judge by new growth; ends "90% likely this is a bronze-leaved wax begonia with powdery mildew". Both right, and it tells you how sure it is.

### DeepSeek

Answered in English both times, after a web search. Photo A: "a Yucca (or possibly a Hippeastrum)" - wrong, though Hippeastrum is at least the same family; mealybugs right, in a table; then "apply a systemic insecticide, such as Acaricide (or similar products containing imidacloprid), both as a spray and a drench" - an acaricide is a mite killer, and imidacloprid products are not sold to amateur gardeners in the EU; neem offered as the alternative. Ends "if you can confirm the exact plant type". Photo B: Begonia, "most likely a rex begonia" (it is a wax begonia; the genus counts); powdery mildew; remove leaves, dry-top watering, airflow, garden sulphur or neem or soda. The protocol expected DeepSeek's image input to read text only; it read both photos as images.

### Grok

Private chat, six searches per photo. Photo A: Clivia miniata with a differential against Agapanthus; mealybugs, "most likely citrus mealybug *Planococcus citri* or longtailed mealybug" - the exact species in the ground truth; six steps including the crown, the pot rim and the drainage holes, don't compost the cuttings, systemic only if it comes back. Photo B: wax begonia, bronze-leaved; "powdery mildew (*Erysiphe begoniicola* / *Oidium begoniae*)" - the ground-truth pathogen; notes the fungus does not need wet leaves, stagnant air is the driver; isolate and prune to the bin, a small fan, soil-level morning watering, baking soda or milk or neem every 5–7 days for 3–4 weeks, wipe the pot and tray. The only tool to name both organisms, and nothing wrong in either plan.

### Qwen

Photo A came back as two answers side by side with "Which answer do you prefer? Choose one to continue" - an A/B rating prompt; both say Clivia miniata and mealybugs, both call the watering too frequent, both give a dated plan; the left one is scored and both are in the raw. Photo B: Begonia, "likely a Rex or flowering hybrid"; powdery mildew; mist the leaves before cutting them, drainage, spacing, neem or soda or sulphur, water when the top inch is dry, repeat every 5–7 days. Attached by the paste route at 1280 px.

### Kimi

Photo A: "an agave (most likely Agave americana)" - its visible reasoning tries Dracaena, Yucca, Phormium and Pandanus and never considers Clivia. Mealybugs right. Because it believes the plant is an agave it says the north terrace is "too shaded", tells you to move the pot to the brightest spot or "consider whether this is the right plant for that spot", and to repot into gritty cactus mix - a Clivia in full Lisbon sun scorches. The treatment steps are sound and it adds ants and a scale-insect caveat. Photo B: wax begonia and *Oidium begoniae* (spelled "Odium"), right; a good plan (mist before cutting, neem or sulphur with the Portuguese words, soda, repeat in 7–10 days, morning root watering) - delivered in Russian. Its reasoning shows it noticing the user wrote English, weighing a "Language: ru" session setting, and choosing Russian. Attached by the drop route at 1280 px after a first attempt hung.