whichforwhat

Evidence: Presentations

What we asked for, and what we handed over. Every answer the tools gave is kept exactly as it came back — listed below, one file each, unedited.

Brief

Input we gave itRaw file

Presentations — the brief, the fixture and the scoring

Sent as one message, free tier, fresh chat (Claude: incognito), memo pasted inline under the brief so every tool can read it without a file upload. If a tool asks a question instead of building, the answer is “Use the memo as is” once; a second question ends the run.

Make me an 8-slide deck for my two-person team about our shop’s Q2 results, using only the facts in the memo below - do not add any. Slides: 1 title, 2 revenue by month, 3 revenue by category, 4 top products, 5 orders by region, 6 the two data problems we found, 7 three actions for Q3, 8 closing. Short speaker notes on every slide. Give me the deck as a .pptx file, or a link I can open in PowerPoint or Google Slides.

[the memo, verbatim from q2-memo.md]

The fixture

q2-memo.md: seven bullets about a fictional Lisbon online shop’s second quarter, built from the Excel test’s cleaned data (../best-ai-for-excel/fixtures/). It carries 25 checkable facts: 120 orders, 20 customers, €28.30 average; three monthly figures and the quarter total; three category figures; three products with unit counts; five regional order counts and two regional revenues; two data problems with their counts; three Q3 actions. Anything on a slide that is not in the memo is invented.

Scoring

  • Deck delivered - a .pptx that opens, or a link that opens in PowerPoint or Google Slides. A deck described in text, an outline, or a “here is the structure” reply is not a deck.
  • Slides - count against 8, and whether the eight subjects appear in the order asked.
  • Facts right of 25 - every number and name on the slides checked against the memo, read from the file with python-pptx.
  • Invented - facts, numbers, names or claims on the slides that the memo does not contain.
  • Notes - slides carrying speaker notes, of 8.
  • Readability - the deck exported to PDF through Keynote and looked at: text cut off or overflowing, unreadable contrast, empty placeholders, broken charts. Noted, not scored as a number.
  • Time to answer, questions asked before building, refusals, output form (file, link, or text), language.

Q2 memo

Input we gave itRaw file

Q2 2026 memo - Quinta da Ribeira online shop (fictional)

Facts for the team deck. Every number here is checkable; nothing outside this memo is true.

  • Orders: 120 orders in April–June 2026, from 20 customers. Average order €28.30.
  • Revenue: April €1,538.60, May €847.40, June €1,009.50. Quarter total €3,395.50.
  • By category: Home €1,399.00, Pantry €1,109.70, Drinks €886.80.
  • Top products by units: Vinho Verde 750ml 42, Ceramic Bowl 40, Ginjinha 200ml 38.
  • By region, orders: Braga 31, Lisboa 29, Coimbra 24, Algarve 21, Porto 13. Lisboa is first by revenue (€947.90), Braga second (€790.50).
  • Two problems found in the order export: 4 duplicate orders, and 5 orders whose total did not match quantity × price. Both fixed.
  • Three actions for Q3: (1) one date format in the export; (2) a summer bundle of Vinho Verde and Ginjinha; (3) reorder Ceramic Bowls before August.

README

Input we gave itRaw file

Fixtures for the Presentations test

  • q2-memo.md - the memo pasted under the brief: seven bullets of Q2 facts for a fictional Lisbon online shop, built from the Excel test’s cleaned data. 25 checkable facts; nothing outside it is true.
  • brief.md - the message sent, the fixture description and the scoring rules.
  • check_deck.py - reads a returned .pptx with python-pptx (Homebrew Python 3.13): slide count, titles, chart data, notes per slide, and the 25 facts found or missing. --dump prints every slide’s text and notes.
  • check_layout.py - structural readability: slide size, shapes, charts and pictures per slide, font sizes, empty placeholders, boxes likely to overflow.
  • returned/ - the decks the tools sent back, named by tool: claude, grok, copilot, perplexity, kimi (.pptx as delivered), gemini (the canvas’s own Export → PPTX) and gamma (the editor’s Export → PowerPoint). Canva sent nothing back. returned/pdf/claude.pptx.png is a QuickLook thumbnail of Claude’s first slide.

A visual pass through Keynote was planned and failed: Keynote hung on AppleScript open/export twice on this machine, and QuickLook thumbnails timed out for all but one file. Readability is therefore judged from the in-app previews (Claude, Gemini) and the structural check, and said so in the sheet.

Nothing to clean up on any account: no uploads, no memory instructions, no automations. Gemini’s canvas stays in its chat history.

What came back

Unedited, one file per tool

How we marked it

The tools never saw these

Also kept