Evidence: Presentations
What we asked for, and what we handed over. Every answer the tools gave is kept exactly as it came back — listed below, one file each, unedited.
Brief
Presentations — the brief, the fixture and the scoring
Sent as one message, free tier, fresh chat (Claude: incognito), memo pasted inline under the brief so every tool can read it without a file upload. If a tool asks a question instead of building, the answer is “Use the memo as is” once; a second question ends the run.
Make me an 8-slide deck for my two-person team about our shop’s Q2 results, using only the facts in the memo below - do not add any. Slides: 1 title, 2 revenue by month, 3 revenue by category, 4 top products, 5 orders by region, 6 the two data problems we found, 7 three actions for Q3, 8 closing. Short speaker notes on every slide. Give me the deck as a .pptx file, or a link I can open in PowerPoint or Google Slides.
[the memo, verbatim from
q2-memo.md]
The fixture
q2-memo.md: seven bullets about a fictional Lisbon online shop’s second quarter, built from the Excel test’s cleaned data (../best-ai-for-excel/fixtures/). It carries 25 checkable facts: 120 orders, 20 customers, €28.30 average; three monthly figures and the quarter total; three category figures; three products with unit counts; five regional order counts and two regional revenues; two data problems with their counts; three Q3 actions. Anything on a slide that is not in the memo is invented.
Scoring
- Deck delivered - a .pptx that opens, or a link that opens in PowerPoint or Google Slides. A deck described in text, an outline, or a “here is the structure” reply is not a deck.
- Slides - count against 8, and whether the eight subjects appear in the order asked.
- Facts right of 25 - every number and name on the slides checked against the memo, read from the file with python-pptx.
- Invented - facts, numbers, names or claims on the slides that the memo does not contain.
- Notes - slides carrying speaker notes, of 8.
- Readability - the deck exported to PDF through Keynote and looked at: text cut off or overflowing, unreadable contrast, empty placeholders, broken charts. Noted, not scored as a number.
- Time to answer, questions asked before building, refusals, output form (file, link, or text), language.
Q2 memo
Q2 2026 memo - Quinta da Ribeira online shop (fictional)
Facts for the team deck. Every number here is checkable; nothing outside this memo is true.
- Orders: 120 orders in April–June 2026, from 20 customers. Average order €28.30.
- Revenue: April €1,538.60, May €847.40, June €1,009.50. Quarter total €3,395.50.
- By category: Home €1,399.00, Pantry €1,109.70, Drinks €886.80.
- Top products by units: Vinho Verde 750ml 42, Ceramic Bowl 40, Ginjinha 200ml 38.
- By region, orders: Braga 31, Lisboa 29, Coimbra 24, Algarve 21, Porto 13. Lisboa is first by revenue (€947.90), Braga second (€790.50).
- Two problems found in the order export: 4 duplicate orders, and 5 orders whose total did not match quantity × price. Both fixed.
- Three actions for Q3: (1) one date format in the export; (2) a summer bundle of Vinho Verde and Ginjinha; (3) reorder Ceramic Bowls before August.
README
Fixtures for the Presentations test
q2-memo.md- the memo pasted under the brief: seven bullets of Q2 facts for a fictional Lisbon online shop, built from the Excel test’s cleaned data. 25 checkable facts; nothing outside it is true.brief.md- the message sent, the fixture description and the scoring rules.check_deck.py- reads a returned .pptx with python-pptx (Homebrew Python 3.13): slide count, titles, chart data, notes per slide, and the 25 facts found or missing.--dumpprints every slide’s text and notes.check_layout.py- structural readability: slide size, shapes, charts and pictures per slide, font sizes, empty placeholders, boxes likely to overflow.returned/- the decks the tools sent back, named by tool: claude, grok, copilot, perplexity, kimi (.pptx as delivered), gemini (the canvas’s own Export → PPTX) and gamma (the editor’s Export → PowerPoint). Canva sent nothing back.returned/pdf/claude.pptx.pngis a QuickLook thumbnail of Claude’s first slide.
A visual pass through Keynote was planned and failed: Keynote hung on AppleScript open/export twice on this machine, and QuickLook thumbnails timed out for all but one file. Readability is therefore judged from the in-app previews (Claude, Gemini) and the structural check, and said so in the sheet.
Nothing to clean up on any account: no uploads, no memory instructions, no automations. Gemini’s canvas stays in its chat history.
What came back
How we marked it
- fixtures/check_deck.pypy3 KB
- fixtures/check_layout.pypy2 KB
Also kept
- The test (2026-09)The test, written up
- fixtures/returned/claude.pptxpptx221 KB
- fixtures/returned/copilot.pptxpptx173 KB
- fixtures/returned/gamma.pptxpptx2.3 MB
- fixtures/returned/gemini.pptxpptx230 KB
- fixtures/returned/grok.pptxpptx166 KB
- fixtures/returned/kimi.pptxpptx77 KB
- fixtures/returned/pdf/claude.pptx.pngpng75 KB
- fixtures/returned/perplexity.pptxpptx46 KB
- results-2026-09.yamlyaml5 KB