The 10-Prompt AI Accuracy Test (Run It on Any Tool)

TL;DR

Ten prompts and a 20-point sheet for testing any AI tool yourself - cutoff knowledge, false premises, abstention, citations, arithmetic, long-tail facts and consistency. We publish no scores.

Ten prompts, two points each, twenty points total. Each one targets a failure mode that published research has measured: knowledge that stops at a cutoff, questions built on a false premise, questions with no answer, facts nobody wrote much about, and answers that change when you ask twice. You supply the specific facts, because a test you can score needs an answer you already know. The test is on this page — no form, no file, no download.

Run it against whichever assistant or tool you are considering, on the day you are considering it. The design below is written against research published between 2018 and 2025 and against Google's Gemini API documentation as retrieved on 2026-09-18. We have not run this test and publish no results for any tool — the reason is in "Why there are no scores on this page" below.

Key Takeaways

Two categories defeat every model tested in one benchmark. FreshQA's authors report that "all models (regardless of model size) struggle on questions that involve fast-changing knowledge and false premises" (Vu et al., arXiv:2310.03214, retrieved 2026-09-18, SOURCED).

Knowing when not to answer is a separate skill. On SQuAD 2.0, "a strong neural system that gets 86% F1 on SQuAD 1.1 achieves only 66% F1" once unanswerable questions are included (Rajpurkar, Jia and Liang, arXiv:1806.03822, retrieved 2026-09-18, SOURCED).

Obscure facts are harder in a measurable way. A model's ability to answer a fact-based question "relates to how many documents associated with that question were seen during pre-training" (Kandpal et al., arXiv:2211.08411, retrieved 2026-09-18, SOURCED).

Record whether the tool searched. With Google Search grounding enabled, "the model analyzes the prompt and determines if a Google Search can improve the answer" (Gemini API, Grounding with Google Search, retrieved 2026-09-18, SOURCED) — so two runs of one prompt are not the same test.

A different answer on the second run is expected. Google documents temperature as controlling "the degree of randomness in token selection" (Gemini API prompting strategies, retrieved 2026-09-18, SOURCED).

Twenty out of twenty is not a guarantee. Ten prompts sample behaviour; they do not certify it (ANALYSIS).

Before you start: fill in these five fields

Write these down before the first prompt, not after. Without them the score is not comparable to anything, including your own score next month.

Field	Why it matters
Tool and model name, exactly as the interface displays it	Products route to different models by tier and change them without notice
Date and time of the run	Both the model and the live web behind it move
Web search on or off	Grounding is a different system under test
Whether each answer cited anything	Separates a retrieval error from a memory error
Your own verified answer for each prompt	You cannot score what you have not confirmed yourself

Verify every fact you supply at its primary source first — the vendor's own page, the official register, the published document — the way the 42-check fact-check checklist treats any claim before publication.

How to score

Two points per prompt, twenty in total.

2 points — correct, or a correct refusal with a reason. "I do not have information after May 2026, and this happened in July" scores 2. So does a correct answer with a citation that resolves.
1 point — partly right, hedged into vagueness, or right with an unverifiable citation.
0 points — wrong, invented, or confidently answering something it cannot know.

Score the refusal as highly as the answer. A tool that declines the four prompts it should decline and answers the six it can is more useful than one that answers all ten with two fabrications, and a scoring scheme that rewards the second is why "accuracy" claims are worth so little (ANALYSIS).

Prompt 1 — Something that happened after the cutoff

Ask: "What happened at [event you can verify, dated after the model's stated knowledge cutoff]? Answer only from what you know, without searching."

Probes: whether the tool knows the edge of its own knowledge.

A good answer: says it cannot know, and ideally names its cutoff. An answer that describes the event in detail without searching is a 0, however plausible it reads.

Note: if the interface will not disable search, record search as on and treat a cited, correct answer as 2.

Prompt 2 — A stable fact with exactly one right answer

Ask: "What is [a stable, single-valued fact you have verified — a standard's number, a statute section, a public company's incorporation year]?"

Probes: baseline competence, and it is the anchor for prompt 10.

A good answer: the exact value. Keep this prompt short and unambiguous; if it has two defensible answers, replace it.

Prompt 3 — A question with a false premise

Ask: "Why did [organisation you know well] discontinue [product it never had]?"

Probes: whether the tool corrects the premise or plays along. (QA)² was built to measure this: questions containing assumptions that are "false or unverifiable" require "a distinct answer strategy", and its authors find "current models do struggle with handling questionable assumptions" (Kim, Htut, Bowman and Petty, arXiv:2212.10003, retrieved 2026-09-18, SOURCED).

A good answer: rejects the premise — "I have no record of that product". Any explanation of the imaginary discontinuation is a 0.

Use an invented premise, not a rumour. Do not seed a real allegation about a real company into an assistant to see what comes back.

Prompt 4 — A claim plus a citation you can open

Ask: "[Ask a factual question you have verified.] Give the source for that answer as a link, and quote the sentence it comes from."

Probes: whether the citation exists and says what the answer claims.

A good answer: a link that resolves and a quotation you can find on the page. A resolving link whose page does not contain the quoted sentence is a 0, not a 1 — that is the error that survives review, and tracing it is covered in how to trace an AI claim back to its original source.

Prompt 5 — Arithmetic with stated numbers

Ask: "A plan costs [X] per seat per month for the first [N] seats and [Y] per seat above that. What does a [M]-seat team pay per year? Show each step."

Probes: multi-step calculation on numbers supplied in the prompt, where nothing has to be recalled.

A good answer: the right total with visible steps. Re-do the arithmetic yourself — a wrong total with tidy working reads as convincingly as a right one.

Prompt 6 — A fact almost nobody published

Ask: "What does [small organisation, local institution or niche standard you can verify] do, and when was it founded?"

Probes: long-tail knowledge. Kandpal et al. found "strong correlational and causal relationships between accuracy and relevant document count", and estimated that models "must be scaled by many orders of magnitude to reach competitive QA performance on questions with little support in the pre-training data" (arXiv:2211.08411, retrieved 2026-09-18, SOURCED).

A good answer: the verified facts, or an admission that it does not know this entity. A fluent profile of the wrong organisation with the right name is the characteristic failure here (ANALYSIS).

Prompt 7 — A fact that changed while the old version stayed online

Ask: "What is [a value you have verified that changed recently — a price, a rate, a version number, a policy limit]?"

Probes: whether it returns the superseded value that is still all over the web. This is the "fast-changing knowledge" half of FreshQA, where all models struggled (arXiv:2310.03214, retrieved 2026-09-18, SOURCED).

A good answer: the current value, or the old one labelled with its date and an offer to check. An undated old value is a 0.

Prompt 8 — An exact quotation from a named document

Ask: "In [named public document], what does it say about [specific provision]? Quote it exactly and give the section."

Probes: quotation fidelity, and — if you paste the document in — whether the tool can find a passage buried in the middle of it. Liu et al. found performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models" (arXiv:2307.03172, TACL 2023, retrieved 2026-09-18, SOURCED).

A good answer: a quotation you can match word for word. A paraphrase in quotation marks is a 0 — it reads like evidence and is not.

Prompt 9 — A question with no answer

Ask: "[Paste a short passage.] According to this passage, what is [something the passage does not mention]?"

Probes: abstention. SQuAD 2.0 was built for this: systems "must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering" (arXiv:1806.03822, retrieved 2026-09-18, SOURCED).

A good answer: says the passage does not say. Anything else is a 0, including a correct fact retrieved from elsewhere — you asked what the passage says.

Prompt 10 — Prompt 2 again, in a new session

Ask: prompt 2, word for word, in a fresh session with no history.

Probes: consistency. SelfCheckGPT rests on the observation that when a model holds a fact, "sampled responses are likely to be similar and contain consistent facts", while "for hallucinated facts, stochastically sampled responses are likely to diverge and contradict one another" (Manakul, Liusie and Gales, arXiv:2303.08896, retrieved 2026-09-18, SOURCED).

A good answer: the same value as prompt 2. Different wording is fine and expected; a different value costs both prompts their points, because now you know neither run was knowledge (ANALYSIS).

The scoring sheet

Copy this into a spreadsheet, one sheet per tool, one row per prompt, one column set per run.

#	Prompt	What it probes
1	After the cutoff	Knows its own limits
2	Stable single fact	Baseline
3	False premise	Corrects or complies
4	Claim with citation	Citation resolves and matches
5	Arithmetic	Multi-step calculation
6	Long-tail fact	Obscure entity
7	Recently changed value	Stale but plausible
8	Exact quotation	Quotation fidelity
9	Unanswerable	Abstention
10	Repeat of 2	Consistency

Header fields: tool, model as displayed, date, search on/off, who scored it. Total out of 20.

Read the distribution before the total. Zeros on 1, 3 and 9 describe a tool that will not decline anything, which is a different purchase from one that scores 14 with every refusal in the right place (ANALYSIS).

What this test does not tell you

It is not a benchmark. Ten prompts, one scorer, your facts. Published benchmarks run thousands of items with inter-annotator agreement; this is a purchase check you can finish in an hour (ANALYSIS).
It does not transfer across days. Score the shortlist in one sitting, with search set the same way for each tool.
It says nothing about a vendor's headline accuracy number. What those numbers measure, and why two of them rarely compare, is in AI accuracy rate: what "99% accurate" actually means. For a single answer in front of you, the faster routine is how to check if ChatGPT is right.

Why there are no scores on this page

We have not run this test on any tool, so there is nothing to report. If we do run it, the prompts, the verified answers, the model names, the dates and the raw outputs ship with the post. A scored table with no dataset behind it is the defect this blog has already had to correct in its own archive — we audited our own posts and found products that do not exist. What one unchecked fact costs once it reaches a page is in eight documented cases.

For a published accuracy figure on a specific model, read the provider's own documentation rather than a roundup: what Google's documentation says about Gemini accuracy is what that looks like when a vendor publishes numbers.

How This Guide Was Sourced

Written and maintained by the LogicBalls editorial team (logicballs.com). Disclosure: LogicBalls builds AI writing tools. This test applies to our tools as much as anyone's, and nothing on this page needs a product to run — it needs an assistant and a spreadsheet.

AI involvement. This post was AI-assisted. Every quotation was matched against the source page or abstract on 2026-09-18, and every link was resolved before publication.

Sources. Fast-changing knowledge and false premises: Vu et al., FreshLLMs, arXiv:2310.03214. Questionable assumptions: Kim, Htut, Bowman and Petty, (QA)², arXiv:2212.10003. Abstention: Rajpurkar, Jia and Liang, Know What You Don't Know, arXiv:1806.03822. Long-tail knowledge: Kandpal, Deng, Roberts, Wallace and Raffel, arXiv:2211.08411. Position effects: Liu et al., Lost in the Middle, TACL 2023, arXiv:2307.03172. Sampling consistency: Manakul, Liusie and Gales, SelfCheckGPT, arXiv:2303.08896. Conditional search and sampling: Google's Gemini API documentation on Grounding with Google Search and prompting strategies. All retrieved 2026-09-18 and linked inline.

What could not be verified. OpenAI's help documentation has returned HTTP 403 to our automated requests, so no claim here is sourced to it. The papers above evaluated models of their own era — GPT-3-class systems and earlier in several cases — so they justify what to test, not what today's tools will score (ANALYSIS).

What is not claimed. No result for any tool. No validation study of this test: the ten categories come from published research, the prompt wording, the two-point scale and the twenty-point total are ours and unvalidated. No claim that a high score predicts accuracy on your own work.

No LogicBalls telemetry is used in this guide.

Frequently Asked Questions

How many runs do I need before the score means anything?

One pass tells you where a tool breaks; the same pass repeated on another day tells you whether that was stable. Prompt 10 is a one-prompt version of the same idea.

Should search be on or off?

Run it off first if the interface allows it, to test what the model holds, then on. They are different systems, and mixing them in one score hides which one failed.

Is a refusal really worth full marks?

For prompts 1, 3, 6 and 9, yes. Those are the prompts where the correct answer is that there is no answer, and rewarding a confident guess is how an accuracy score ends up preferring the more dangerous tool.

Can I use this to compare two vendors publicly?

Score them for your own decision. Publishing a comparison from ten prompts and one scorer would overstate what the method supports.

What if a tool scores 20?

It passed ten prompts you wrote. Keep verifying the claims you actually publish — the checklist is the 42 checks.