Measuring AI visibility: why “rank in ChatGPT” is misleading, and what to track instead
There’s no stable rank in ChatGPT: in a SparkToro study, the same prompt returned the same brand list under 1% of the time. Track how often you’re named instead. Run a fixed set of buyer prompts several times per assistant, report “named in X of N” with a confidence interval, plus share of voice and the sources cited.
What this page is. AEOLedger sells answer engine optimization services, so we have an interest here too. Every factual claim below links to its source, each source’s commercial interest is noted, and where good evidence points the other way, it sits next to the claim.
Why rank doesn’t exist in AI answers
The consistency evidence
SparkToro and Gumshoe ran 12 prompts 60–100 times each across ChatGPT, Claude and Google’s AI Overviews and AI Mode: 2,961 runs in all. ChatGPT and Google returned the same list of brands under 1% of the time, and the same list in the same order under 0.1% of the time. Presence was steadier than position: in one category the leading brands appeared in 55–77% of 994 responses.[1] Rand Fishkin’s verdict on AI “ranking position” tools: “full of baloney”.
Even fixed settings don’t fix the answer
The July 2026 GEO survey reports that 9–28% of decisions changed between runs even at temperature 0, and that in one configuration 57.8% of ChatGPT repetitions didn’t run a web search at all.[2] The same prompt can be answered from search one time and from model memory the next.
Contrary evidence: in narrow categories, leaders appear almost every time.[1] For a dominant brand, a named rate near 100% is the finding, and order may matter more. The study also used consumer categories, not B2B SaaS, and was co-run by a tracking vendor.
What to track instead
Named rate, with N
named rate = runs in which the brand is named ÷ total runs
Always show N next to the rate. “Named in 4 of 9 answers” and “named in 40 of 90 answers” are both 44%, and mean very different things.
Share of voice, with the denominator stated
share of voice = brand’s named count ÷ named count of all tracked brands
The number depends entirely on which competitors you track. Name them every time you report it.
Cited sources
Log which domains each assistant cites for your prompts: your own pages, review sites, forums, media. That’s your map of where to be represented. It undercounts influence, though: Peec found ChatGPT’s Deep Research reads far more pages than it cites.[3]
Accuracy
Being named with last year’s pricing or a discontinued feature is a problem in itself. Note what each answer says about you, not just whether you appear.
Splits that matter
Report by assistant and by prompt type (shortlist, alternatives, head-to-head, problem-shaped). A brand can be strong in Perplexity and missing in ChatGPT, or named in shortlists and absent from “alternatives to [leader]” questions.
Business signals
In GA4, create a channel group for referrers such as chatgpt.com, perplexity.ai, claude.ai, gemini.google.com and copilot.microsoft.com. Treat it as a floor: some assistant visits arrive with no referrer and look like direct traffic.[4] Add “how did you hear about us?” to demo forms, and watch branded search.
Sample size and uncertainty
A named rate from a small sample is noisy. A 95% Wilson interval shows the range the true rate plausibly falls in:
| Result | Named rate | 95% interval | What it’s good for |
|---|---|---|---|
| 4 of 9 | 44% | 19–73% | One prompt, 3 assistants, 3 runs. Spotting absence, not tracking change. |
| 18 of 75 | 24% | 16–35% | One assistant across 25 prompts. |
| 40 of 300 | 13% | 10–18% | A full audit: 25 prompts, 4 assistants, 3 runs. Large enough to compare periods. |
How many runs per prompt?
We use 3 per assistant, in fresh sessions. More runs narrow the interval for that prompt but cost time; the interval matters more than the run count, so report both. Nobody has published the right number for B2B SaaS prompts yet; it’s an open question.
How many prompts?
Our audits use 25: enough to cover shortlist, alternatives, head-to-head and problem-shaped questions with a few of each. More prompts widen coverage; more runs tighten each estimate.
When an improvement is real
Only claim an improvement when the re-measured interval clears the earlier one. If the intervals overlap, the change may be noise.
Worked example
Fictional company, illustrative numbers A SaaS company measures 25 prompts × 3 runs in each assistant, before and after a round of fixes: 75 runs per assistant per period.
| Assistant | Before | After | Verdict |
|---|---|---|---|
| ChatGPT | 6 of 75 (8%, 4–16%) | 9 of 75 (12%, 6–21%) | Not yet shown: the intervals overlap, so this could be noise |
| Perplexity | 10 of 75 (13%, 7–23%) | 27 of 75 (36%, 26–47%) | Real improvement: the new interval clears the old one |
ChatGPT went from 8% to 12%, which sounds like a 50% improvement, but the intervals overlap heavily, so it isn’t evidence of anything yet. Perplexity’s rise from 13% to 36% clears the earlier interval: that one is real.
Common measurement mistakes
- One screenshot as proof. Given the variation above, one answer shows nothing either way.
- A composite “visibility score” that hides the sample size and the split by assistant.
- The wrong denominator. Some dashboards compute citation share only over answers that contain citations, which inflates it.[2]
- API vs app mismatch. API answers can differ from the consumer apps in model, settings and search behaviour.
- Logged-in personalisation. History and memory change answers. Use fresh, logged-out sessions where possible.
- Ignoring location. Record the country each run comes from.
- Changing the prompts mid-study. Freeze the set; start a new study for new prompts.
Questions to ask any AI visibility tool or agency
- How many runs per prompt, per assistant?
- Do you show N next to every rate?
- Do you report confidence intervals?
- Do you query the consumer apps or the APIs, and which models?
- What is the denominator for share of voice and citation share?
- Is the prompt set frozen between measurements?
Do a free version yourself
- Write 10–25 questions your buyers ask, mixing shortlist, alternatives, head-to-head and problem-shaped prompts. Freeze the list.
- In a spreadsheet, add columns for date, assistant, model label, prompt, run number, brands named in order, cited URLs and notes.
- Run each prompt 3 times per assistant in fresh, logged-out sessions, a few minutes apart. Paste the prompt exactly; no follow-ups.
- Count named rate and share of voice per assistant, and compute a Wilson interval for each rate.
- If you’re never named, check that assistants can read your site first, with our AI crawler checker.
- Re-run the identical set after changes and compare intervals.
Or request a free snapshot, and we’ll run a small version by hand. The full protocol is on our methodology page.
Questions
Can I rank #1 in ChatGPT?
There is no #1 to hold. Repeated runs of the same prompt almost never return the same list in the same order. What you can raise is how often you’re named.
How many times should I run each prompt?
At least 3 per assistant, in fresh sessions. More runs narrow the uncertainty; the interval matters more than the run count, so always report both.
What is AI share of voice?
Your named count divided by the total named count of all tracked brands over the same runs. Always state which brands are tracked, because the number depends on it.
Are AI visibility tools accurate?
They’re consistent, not necessarily representative. Check whether they query the APIs or the consumer apps, how many runs per prompt they use, and whether they show sample sizes.
How do I see ChatGPT traffic in GA4?
Create a channel group for referrers such as chatgpt.com, perplexity.ai, claude.ai and gemini.google.com. Treat it as a floor: some assistant visits arrive without a referrer.
Why does ChatGPT give different answers each time?
Answers are sampled, search may or may not run, and models and indexes change. That’s normal, and it’s why single checks mislead.
Should I measure through the API or the app?
Measure the app your buyers use. API answers can differ in model, settings and search behaviour; record which you used.
Sources
All links checked on 17–18 October 2026. Vendor interests are noted for each source.
- Search Engine Journal, coverage of SparkToro and Gumshoe’s AI recommendation consistency study, 30 January 2026. Coverage of SparkToro and Gumshoe’s study. Gumshoe sells AI visibility tracking. Not peer reviewed; consumer categories, not B2B SaaS. Original: sparktoro.com.
- Martinez, critical survey of 45 generative engine optimization studies (arXiv 2607.14035), submitted 15 July 2026. Single-author preprint reviewing 45 studies; not peer reviewed. Some figures quoted here come from PPC Land’s summary (ppc.land), not the paper itself.
- Peec AI research, pages read vs cited in ChatGPT Deep Research, June 2026. Vendor. No sample size published.
- SE Ranking, AI traffic research study, 18 June 2026. Vendor. 101,574 sites in Google Analytics, January–April 2026; some assistant visits arrive without a referrer.
How to cite and changes
AEOLedger. “Measuring AI visibility: why ‘rank in ChatGPT’ is misleading, and what to track instead.” Guide v1.0, 18 October 2026. https://aeoledger.com/guides/measuring-ai-visibility/
- v1.0, 18 October 2026: first published.
Spotted an error or a better source? Write to hello@aeoledger.com. Corrections are dated here.