AI visibility in Search Console: proving a change worked

“What about visibility in AI, how do we measure it?” Every marketing team has been asking for a year. Google now answers part of it: Search Console shows your pages’ impressions in generated answers. A company specialising in SEO testing has even announced that it detects, for the first time with statistical confidence, the effect of a site change on that visibility. The news matters less for the number than for the method.

What Search Console really shows

According to the first write-ups published by specialists (Google still documents little on the subject), the report has two views: search, which groups AI Overviews and AI Mode, and Discover. It shows impressions by page, country, date and device. Data starts on May 18, 2026, with no earlier history, and it is rolling out in waves: your property may not have it yet.

What is missing matters as much. No clicks, no click-through rate, no position, and above all no queries: you know a page was shown in a generated answer, not for which question. Nothing comes out through the API or the BigQuery export either. Teams using these figures today collect them by hand, page by page, which the author of the post calls “a very painstaking and manual process”.

Why a rising curve proves nothing

Take a page you rewrote to answer questions better. Three weeks later, its AI Overviews impressions climb. A win? Not necessarily. Google extends its generated answers to new queries and new countries almost every month. The curve might have climbed without you.

That is the trap of most AI visibility trackers: they show a trend, never a cause. The chart shared in the post makes it plain: the curve of the changed pages rises both before and after the change. By eye, you cannot tell whether the change helped.

The controlled test: the only proof that holds

The method comes from clinical trials. Take a set of comparable pages, such as product pages or service pages built on the same template, and split them into two groups. The first gets the change, the second stays as it is. The control group goes through the same waves as the rest of the market: Google updates, seasonality, AI Overviews expansion. From it, you forecast what the changed group would have done without the change. The gap between that forecast and reality is your effect.

The result comes with a margin of uncertainty, a credible interval: “between +4% and +11% impressions, with 95% confidence” says far more than “it went up”. And if the interval crosses zero, you conclude nothing, which keeps a useless change from being rolled out across the whole site.

What it takes to do it yourself

• Pages in numbers. A test runs on dozens of pages from the same template, not three articles. Template-driven sites (catalogues, local pages, listings) suit it best.
• Regular collection. With no API, you collect the AI impressions of every page in both groups, at the same pace, before and after.
• One change at a time. Rewriting the intro, adding a direct answer at the top, structuring data: each hypothesis gets its own test.
• Patience. Allow several weeks for the gap to rise above the noise.

What it changes for a team: you stop arguing about what “works for AI” and decide on evidence, page by page. Changes that bring nothing no longer cost the time to roll them out everywhere.

What it does not solve

Search Console only covers Google. ChatGPT, Perplexity and other assistants stay outside this report, and measuring them still takes the protocol in our article on measuring visibility in AI. And an impression is not a visit: being shown in a generated answer can also mean the user no longer needs to click, as we saw with AI Overviews and traffic.

The progress is real all the same: for the first time, visibility in Google’s AI can be measured with the same statistical tools as the rest of SEO. Teams that test before rolling out will get ahead of those that follow curves.