When you ask an AI assistant to recommend a professional—say, an estate agent—the response can take a few shapes. It might name a specific person, a firm named after its founder, or a regional office. For an analytics dashboard scanning exclusively for name-like text, those three distinct outcomes are easily conflated.
A second blind spot emerges when a tracking tool only checks AI responses against a predetermined list of names. While that approach checks whether the listed names occur, it tells you nothing about how many other people the assistant recommended instead.
A recent study, Who Gets Named, explores both of these measurement traps. Researchers analyzed 2,400 web-grounded API responses across various European markets and professional-service categories. Their main detector classified 619 responses (roughly 25.8%) as naming an individual. In a separate test, they cross-referenced the AI outputs against a preset roster of 939 professionals gathered from public LinkedIn searches.
The core value of the paper lies in the gap between those two methods. One instrument asks whether the AI names a human being at all; the other asks whether an identified name happens to exist on a specific list.
A Name-Shaped String Is Not Always a Person
Across the dataset, the study surfaced 27,293 name-like text strings, but only 128 matched the LinkedIn roster. Dividing those figures yields 0.47%—a dramatic number, though one that requires context.
That large denominator is made up of candidate strings, not verified professionals. Because geographic locations and eponymous businesses look just like personal names to a basic parser, treating every candidate string as an uncaptured professional artificially deflates the roster's apparent coverage.
A clearer picture emerges when looking at the 1,876 text spans the study's detector specifically classified as individuals. The 128 roster matches account for 6.8% of those mentions. Even here, the metric tracks total mentions (which can include repeats of the same person) rather than unique individuals. While 6.8% better reflects how often the preset roster overlaps with actual person-naming, it still doesn't measure the roster's share of the entire professional market.
Furthermore, automated detection has its own limits. Evaluated against roster-based ground truth, the study's roster-free detector achieved 96.9% precision and 61.7% recall. High precision is encouraging, but a 61.7% recall rate indicates that a substantial share of naming covered by the roster-based validation was missed. In practice, a score of zero on a dashboard shouldn't be taken as proof that no one was recommended; it is an invitation to inspect what the detector missed.
Separating Roster Coverage from Total Recommendations
Checking a preset roster makes sense when answering a narrow question, such as, "Did the AI mention any of our firm's partners?" It becomes misleading, however, when that same metric is treated as a complete picture of who the AI recommends.
If an assistant recommends three competitors who aren't on your tracking list, a roster-based dashboard simply logs zero matches. That is an accurate reflection of your partner coverage, but a poor reflection of how the AI handles personal recommendations in your sector.
To avoid conflating these metrics, service businesses auditing their AI visibility should track three distinct questions:
- Does the response name any individual at all?
- Who specifically does it appear to name?
- Does that person appear on our monitored roster?
When an entity is ambiguous, it is better to flag it as uncertain than to force a false match. Eponymous firms require special scrutiny: a mention of "Smith Insurance" is not, by itself, evidence of a recommendation for a broker named Smith. Telling them apart requires looking at the surrounding sentence, the cited URL, and the actual business identity.
Source Types Are Clues, Not Levers
The researchers also examined the citations behind the AI responses, comparing answers that named individuals against those that did not. Responses that named people were slightly more likely to cite personal websites and category portals. Raw citation volume alone did not explain the difference.
This correlation makes sense intuitively—a personal bio page gives an AI different grounding material than a general corporate directory. However, correlation is not a growth playbook. The differences in source types were modest, and behavior varied across models and service categories. The study's design does not prove that buying a portal listing or launching a personal website will trigger AI recommendations, nor does it suggest you can scrub unwanted personal mentions simply by altering a single source type.
Auditing the Data Behind the Dashboard
If you are evaluating person-level AI visibility, don't stop at top-line percentages. Ask to see a raw sample of counted matches, rejected strings, and edge cases alongside the full AI responses. Cross-check flagged names against firm names and cities. Keep a log of monitored professionals who never appear, but never confuse their absence with the idea that the AI isn't recommending individuals.
Finally, audit your roster. Does your list represent the actual professionals buyers care about in your market, or is it just a convenient scrape from a single social platform? Even the most accurate matching algorithm cannot fix a poorly defined baseline.
