Landing on a page about the right topic is no guarantee that you will actually get the data. A search result might summarize a study without sharing the raw numbers, drop you on a broad portal homepage, or lock the figures inside an interactive chart with no export button.
When an AI agent is tasked with retrieving a functional dataset, matching the topic is just the entry fee. The agent has to land on an actual data record with a file it can extract.
A recent study, Do Data Agents Need Semantic Metadata?, tested this exact hurdle across 58 queries. The researchers ran two agents powered by the same Gemini 2.5 Pro model: one routed through standard Google Search, and the other through Google Dataset Search, which relies on structured dataset markup.
The specialized index delivered a much higher rate of usable datasets among the results it returned, though it failed to answer nearly a third of the questions. Because the test compares two entirely different search products end-to-end, it reflects how the whole pipeline behaves rather than isolating the exact impact of adding schema markup to a webpage.
Judging Results by What They Let You Do
To count as a success under the study's FAIR-aligned grading, a returned page had to clear three bars: it needed to match the topic, live on an appropriate page type, and provide machine-readable data. (Note that this is a practical grading rubric for the test, not a full audit of every FAIR data principle.)
By that yardstick, the Dataset Search agent hit 46.4% precision, compared to 28.0% for the standard Google Search agent. When the specialized agent returned an answer, it was much more likely to be something the system could actually use.
Coverage told the opposite story. The standard search agent answered 56 of the 58 queries, while the Dataset Search agent answered only 40. Looking at precision alone hides how often the specialized tool came up empty.
This is a classic retrieval trade-off. Restricting an index can cut down on dead ends and boost precision, but it can also leave fewer answers available. Which failure mode you prefer depends entirely on your workflow. An automated data pipeline might prefer a clean "no result" over a link to a narrative blog post, whereas a human researcher might appreciate the blog post as a stepping stone.
Better Destinations, Not Better Topic Matching
Interestingly, both agents were roughly equal at finding pages on the right topic, hitting high relevance around 60% of the time. The real separation happened in what kind of pages they surfaced and whether the data could actually be pulled.
The Dataset Search agent regularly landed on dedicated registry pages—the kind of records that neatly package the description, provider, variables, dates, and download links in one place. Meanwhile, the general search agent tended to drop the user on narrative articles or general portals.
This doesn't mean standard web search is blind to page structure, or that metadata doesn't help with topical relevance; it simply reflects how these two tools performed on this specific benchmark.
Think of the difference between a local news story covering a city transit survey and the city's official catalog page for that survey. Both match the topic. Only the catalog record hands the agent a clean CSV link and the license terms. The news article is great context for a reader, yet it's a dead end for an automated retrieval task.
You Can't Separate the Markup from the Index
Even though both agents used the exact same Gemini model, Google Search and Google Dataset Search are distinct products with their own crawling rules, inclusion filters, and proprietary ranking algorithms.
Because the index and the markup change at the same time, this experiment measures end-to-end system utility. You cannot credit the precision jump solely to `schema.org` tags, nor can you blame the missed queries entirely on publishers making markup mistakes.
It is also worth noting that the researchers had to step in and grade pages by hand whenever their automated scrapers failed. That alone is a great reminder that a live URL and a 200 OK status code don't guarantee an agent can automatically parse what's on the page.
Keep the scope of this task in mind, too. Finding downloadable files in a specialized vertical says nothing about whether slapping generic structured data onto a commercial site will get a brand mentioned more often in everyday AI chat responses.
Check the Actual Path to the File
If you publish data, look at the actual landing pages your users and agents hit. Does the page clearly state what the dataset covers, when it was updated, who owns it, and how to grab the file? Don't just verify that a download button exists—click it to make sure the file actually comes through.
Make sure your visible page text and your underlying structured markup tell the same story. Passing a schema validation tool only proves your syntax is clean; it doesn't prove the file link works or that the data answers the user's question. Getting indexed in a specialized catalog only helps if the landing page actually delivers the goods.
Finally, if you are building or testing retrieval agents, always pair precision metrics with query coverage. Break down your failures by category: missing the topic, landing on an article instead of a record, hitting a broken download, and returning nothing at all are four very different problems that require different fixes. The real goal is getting usable records across a wider share of the questions you actually care about.
