Why 85.6% of Bali Venues Never Appeared in an AI Recommendation Audit

When a restaurant doesn't show up in an AI response, there are a few potential culprits. The prompt might not match what the business offers. The model's retrieval step might favor competitors. The assistant might use an alternate name for the venue. Or the response might simply cap out at a handful of suggestions.

To figure out why a business is missing, looking at the final output isn't enough. You need a clear baseline of every business that could have been recommended, alongside the exact prompts used to test the system.

A recent study, Invisible to the Machine, applies this exact framework to two markets in Bali: Canggu and Ubud. The researchers built a baseline list of 4,776 food-and-drink venues using Google Places data within strict geographic lines. They then measured that baseline against responses generated via search-enabled APIs across four different AI systems.

The headline finding: 85.6% of those 4,776 venues were never mentioned in the sampled outputs. While that shows a massive coverage gap within the test parameters, it requires context. It doesn't mean those businesses are permanently locked out of every possible prompt, language, or consumer-facing app.

The baseline changes the question

Most casual AI audits start with a short list of prominent local competitors. If the chatbot names most of them, it is easy to assume the model has a solid grasp of the local market.

Starting with a comprehensive market list flips the premise. Instead of asking whether top brands show up, you can ask how much of the actual market makes it into the sample at all.

For the main phase of the study, the authors ran 96 persona-based prompts, gathering 2,208 search-grounded responses. Because the prompts were written in English and tailored to tourists and digital nomads, the findings reflect that specific slice of usage, those specific API setups, and that specific window in time. They shouldn't be read as a universal map of what every local user sees in a consumer chat interface.

The baseline list has its own limits, too. Because it relies on Google Places data and specific map boundaries, any business missing from that initial dataset isn't part of the equation. Calling the list a census just clarifies how the denominator was built—it doesn't mean the list captures every conceivable way to define the local dining scene.

Showing up is different from ranking well

The study draws a hard line between two distinct metrics: whether a venue appears in the sample at all, and where it ranks once it gets there. For anyone auditing AI visibility, keeping these separate is critical.

If a business never shows up in your test runs, it doesn't have an average rank. Plugging in an arbitrary placeholder number just obscures the real issue: complete absence. Conversely, a business that shows up consistently at the bottom of the list has a very different problem than one that never gets named.

When looking at what separates included venues from ignored ones, the researchers found links to basic digital footprint signals: having a website, listing prices, accumulating reviews, and earning third-party mentions. Interestingly, star ratings played a different role depending on whether the test measured initial inclusion or final ranking.

Still, correlation here isn't causation. Adding a website or a price tier won't automatically trigger AI recommendations. Well-documented businesses often share other advantages—they might be older, larger, or simply more famous offline. Even a star rating is just a platform metric, not an objective measure of food quality.

These correlations give auditors a solid list of variables to check, but they don't form a guaranteed optimization checklist, nor do they prove that AI models use those fields as strict pass/fail filters.

Nail down entity matching before scoring visibility

In any local audit, messy naming conventions will trip you up. An AI model might shorten a restaurant's name, use an outdated brand, or mention a multi-location chain without specifying the neighborhood. If you rely on strict text matching, you may mark real recommendations as misses or give credit to the wrong storefront.

A reliable audit keeps the raw output and cross-references fuzzy names against street addresses or other unique identifiers, logging edge cases separately. A clean-looking visibility score is useless if the underlying entity resolution is flawed.

Data freshness is another major hurdle. During the study, the models recommended 14 permanently closed venues a total of 93 times. Recommending a dead business is a completely different failure mode than ignoring an obscure open one. Good auditing separates stale data, ambiguous name matches, and true omissions into distinct buckets.

Outputs also shifted from run to run. The variance shown in the study's retests is a strong reminder to log multiple generations rather than treating a single screenshot as permanent truth. And because different AI products behave differently, any useful report needs to specify the exact model and interface being tested.

Demand the methodology behind the metric

Before making decisions based on a local AI visibility score, ask to see the underlying setup: the baseline business list, the map boundaries, the exact prompts, the language, the model interface, and the multi-run logs. Then, find out how entity names were reconciled and what counted as a valid mention.

With that context in hand, you can tell the difference between a business that was genuinely skipped, one that was misidentified, and one that made the cut but lost out on ranking. Without that context, calling a business "invisible to AI" is just a slogan backed by a number you can't verify.