Executive summary and key findings#
TL;DR
We studied what AI models remember about 1,000 public B2B SaaS brands before they use web search, whether the AI model understood what each company does, and which online signals were linked to both.
Nearly half of the brands were never named, and about one in five category choices did not match the verified product category. Google's open-weight Gemma 4 model named 510 of 1,000 brands. Of its 981 category choices, 194 (19.8%) differed from ground truth.
This study found that stronger public-web presence was linked to 12.58 points higher recall and 7.93 points higher category accuracy per standard-deviation increase. Social media prominence was linked to 9.75 points higher recall but just 0.21 points higher category accuracy.
As the AI visibility market matures, marketers are asking more precise questions about what sits behind an AI answer. People often talk about whether a brand is "in the training data," but models change, web search can add fresh information, and the answer a buyer sees does not reveal what the model already knew. This study isolates that earlier layer.
The study separates two questions that matter for AI brand visibility: whether a model recalls a brand at all, and whether it understands what the company does. We then tested which naming, website, social, review and public-web signals were associated with each outcome.
Gemma 4 was used for the main audit because its internal scoring can be inspected. That access let us measure what was stored inside the model before retrieval could add new information. We also tested the same cohort through closed AI assistants, including GPT-5.6 Luna, Gemini 3.5 Flash Lite and Claude Haiku 4.5.
Key findings on AI brand recall and accuracy#
- Nearly half the brands were missing from the main recall test. Gemma 4 named 510 of 1,000 brands across twelve unprompted category samples, leaving 490 unnamed (49.0%).
- Being remembered did not mean being understood correctly. Gemma 4 chose a category for 981 brands, matching verified ground truth for 787 and choosing a different category for 194 (19.8%). The remaining 19 brands produced no category preference.
- A stronger public-web presence was associated with both recall and category accuracy. After adjustment, a one-standard-deviation increase in prominence in Common Crawl, a large public-web dataset, was associated with 12.58 points higher recall and 7.93 points higher category accuracy.
- A larger social presence was associated with recall far more strongly than category accuracy. The adjusted association was 9.75 points higher recall and 0.21 points higher category accuracy per standard-deviation increase.
- Customer ratings had almost no relationship with measured model sentiment. Across 921 brands with G2 or Trustpilot ratings, the correlation between customer ratings and model sentiment was 0.030 (p = 0.36).
Cite this study
Discovered Labs (2026). What AI models know about 1,000 B2B SaaS brands before web search.
https://discoveredlabs.com/research/ai-model-brand-recall-study
Charts and tables on this page may be reproduced with attribution and a link to the source.
Methodology and scope#
- Cohort: exactly 1,000 public B2B SaaS brands across 333 product categories.
- Recall test: twelve unprompted category samples on Gemma 4 12B. A brand counts as recalled when it appears in at least one sample.
- Category test: a five-candidate stored-category ranking on Gemma 4 12B, checked against the study's verified ground truth.
- Association analysis: 79 brand-level variables fitted separately against recall and category accuracy, with the confounder block and family correction described in the technical paper.
- Assistant cross-check: We also ran the 1,000-brand cohort through GPT-5.6 Luna, Gemini 3.5 Flash Lite and Claude Haiku 4.5. This adds a practical view of how brand recall varies across assistants alongside the main Gemma 4 analysis.
Read the full technical paper for the model definitions, controls, confidence intervals and validation details.
The 4 pillars of AI visibility measurement#
At Discovered Labs, we measure AI visibility across four pillars: what the model already knows, how it forms an answer, what it retrieves, and what ultimately reaches the buyer. We probe the system end to end so that an absence comes with its cause rather than just its fact.
An AI answer is the end of a pipeline. The model reaches first for what its weights already hold, retrieval may add current documents, and a response is written from whatever remains available. Almost every AI visibility tool built so far measures that final response and nothing else. It asks repeatedly, records what comes back, and turns the result into a visibility score. The final response is the step closest to revenue, but it cannot explain how the model arrived there.
The same brand can be strong in one pillar and absent from another, which is why a single AI visibility score can hide where the underlying problem begins.
Framework: Discovered Labs AI visibility measurement approach.
1. Parametric probing: Measure what the model knows before web search#
Parametric probing reads what the model carries in its own weights before it has seen a retrieved document. Those weights are fixed when training ends, identical for every user, and unchanged by anything you publish afterward. Reading them means measuring the model rather than asking it, which requires open weights or token probabilities that most providers no longer expose.
Parametric probing shows what a model already believes about your brand when nobody has handed it anything to read. For marketers, it separates a long-term training-data problem from a live retrieval problem.
2. Mechanistic interpretability: Trace how the model forms an answer#
Mechanistic interpretability examines what the model does while an answer forms. The cheap read is chain-of-thought, taking the model at its word about where a brand fell out of the shortlist. The stronger read inspects the computation itself through techniques such as logit lens and activation patching, which trace how internal signals affect the answer.
Mechanistic interpretability shows where a brand was lost during the model’s deliberation, even when information about that brand was available. For marketers, it identifies which category, comparison or proof signals may need strengthening.
3. Retrieval analysis: Track what the model fetches and reads#
Retrieval is a pipeline of its own. A question can fan out into several queries, which are routed across different stores before the results are filtered, deduplicated and trimmed to fit the context window.
Being fetched is not being used. In one measured session, the most-read source was never cited at all, and roughly two thirds of a page was cut before the model saw any of it.
Retrieval analysis shows whether useful information about a brand was in front of the model when it needed it, and which step dropped that information. For marketers, it points toward crawlability, source coverage and content that answers the retrieved query directly.
4. Response analysis: Measure what buyers see and what gets cited#
Response analysis measures what reaches the buyer and which sources the answer cites. This is where most AEO tooling sits because it requires no special access. Ask repeatedly, record what comes back, and compare the results across prompts, models and time.
The final response is also the most volatile of the four pillars because anything retrieval-backed can change the moment you publish. Response analysis shows what buyers see today and whether the brand information they receive is correct, different from the verified facts or missing entirely.
For marketers, the response is the commercial outcome and the feedback loop. A missing or inaccurate answer should be traced back through the other three pillars to identify what needs to change.
How models learn, and what they remember#
Frontier AI models are trained in stages, from corpus assembly through pre-training, mid-training and post-training. Each stage affects a brand differently, from whether information reaches the model to how deeply it is stored and how it is later expressed.
What is parametric memory?#
Parametric memory is the information an AI model stores as patterns in its weights during training. It is the layer the model can use before web search or retrieval adds current documents, and it is the layer measured throughout this study.
Swipe to explore →
Pre-training and mid-training write factual associations into model weights; corpus admission controls what enters, while post-training shapes how stored knowledge is expressed.
Framework: model-training stages and measurement routes described in the accompanying methodology.
Underneath every training stage sits the same operation. The model reads the corpus a few tokens at a time, predicts the next token, and every wrong guess nudges billions of weights toward the right answer.
Three variables determine how much influence a token of your text has on the final weights:
- Repetition: How often the same fact recurs across the corpus in independently phrased forms. One page in a fifteen-trillion-token corpus, seen once, moves the weights about as much as one raindrop moves a river. Every independent restatement creates another pass over the same weight directions. That is how the US Constitution's 218 training copies end up recitable. Repetition is the strongest single dial, and the one we can verify against ground truth below.
- Training stage: When those passes happen. Passes do not simply accumulate because the effect of each one depends on the learning rate when it lands. The same passage counts for more late in training, which is why labs anneal curated, high-quality corpora into the final stretch. Exposures seen only early can partially fade as later training runs over them. Mid-training is the one stage where deliberate inclusion is possible.
- Pipeline processing: The data engineering between a source and the training run. Admission filters and deduplication decide whether text arrives at all. The channel carrying it, whether a filtered crawl, licensed corpus, curated collection or synthetic rewriting, affects its chances. Reference-shaped content survives at far higher rates than conversational prose, which the source survey measures across 30 source types.
Swipe to explore →
Repetition, training timing and pipeline treatment jointly determine how strongly a passage changes a model's weights.
Framework: model-training stages and measurement routes described in the accompanying methodology.
The reason to care is lifespan. Retrieved text influences one response and competes for limited context, while information stored in the weights remains available across every answer built on that model.
Training pipeline: how information reaches model weights#
AI models learn from several sources, including filtered web crawls, licensed corpora, curated books and code collections, first-party material and synthetic text. Material from a web crawl must survive corpus-admission filters before it reaches the weights.
- Deduplication: Exact copies collapse to one document. Near-duplicate detection also prunes paraphrased clones.
- Perplexity filtering: Text that a small reference model finds unusually difficult to predict is dropped. This catches both machine-generated filler and genuinely unusual writing.
- Quality classifiers: A model trained to distinguish curated writing from random web text scores every document, so a good page formatted like a link farm can die here.
- Additional filters: Language identification, boilerplate and navigation stripping, toxicity and PII filters, and per-domain blocklists. Each filter varies by lab and by training run.
Published open pipelines show how aggressive this filtering can be. RefinedWeb discards roughly nine in ten raw documents before training. The most-used subsets of FineWeb retain only a low single-digit percentage.
For brands, this creates two practical consequences:
- The public web is one source among several: Licensed corpora, curated book and code collections and synthetic text never pass through the public-web pipeline. They still carry product discussion into training, reducing the relative influence of the open web marketers can actually reach.
- Verbatim syndication adds retrieval reach, but little training evidence: A press release republished verbatim to forty sites survives admission as one document. It can win citations at answer time while adding almost nothing to the weights. Training rewards independent restatement: the same fact reached and phrased differently by different authors on different domains.
Training continues after corpus admission, and each stage affects a brand differently:
- Pre-training: The filtered training mixture is where facts about a brand enter the weights.
- Mid-training: Curated technical corpora and long-context data are annealed into the final stretch of pre-training. This is the one stage where a recognized authority in a narrow domain can be included deliberately rather than by crawl statistics.
- Post-training: Instruction tuning and preference optimization are now built largely on expert-rated examples and model-generated synthetic data. Post-training adds no facts at all. It decides how knowledge already stored in the weights is expressed, including whether a model names a specific brand or hedges into a category answer.
Swipe to explore →
Each measurement method attaches to a different stage of the training pipeline, so no single test answers every question.
Framework: model-training stages and measurement routes described in the accompanying methodology.
Model access: what open and closed AI models let us measure#
Model access determines how directly we can measure what an AI model holds in its weights. Open-weight models allow the full two-detector reading. Models with log-probability APIs, which expose the probability assigned to each output token, allow a partial reading. Text-only systems leave verbatim tail probing, which provides near-conclusive evidence when it fires and weak evidence when it stays quiet. This study tests models across that full spectrum.
Swipe to explore →
The more model internals a provider exposes, the more directly researchers can measure stored knowledge.
Framework: model-training stages and measurement routes described in the accompanying methodology.
Open models give us an inspectable view into the wider model families they come from. Gemma is distilled from the same pipeline as Gemini, and gpt-oss has the same relationship with its closed siblings. The open-model reading therefore provides a floor on what the closed family knows.
Distillation can thin the tail of what the teacher model learned, so a faint imprint in the teacher may disappear in the student. That is why every score in this study remains attached to the model we tested.
Swipe to explore →
An inspectable student shows which patterns survived in its own weights after distillation and direct training.
Framework: model-training stages and measurement routes described in the accompanying methodology.
In this study, asking what an AI model "knows" about a brand means measuring how strongly training text about the brand changed the model's weights and how much of that trace survived the rest of training.
We call the depth of that surviving trace the imprint. The training run fixes a brand's imprint, so it does not change with the person asking the question and can be measured consistently.
What the crawl cannot tell you
Common Crawl is worth tracking, and Metehan's referring-domains web graph is a reasonable way to check it. Common Crawl covers one of several training sources. The graph measures the crawl itself, while a weight-level audit asks five downstream questions the crawl cannot answer:
- Influence: How deeply a brand's text marked the model's weights, which corpus presence alone cannot show.
- Accuracy: What the model asserts the brand is, and whether that answer is correct, deviated or unadded.
- Co-occurrence: Which ideas, categories and companies the model stores alongside the brand's name.
- Sentiment: How the model evaluates the brand across the dimensions a buyer weighs.
- Competitive: The same four readings for every rival, because a buyer chooses one vendor from a shortlist.
What is ModelRank, and how is it measured?#
ModelRank is a measurement system developed by the AI research team at Discovered Labs to evaluate what a specific AI model has learned about a brand before live web retrieval. For each model, it produces a competitor-relative score from 0 to 100 and four underlying readings: base imprint, co-occurrence, factual accuracy, and sentiment. Most B2B SaaS companies have no baseline for this. A model naming a competitor instead can reveal a problem, but that answer alone does not show whether the weakness is imprint depth, co-occurrence, factual accuracy, sentiment, or competitive position, or which one to work on.
ModelRank
A log-scaled 0 to 100 reading of accumulated influence in a specific model's weights. It is calculated separately for each model and read against the competitors the brand is chosen alongside. ModelRank carries four underlying readings: base imprint, co-occurrence, factual accuracy, and sentiment. Together, the score and readings show which of the five questions is weak.
- Near 0: The model holds nothing it can be shown to have learned from the brand. At the floor, the score reports uncertainty instead of pretending to an exact number.
- Mid range: Text from the brand left a trace the detectors can find, but not one the model can reproduce. Nearly every brand in this study sits here.
- Near 100: The model can continue the brand's text from a single line. Famous documents sit here. No B2B SaaS brand in the 1,000-company cohort reaches this range.
Swipe to explore →
ModelRank combines detector evidence with factual accuracy and sentiment to show where a brand's stored representation is weak.
Method: Discovered Labs ModelRank protocol described in this study.
The four readings under the score#
- Base imprint: How deeply the model's weights hold information from the brand. If training on that text nudged the model, it leaves a readable residue: the model is less surprised by the text than context alone would justify. The signal is especially useful around names and numbers because context cannot guess them. A second detector scrambles the surface form and checks whether the model's confidence collapses. Understanding survives paraphrase. Memorization does not. Agreement between detectors approaching from different directions is the main reason to trust either.
- Co-occurrence: Which categories, ideas, and companies the model holds beside the brand name. This is the reading marketers can work on most directly through what other people publish next to the brand for future model training. It also helps explain why factual accuracy and sentiment read as they do.
- Factual accuracy: Whether the claims stored about the brand actually belong to it. We ask the model to rank the true value against matched distractors. We label the outcome correct, deviated, or unadded. When the result deviates, it is almost always because a wrong category sits closer to the brand name in the model's stored associations than the right one. This points to strengthening co-occurrence with the right category because a correction alone does not change that association.
- Sentiment: The evaluative direction the training data taught the model to attach to the brand, read across 12 buying dimensions. Following Tigges et al. (2023), we fit that direction from contrast pairs and project the brand token onto it. A weak dimension means the model did not learn a clear evaluative association between the brand and that dimension. In this study, that points back to a co-occurrence problem: the writing in the training data did not place the brand beside that dimension strongly enough.
Sentiment positions are comparable within the same buying dimension only. A position on one dimension cannot be compared directly with a position on another.
Expand how the sentiment reading is taken
The direction is fitted once from contrast pairs, then every brand is projected onto the same axis. Each dimension uses its own prompt frame, so positions are comparable within a dimension and not across them.
Swipe to explore →
Every brand is projected onto the same learned sentiment direction within each buying dimension.
Method: Gemma 4 12B contrast-pair projection across 12 dimensions; technical paper §4.4.
How the detectors were validated#
Before applying them to brands, every detector was tested on documents whose presence in the training data could be checked directly. The normalized result used throughout this report means the detector ranks a true member above a true non-member 9 times in 10. Its AUC is 0.90, compared with 0.50 for chance.
The raw detector scored 0.83. Normalizing it against a reference model lifted the result to 0.90. On verbatim memorization, it separated perfectly at 1.00.
Swipe to explore →
Normalization lifted detector separation from 0.83 to 0.90 AUC, while verbatim memorization reached 1.00.
Source: Pythia-2.8B labeled member/non-member validation; AUC, where 0.50 is chance.
- Labeled sets: Passages known to be inside a public training corpus on one side, with post-cutoff text and entity-swapped mutations on the other.
- Dose response: The detector response grew stronger as the number of copies in the training corpus increased. We tested this using Pythia, whose training data is public, with copy counts looked up through infini-gram. The US Constitution, recorded at 218 copies, is reproduced verbatim by every provider we probed. Moby Dick, at 5.5 copies, scores at the same level as documents that were never present.
- Benchmark: The test setup reproduces published results on the standard membership-inference benchmark.
- Prior work: Memorization and its growth with duplication are established findings from Carlini, Kandpal, and Shi. The literature does not apply those methods to commercial brands.
Validation protocol and result files are available on request.
What is actually in the training data?#
The models we tested showed the strongest weight-level traces for code and technical material, structured lists, and numerical content. Personal opinion, flowing prose, and question-and-answer content produced weaker signals, while passage format mattered more than the site where it appeared.
We sampled more than 2,000 passages from more than 30 source types, from Stack Overflow answers to press releases, and measured how strongly each registered in the models' weights.
Three ways we tested what made it into the weights
- Maximum imprint: Famous books that the models can continue from a single line establish the top of the scale. Every lower reading is normalized against this ceiling.
- Source traces over time: More than 30 source types were compared across four model generations. Training cutoffs separate content published early enough to enter the weights from content published too late.
- Content shape: Passages were grouped by format to identify the characteristics associated with the strongest traces.
What maximum imprint looks like#
Given only the end of a famous opening line, the models continue most of these books word for word.
| Book | Given the opening line's tail, the models... |
|---|---|
| Pride and Prejudice (1813) | continue verbatim: "a good fortune, must be in want of a wife" |
| A Tale of Two Cities (1859) | continue verbatim through four clauses of the famous antithesis |
| Moby Dick (1851) | continue verbatim for 31 words |
| 1984, Harry Potter, The Hobbit, The Great Gatsby, The Hunger Games | continue each opening word for word |
| Gone Girl (2012) | continue a mid-book passage word for word from a single line |
| The Da Vinci Code, The Martian | drift into plausible but invented continuations, showing an imprint without word-for-word recall |
Books repeated across hundreds of independent copies establish the maximum-imprint benchmark. Every other reading in this report is normalized against that ceiling.
How training cutoffs reveal what entered the weights#
Each model generation has a training cutoff. For every source, we compared passages published before the cutoff with passages published after it.
A stronger reading for the earlier passages is evidence that the source entered the weights. If the two groups score alike, the detector cannot confirm passage-level memory. The similarity may come from the style of the writing instead.
We ran the same survey across four generations of the same model family. Each was a base pretrained model, before chat-specific post-training, which allowed us to compare the traces left by successive training pipelines.
Because the models are distilled from larger teacher models, a faint trace in the teacher can disappear in the smaller model. A positive reading supports ingestion. A null reading means the detector found no fingerprint, but it does not prove that the source was absent from training.
Swipe to explore →
YouTube had the largest positive before-versus-after-cutoff detector gap in Gemma 4, while several sources clustered close to zero.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
Swipe to explore →
YouTube had the largest positive detector gap in Gemma 3, while most source differences sat closer to zero.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
Swipe to explore →
YouTube showed the largest positive detector gap at both tested Gemma 2 sizes.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
Swipe to explore →
Gemma 1 showed wider positive gaps for social and community sources than later generations.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
Swipe to explore →
The Guardian and Urban Dictionary were among davinci-002’s largest positive gaps; G2 and LinkedIn were negative.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
Swipe to explore →
babbage-002 showed its widest negative source gap on Urban Dictionary and its strongest positive gap on AO3 fanfiction.
Source: Discovered Labs survey of 2,000+ passages across 30+ source types; positive detector gaps support ingestion.
What changed across model generations#
The values below compare passages published before and after each training cutoff. A positive score means earlier content left a stronger trace. Zero means no measured difference. A negative score means the later content scored higher.
- YouTube remained the strongest source, but its signal weakened. Its score fell from +1.89 to +1.61, +1.38, and +1.18 across model generations 1 to 4. No source replaced it at the top, while the wider open-web profile flattened beneath it.
- Editorial and research sources became more detectable. PubMed went from no detectable trace to +0.36, The Guardian rose from +0.32 to +0.75, and Forbes moved from flat to +0.41.
- Court opinions recorded the largest reversal. Their score fell from +1.43 to −1.61, a larger reversal than LinkedIn's decline from +1.53 to −0.58. Two of the deepest traces in generation 1 had reversed by generation 4.
- Stack Overflow shows that reference-like formatting is not enough by itself. Its score fell from +1.08 to zero in generation 4, despite holding at +1.30 in generation 3 and reappearing in the 31-billion-parameter model.
- Review platforms produced sharply different results. Amazon reviews held at +1.05, while Trustpilot fell to −0.14 and G2 sat at zero. Reddit was null in every generation, including generations trained before any licensing deal.
- The average signal weakened, but individual sources changed more dramatically. The mean score declined from 0.551 to 0.496, 0.442, and 0.375 across successive generations. This is consistent with licensed and synthetic sources taking a larger share, while individual sources moved much more than the average.
The shape rule: what survives the pipeline#
Passage format was the strongest predictor of a detectable trace. The site where the passage appeared mattered less.
Swipe to explore →
Technical, structured and numeric passages left stronger traces than prose, reviews and question-and-answer formats.
Source: Discovered Labs content-shape analysis; technical paper §5.9.
The material with the strongest traces looked like a reference shelf: code, lists, numbers, and technical content. Personal opinion, flowing prose, and question-and-answer formats produced weaker signals.
Marketer action: build a reference layer for your core brand facts
Start with the facts you most need models to describe correctly, such as your category, product definition, target customer, integrations, pricing model, and key differentiators. Give each fact a stable, maintained home in a documentation page, glossary entry, comparison table, specification sheet, or benchmark page.
Then convert the important facts inside campaign content into permanent reference assets:
- Turn an integration announcement into a maintained integration page.
- Add a results table to each case study, including the starting point, outcome, and timeframe.
- Publish an HTML findings page alongside each research PDF, with the definitions and core numbers in the page copy.
- Give category definitions and product specifications their own clearly labeled sections instead of burying them inside thought-leadership articles.
Use editorial and social content to explain and distribute these facts, but do not leave a LinkedIn post, Reddit thread, campaign page, or PDF as their only authoritative home.
The audit: what AI knows about 1,000 B2B SaaS brands#
We audited 1,000 public B2B SaaS brands across 333 product categories, from CRM to observability, using two baseline checks:
- Recall: does the model name the brand when asked which companies exist in its category?
- Category accuracy: when tested directly, does Gemma 4 rank the brand's verified product category above four matched alternatives?
The baseline checks flag which area needs closer examination. ModelRank adds four readings: base imprint shows how strongly the brand registers, co-occurrence shows which entities appear alongside it, factual accuracy shows which answer the model prefers and by how much, and sentiment compares how the brand is described across 12 buying dimensions and relative to competitors.
AI brand recall: Gemma 4 named 510 of 1,000 B2B SaaS brands#
To measure recall, we gave Gemma 4 two unbranded sentence-completion prompts for each product category. One was "the most widely used CRM software tools for businesses are." We sampled each prompt six times, producing 12 completions per category. A brand counted as recalled if its name appeared at least once.
Swipe to explore →
Gemini 3.5 Flash Lite named 73.3% of the 1,000-brand cohort in the separate assistant cross-check.
Source: separate four-sample-per-category behavioral cross-check; n = 1,000; technical paper §5.10.
Swipe to explore →
Gemma 4 named 510 of the 1,000 brands across 12 unprompted completions per category.
Source: 1,000 brands across 333 categories; 12 completions per category; technical paper §§4.1 and 5.1.
Swipe to explore →
Gemma 3 named 51.1% of the same 1,000-brand cohort in the generation comparison.
Source: 1,000 brands across 333 categories; 12 completions per category; technical paper §§4.1 and 5.1.
Swipe to explore →
Gemma 2 named 44.3% of the same 1,000-brand cohort under the identical Gemma protocol.
Source: 1,000 brands across 333 categories; 12 completions per category; technical paper §§4.1 and 5.1.
Swipe to explore →
Gemma 1 named 39.8% of the same 1,000-brand cohort under the identical Gemma protocol.
Source: 1,000 brands across 333 categories; 12 completions per category; technical paper §§4.1 and 5.1.
Swipe to explore →
GPT-5.6 Luna named 78.5% of the 1,000-brand cohort in the separate assistant cross-check.
Source: separate four-sample-per-category behavioral cross-check; n = 1,000; technical paper §5.10.
Swipe to explore →
Claude Haiku 4.5 named 60.2% of the 1,000-brand cohort in the separate assistant cross-check.
Source: separate four-sample-per-category behavioral cross-check; n = 1,000; technical paper §5.10.
- Gemma 4 named 510 brands and left 490 unnamed in this 12-completion test. Half of the cohort failed to appear when the model was asked what tools existed in each brand's product category.
- Recall increased through Gemma 3, then held level. On the identical cohort, prompt set and matcher, recall was 39.8% on Gemma 1, 44.3% on Gemma 2, 51.1% on Gemma 3 and 51.0% on Gemma 4. Marketers should work on persistent recall gaps directly because another model release may leave them unchanged.
- Closed assistants provide separate behavioral benchmarks. Claude Haiku 4.5 named 60.2% of the cohort, Gemini 3.5 Flash Lite named 73.3% and GPT-5.6 Luna named 78.5%. Their recall rates came from four samples per category, while the Gemma 4 rate came from 12. Closed models also do not expose their weights, so keep each assistant as a separate benchmark.
- Use your category as the competitive benchmark. Project management surfaced 8 of 12 brands and CRM surfaced 7 of 12, while education technology surfaced 2 of 11 and databases surfaced 2 of 10. Category size and category visibility can diverge, so compare your brand with the competitors in its own category.
- Repeated samples are essential at brand level. Reseeding the same prompts changed the named or unnamed result for 13% of brands. Run the category prompts several times and track the trend before setting priorities.
Marketer action: build a category-level recall benchmark
Create a fixed set of unbranded sentence-completion prompts from the category phrases your buyers use. Run each prompt multiple times on every assistant you track, record how often your brand and main competitors appear, and keep each assistant as a separate series.
Repeat the same prompts and sampling settings quarterly. Investigate phrases where your brand remains missing across runs or a small group of competitors dominates. Use those gaps to prioritize clearer category language on owned reference pages and accurate third-party descriptions, then use later recall tests as the feedback loop.
AI brand accuracy: what 194 different category choices reveal#
AI brand accuracy measures whether a model associates a brand with its verified product category. Gemma 4 matched the study's category for 787 brands and chose a different category for 194 of the 981 brands it classified (19.8%). The remaining 19 brands produced no category preference.
For each brand, the study scored five sentences that differed only in the product category. One contained the verified category, while four contained length-matched distractors. Gemma 4 ranked the sentences using mean token log-probability.
- Consistent: The verified category ranked first. The margin over the runner-up measured how strongly it was preferred.
- Unaddressed: No category separated from the others by a meaningful margin.
- Deviated: One of the four distractor categories ranked first.
Swipe to explore →
Gemma 4 matched 787 of 981 category commitments, differed on 194, and produced no preference for 19 further brands.
Source: Gemma 4 12B five-candidate category ranking; 981 commitments and 19 no-preference results; technical paper §§4.2 and 5.3.
Category errors ranged from thin margins to strong ones
Fourteen brands were confidently assigned a different category, while another 180 received a different category at a lower margin. Both appear as ordinary category answers to buyers because assistants do not display the underlying score.
- Well-known brands were misfiled too. Gemma 4 placed Okta under CI/CD instead of identity, Square under card issuing instead of SMB commerce, Akamai under CI/CD instead of CDN, and HashiCorp under workflow orchestration instead of infrastructure security.
- Some matching choices had very small margins. Eighty-four of the 787 matching choices won by less than 0.2. Sumo Logic held observability by 0.020, while Conductor held SEO by 0.024.
Different assistants fail in different ways
A separate output-based test ran the same 1,000 brands through GPT-5.6 Luna, Claude Haiku 4.5 and Gemini 3.5 Flash Lite. Among answers that chose a category, the different-category rate was 19.8% for Gemma 4, about 2% for Claude, 0.8% for Gemini and 0.4% for GPT. GPT declined to classify 287 brands, while Gemini declined on 8.
The closed assistants saw the category options and could answer "I do not know." Gemma 4 ranked candidate sentences with no abstention, so the lower closed-model error rates include a different test and different refusal behavior.
Swipe to explore →
The strongest 609 category commitments reached a 95% accuracy bar, while 742 reached a 90% bar.
Source: 981 Gemma 4 category commitments ranked by decision margin; technical paper Figure 3.
Category errors were concentrated among brands with smaller margins. Accuracy reached 90% across 742 brands and 95% across 609 when the brands were ranked by score margin. The other 258 brands carried almost all the different-category choices.
Across the full cohort, more than 400 brands had either a different category or a weakly held result.
Swipe to explore →
Different-category rates ranged from 0% in IT and internal operations to 37% in regulated verticals.
Source: full 1,000-brand cohort under the corrected Gemma 4 category probe; technical paper Figure 4.
Swipe to explore →
Gemma 4 held product category far more clearly than pricing, free-tier and self-serve facts.
Source: Gemma 4 12B; category n=1,000; integrations n=2,400; free tier n=402; pricing n=585; self-serve n=438.
Commercial facts were much weaker than category knowledge
Gemma 4 selected the verified product category for 78.7% of brands in a five-way test where chance was 20%. For the four yes-or-no facts, an AUC of 0.500 represents chance. Integrations reached 0.614, free-tier availability 0.547, published pricing 0.526 and self-serve signup 0.450.
Integration facts appear in both vendors' documentation, comparison pages and changelogs. Pricing often lives on one vendor page. That wider restatement helps explain why integrations were held more clearly.
Having a help center was associated with 3.81 points higher adjusted category accuracy per standard deviation (n = 869, 95% CI 1.16 to 6.46, adjusted p = 0.020).
Category errors clustered by label and industry
The 194 different-category choices covered 94 categories, with 10 categories absorbing 33% of them. Compact technical labels acted as category magnets. CI/CD attracted 9 misfiled brands while containing 2 cohort brands of its own, and web hosting attracted 6 while containing 1.
Category accuracy also varied sharply by industry:
- Regulated verticals: 37% different-category rate.
- Developer tools: 30%.
- Customer experience: 24%.
- Finance and legal: 24%.
- Sales and marketing: 18%.
- HR and operations: 16%.
- Security and identity: 12%.
- Data and analytics: 10%.
- Product and design: 10%.
- IT and internal operations: 0%.
Marketer action: standardize your category and publish current commercial facts
Choose one primary category and write a plain sentence that states what the product does. Keep that meaning consistent across the homepage, product pages, help center, partner pages, directories and company profiles.
If an assistant selects another category, search for that label across the same sources and correct descriptions that are inaccurate, ambiguous or outdated. Keep pricing, free-tier availability and signup requirements current on dedicated pages that assistants can retrieve.
After major changes, check how each assistant categorizes the brand and use any different or missing answer to choose the next pages and profiles to update.
AI brand sentiment reflects how companies are written about#
AI brand sentiment reflects the evaluative language models learn around a company. Across 921 brands with G2 or Trustpilot ratings, customer ratings were almost unrelated to the sentiment stored in the weights. The correlation was 0.030 (p = 0.36).
We measured that learned portrayal across 12 buying dimensions, including onboarding, ease of use, reliability, documentation, support and pricing.
Swipe to explore →
Onboarding, ease of use and reliability showed the widest gaps between the best- and worst-regarded brands. Pricing showed the narrowest.
Source: Gemma 4 12B; top and bottom 100 brands by mean z-score across 12 dimensions; technical paper Figure 8.
Onboarding and ease of use showed the largest sentiment gaps
Comparing the 100 best-regarded brands with the 100 worst-regarded brands, onboarding produced the largest gap at 3.3 standard deviations. Ease of use followed at 3.2, reliability at 3.1 and documentation at 3.0. Pricing produced the smallest gap at 1.8. The model's view of a company varied most around what the product feels like to adopt and run.
A broad reputation signal accounted for 55% of the variation across the 12 dimensions, and the median correlation between any two dimensions was 0.52. Documentation and onboarding moved together at 0.89. The remaining variation makes each dimension useful for identifying specific strengths and weaknesses.
Where brand names appeared with a product category
A separate category analysis measured how often snippets paired a brand name with its product category. The frequency varied sharply by source:
- News sources connected brands with their categories more often: TechCrunch snippets did so 33.8% of the time and Forbes snippets 28.2%.
- Developer sources made the connection less often: GitHub snippets stated the category 14.4% of the time and Stack Overflow snippets 9.8%. A dependency manifest can record that a tool exists without stating what it does, which helps explain why developer surfaces were associated more strongly with recall than category accuracy.
- Recall and category accuracy shared three web signals: Common Crawl graph rank, referring domains and Wikipedia citations of the brand's domain.
Who leads and who lags, category by category
The tables show who leads and who lags within each category. Zoho led CRM, Moz ranked above Ahrefs in SEO and Monday.com ranked above Atlassian in project management. Lower-ranked brands often trailed on support, onboarding or ease of use.
Choose a category below to see every brand in it, ranked by how the model reads them.
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Insider | +1.26 | product depth | innovation | |||
| Customer.io | +0.71 | ease of use | integrations | |||
| OneSignal | +0.66 | community | support | |||
| Iterable | +0.65 | pricing | reliability | |||
| Klaviyo | +0.52 | ease of use | reliability | |||
| CleverTap | +0.46 | community | support | |||
| Braze | +0.38 | community | support | |||
| Postscript | +0.30 | onboarding | pricing | |||
| ActiveCampaign | +0.23 | ease of use | trajectory | |||
| Attentive | +0.10 | onboarding | community | |||
| Marketo | -0.17 | support | onboarding | |||
| Dotdigital | -0.93 | support | community | |||
| MoEngage | -0.95 | pricing | integrations | |||
| Mailchimp | -0.96 | product depth | support | |||
| Bloomreach | -1.55 | trajectory | community |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Moz | +1.42 | product depth | innovation | |||
| Clearscope | +1.05 | community | pricing value | |||
| Botify | +0.88 | community | trajectory | |||
| Discovered Labs | +0.79 | ease of use | trajectory | |||
| Peec AI | +0.62 | reliability | integrations | |||
| Similarweb | +0.54 | support | pricing value | |||
| Conductor | +0.46 | reliability | pricing value | |||
| Lumar | +0.41 | support | documentation | |||
| Surfer | +0.26 | ease of use | trajectory | |||
| SE Ranking | +0.26 | support | pricing value | |||
| Frase | +0.17 | support | trajectory | |||
| MarketMuse | +0.09 | trust and security | pricing value | |||
| Semrush | +0.03 | community | pricing value | |||
| BrightEdge | -0.09 | support | integrations | |||
| Ahrefs | -0.20 | community | ease of use | |||
| Profound | -1.93 | product depth | onboarding |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Zoho | +1.49 | product depth | integrations | |||
| HubSpot | +1.14 | product depth | innovation | |||
| Attio | +0.95 | reliability | trust and security | |||
| Close | +0.52 | pricing | reliability | |||
| SugarCRM | +0.29 | support | pricing | |||
| Creatio | +0.29 | onboarding | pricing | |||
| Insightly | +0.12 | documentation | trust and security | |||
| Salesforce | +0.07 | trajectory | product depth | |||
| Freshworks | +0.00 | support | community | |||
| Copper | -0.09 | trajectory | support | |||
| Keap | -0.49 | support | community | |||
| Pipedrive | -0.59 | product depth | support |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Tidio | +1.43 | community | pricing | |||
| LiveChat | +1.30 | ease of use | integrations | |||
| Gladly | +1.09 | community | support | |||
| Kustomer | +0.98 | onboarding | support | |||
| Gorgias | +0.94 | community | trajectory | |||
| Freshdesk | +0.58 | ease of use | pricing | |||
| Intercom | +0.55 | documentation | trajectory | |||
| Front | +0.47 | pricing | support | |||
| Zendesk | +0.32 | documentation | support | |||
| Help Scout | +0.27 | innovation | integrations | |||
| Hiver | -0.17 | ease of use | trust and security | |||
| Pylon | -0.48 | trust and security | support |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Monday.com | +1.43 | reliability | integrations | |||
| ClickUp | +0.66 | community | pricing | |||
| Asana | +0.62 | community | pricing | |||
| Basecamp | +0.51 | innovation | product depth | |||
| Linear | +0.50 | trajectory | trust and security | |||
| Trello | +0.48 | community | pricing | |||
| Shortcut | +0.17 | trajectory | support | |||
| Wrike | +0.05 | community | innovation | |||
| Height | -0.03 | documentation | support | |||
| Teamwork | -0.19 | reliability | community | |||
| Smartsheet | -0.32 | product depth | trajectory | |||
| Atlassian | -1.46 | pricing | onboarding |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Paddle | +1.44 | product depth | innovation | |||
| Adyen | +0.79 | community | trajectory | |||
| Flywire | +0.64 | pricing | trajectory | |||
| Stripe | +0.60 | pricing | reliability | |||
| Checkout.com | +0.35 | onboarding | trajectory | |||
| Wise Platform | -0.01 | trust and security | documentation | |||
| Payoneer | -0.44 | documentation | trajectory | |||
| Rapyd | -0.50 | support | ease of use | |||
| Lemon Squeezy | -0.79 | pricing | integrations | |||
| GoCardless | -0.81 | pricing | integrations | |||
| Airwallex | -1.00 | product depth | ease of use |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Make | +0.81 | product depth | support | |||
| Tray.ai | +0.76 | documentation | integrations | |||
| Paragon | +0.66 | pricing | reliability | |||
| Merge | +0.52 | pricing | reliability | |||
| Zapier | +0.34 | onboarding | reliability | |||
| Workato | +0.06 | support | product depth | |||
| n8n | -0.39 | pricing | integrations | |||
| SnapLogic | -0.40 | support | innovation | |||
| MuleSoft | -0.55 | support | innovation | |||
| Boomi | -0.57 | support | innovation | |||
| Celigo | -1.14 | support | community |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Tableau | +1.15 | documentation | onboarding | |||
| Zoho Analytics | +0.62 | onboarding | pricing | |||
| Metabase | +0.61 | innovation | trajectory | |||
| Domo | +0.39 | reliability | innovation | |||
| Sigma Computing | +0.37 | onboarding | trajectory | |||
| Hex | +0.24 | innovation | support | |||
| Mode | +0.23 | pricing | trust and security | |||
| ThoughtSpot | -0.35 | support | documentation | |||
| Qlik | -0.45 | trust and security | integrations | |||
| Looker | -0.46 | pricing | reliability | |||
| Sisense | -0.53 | support | documentation |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Anthology | +1.26 | product depth | innovation | |||
| Kahoot! | +0.70 | community | pricing | |||
| Panorama Education | +0.45 | onboarding | trajectory | |||
| Clever | +0.40 | pricing | trust and security | |||
| Turnitin | +0.13 | onboarding | trajectory | |||
| Nearpod | -0.20 | trust and security | onboarding | |||
| PowerSchool | -0.45 | onboarding | support | |||
| D2L | -0.53 | pricing | community | |||
| Instructure | -0.86 | pricing | community | |||
| Securly | -0.99 | trust and security | community | |||
| GoGuardian | -1.55 | support | community |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Better Stack | +0.86 | onboarding | integrations | |||
| AppDynamics | +0.01 | support | reliability | |||
| Chronosphere | -0.21 | pricing | documentation | |||
| Honeycomb | -0.29 | ease of use | trajectory | |||
| Grafana Labs | -0.30 | community | integrations | |||
| Sumo Logic | -0.35 | support | reliability | |||
| Datadog | -0.74 | product depth | integrations | |||
| New Relic | -0.85 | product depth | ease of use | |||
| Dynatrace | -0.92 | product depth | integrations | |||
| Splunk | -0.95 | community | ease of use |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| Neon | +1.37 | onboarding | community | |||
| Supabase | +0.36 | ease of use | support | |||
| MongoDB | +0.28 | pricing | trust and security | |||
| Timescale | +0.23 | support | documentation | |||
| PlanetScale | -0.07 | support | documentation | |||
| ClickHouse | -0.28 | pricing | documentation | |||
| Redis | -0.30 | documentation | support | |||
| InfluxData | -0.41 | onboarding | community | |||
| CockroachDB | -0.52 | product depth | integrations | |||
| SingleStore | -0.85 | support | documentation |
| Brand | Sentiment | Delta vs cohort | Distinctive for | Weakest on | Recall | Category fact |
|---|---|---|---|---|---|---|
| SentinelOne | -0.21 | support | innovation | |||
| Bitdefender | -0.42 | support | ease of use | |||
| Check Point | -0.49 | onboarding | trajectory | |||
| Palo Alto Networks | -0.59 | support | integrations | |||
| Zscaler | -0.60 | trajectory | ease of use | |||
| Malwarebytes | -0.94 | trust and security | community | |||
| CrowdStrike | -1.03 | support | innovation | |||
| Sophos | -1.07 | trajectory | community | |||
| Trellix | -1.09 | pricing | integrations | |||
| Netskope | -1.40 | product depth | innovation |
Marketer action: publish proof of your product experience
Use the category table to choose the one or two dimensions where your brand trails its closest competitors. Prioritize onboarding, ease of use, reliability and documentation because those dimensions produced the widest gaps in the study.
On product pages and customer stories, replace generic claims with specific evidence about setup, time to value, daily use and reliability. Keep onboarding guides and documentation current, and repeat the same accurate descriptions across software directories, partner pages and press materials.
Brand knowledge: where does it come from?#
Brand knowledge was associated with several parts of a company's web presence, from audience size and open-web prominence to help centers and pages that clearly state the product category. These signals followed different patterns for brand recall and category accuracy.
We evaluated 79 brand-level variables against two outcomes: whether models named a brand unprompted and whether they placed it in its verified product category. At the individual-variable level, results were adjusted for crawl-graph rank and referring domains. Related variables were then grouped into eight broader measures.
Three brand-level signals that stood out#
- Ordinary-word names were associated with lower recall and category accuracy. Brands with ordinary-word names had 14.1-point lower recall and 32.0-point lower category accuracy in the unadjusted results. After adjustment, the effects were -3.99 recall points per standard deviation (n = 885, adjusted p = 0.037) and -9.42 category-accuracy points (n = 869, adjusted p < 0.001).
- Category crowding was associated with recall. Brands in the sparsest third of categories were recalled more often than those in the most crowded third. Category accuracy differed by only 1.5 points, with no reliable pattern.
- Help centers were associated with better category accuracy. Having a help center was associated with 3.81 points higher adjusted category accuracy per standard deviation (n = 869, 95% CI 1.16 to 6.46, adjusted p = 0.020).
The eight measures used to compare channels#
The 79 variables were grouped into eight measures. Two describe overall brand prominence. The other six describe channels and assets that marketing teams can influence more directly.
Overall prominence
- Common Crawl prominence: the brand's rank in the open-web link graph.
- Social prominence: overall audience size across the eight measured platforms.
Marketing channels and assets
- Social presence: followers, audience and brand-category co-occurrence across Reddit, YouTube, LinkedIn, X, Instagram and Facebook.
- Technical authority: mentions, stars and brand-category co-occurrence across GitHub, Stack Overflow, Hacker News, dev.to and Medium.
- Category-explaining coverage: the share of pages mentioning a brand that also state its category.
- Review sites: mentions, brand-category co-occurrence and review volume across G2 and Trustpilot.
- News coverage: mentions and brand-category co-occurrence across the New York Times, the Guardian, Forbes and TechCrunch.
- Owned properties: help-center availability and depth, documentation, pricing disclosure, an integrations directory, familiarity with the brand's own website and whether that site appears in Common Crawl.
The six marketing-channel measures were evaluated after accounting for crawl-graph rank and referring domains.
What the prominence measures revealed#
Common Crawl prominence was associated with both outcomes. Each standard-deviation increase was associated with 12.58 points higher recall (n = 885, adjusted p < 0.001) and 7.93 points higher category accuracy (n = 869, adjusted p < 0.001). It was the only grouped channel measure with strong positive adjusted associations with both outcomes.
Social prominence was strongly associated with recall, while its category-accuracy result was essentially flat. Each standard-deviation increase was associated with 9.75 points higher recall (n = 909, adjusted p < 0.001) and 0.21 points higher category accuracy (n = 892, adjusted p = 0.911).
What this means for marketers
Treat recall and category accuracy as separate marketing problems.
- When recall is weak: make the brand name consistent across social profiles, software directories, partner pages and publisher mentions. If the name is an ordinary word or the category is crowded, include the product category wherever the name appears without context.
- When category accuracy is weak: choose one plain category description and repeat it across the homepage, product pages, help center, documentation, integrations directory, partner pages and review profiles. Pair the brand name with that category in page titles, introductions and profile descriptions.
What predicts being named#
Recall was strongest where brands had wider reach, repeated third-party mentions and coverage that connected the brand with its product category.
- Audience reach was associated with higher recall. Facebook, YouTube and LinkedIn follower counts were each associated with recall differences of roughly 30 points. Referring domains and rank in the Common Crawl graph showed similar differences. Simply maintaining a channel showed little relationship with recall.
- Category-explaining coverage was associated with both outcomes. This measure captures the share of pages that mention a brand and also state its product category. Each standard-deviation increase was associated with 3.92 points higher recall (n = 885, adjusted p = 0.045) and 3.15 points higher category accuracy (n = 869, adjusted p = 0.047).
- Repeated mentions carried more recall signal than simple presence. Brands with more Hacker News mentions had a 58.8% recall rate, compared with 40.8% among brands with fewer mentions. Stack Overflow showed a similar difference, 59.3% compared with 41.6%.
Swipe to explore →
Open-web and social prominence showed the largest adjusted recall associations among the variables that survived correction.
Source: Gemma 4 12B adjusted outcome models; 95% CIs and within-family adjusted p-values; technical paper Tables 5–6.
Swipe to explore →
Across the full model output, recall aligned most strongly with broad reach and repeated third-party presence.
Source: Gemma 4 12B adjusted outcome models; 95% CIs and within-family adjusted p-values; technical paper Tables 5–6.
What predicts being described correctly#
Category accuracy was linked more closely to what the coverage said and what the brand published on its owned properties.
- Coverage volume alone had little relationship with accuracy. A higher share of pages titled after the brand was associated with 10.7 points lower accuracy. Longer average mention text was associated with 22.7 points lower accuracy.
- Coverage that stated the product category was associated with better accuracy. Brands whose coverage named their category were categorized correctly 86.2% of the time, compared with 72.4% for other brands. That is a 13.8-point difference.
- Category context mattered on five of the 17 surfaces tested.
- Stack Overflow was associated with 18.7 points higher recall, followed by dev.to at 16.7.
- Reddit was associated with 13.1 points higher category accuracy, followed by Forbes at 11.6 and YouTube at 9.6.
Swipe to explore →
Category accuracy was positively associated with Common Crawl prominence, owned properties and category-explaining coverage.
Source: Gemma 4 12B adjusted outcome models; 95% CIs and within-family adjusted p-values; technical paper Tables 5–6.
Swipe to explore →
Across the full model output, category accuracy was positively associated with Common Crawl prominence, owned properties and category-explaining coverage.
Source: Gemma 4 12B adjusted outcome models; 95% CIs and within-family adjusted p-values; technical paper Tables 5–6.
Wikipedia coverage was associated with higher recall and lower category accuracy. Having a Wikipedia article was associated with 10.4 points higher recall and 16.7 points lower accuracy. Longer articles showed the same pattern, with 13.0 points higher recall and 14.7 points lower accuracy.
A detailed Wikipedia article can still foreground company history, funding and acquisitions while giving the current product little space. Check whether the opening sentence accurately states what the product is and who it serves. If the description is outdated, publish current independent sources and submit a correction request.
Open-web authority was associated with both recall and category accuracy. Common Crawl graph rank was associated with 29.5 points higher recall and 18.7 points higher accuracy. Referring domains were associated with increases of 29.8 and 17.2 points, while Wikipedia citations of the company domain were associated with increases of 24.4 and 13.2 points.
News coverage explained product categories more often than developer content. TechCrunch snippets stated the brand's category 33.8% of the time and Forbes did so 28.2% of the time. The rates were 14.4% for GitHub and 9.8% for Stack Overflow.
Developer pages frequently record package names, dependencies and technical usage with little product context. Add a plain category sentence to README introductions, package pages and documentation so these mentions also explain what the product does.
The practical response is to pair broader brand distribution with one consistent product description across the homepage, help center, documentation, directories, Wikipedia lead, READMEs and third-party profiles.
How to improve what AI models remember about your brand#
A model can remember your brand, put it in the wrong category, or describe it poorly. It can also hold strong pre-retrieval knowledge while live AI answers barely mention you. Diagnose the weak outcome before choosing the work.
Start with the weakest outcome#
| What you see | What it means | Marketing priority |
|---|---|---|
| Rarely named across repeated, unprompted tests | Brand recall is weak | Expand credible brand presence beyond owned channels. |
| Named, but placed in the wrong category or categorized inconsistently | The category association is weak | Reinforce one clear product-category description. |
| Named and categorized correctly, but described poorly | Brand sentiment is weak | Publish clearer evidence about the product experience. |
| Strong in pre-retrieval tests, but weak in live AI answers | Retrieval is weak | Improve the current pages and third-party sources assistants can find and cite. |
Six priorities for marketing teams
Use the priorities that match the diagnosis above.
- Build open-web prominence around your category. Open-web prominence showed the strongest positive associations with both recall and category accuracy. Earn authoritative mentions in trade media, industry directories, review profiles and the communities your buyers use, including relevant Reddit discussions and YouTube coverage. Make your brand, category and use case easy for each source to state accurately.
- Make your category association explicit and consistent. Category-explaining coverage was associated with both recall and category accuracy. Write one plain sentence stating what the product is, who it serves and what it helps them do. Keep that meaning consistent across your homepage, product and about pages, and editable third-party profiles. If the brand name is an ordinary word, add the category wherever the name appears without context. If models associate the brand with several categories, reinforce the preferred category across owned pages and independent coverage.
- Use social and technical coverage when recall is weak. Social prominence was associated much more strongly with recall than category accuracy. Technical authority was also associated with recall. Use these channels to create repeated brand exposure, and add a clear category explanation to technical pages and the permanent guides they lead to.
- Use documentation as category evidence. Owned reference material was more closely associated with category accuracy than recall. Open your help center, documentation homepage, READMEs and package pages with a clear description of the product, category and audience. Keep capability and integration descriptions current and consistent with third-party profiles.
- Publish specific evidence about the product experience. Customer ratings barely tracked how models described brands. Keep reviews as buyer proof, and document onboarding, ease of use, reliability, support and documentation through product content, customer stories and independent coverage.
- Keep commercial facts current and easy to retrieve. Gemma 4 barely distinguished true from false published-pricing claims, with an AUC of 0.53 where 0.50 is chance. Maintain a current, crawlable and plainly worded pricing page, then align the same facts across directories and product profiles.
Measure pre-retrieval knowledge and live answers separately#
The free ModelRank audit benchmarks pre-retrieval recall, category accuracy and the 12 sentiment dimensions against your competitors. The AI Visibility Tracker monitors live responses.
Run each benchmark per model and compare your brand with the same direct category competitors, because baseline recall differed widely by category. Live answers can change as sources change, while pre-retrieval knowledge changes when models are updated. Monitor live answers regularly and repeat ModelRank after major model releases.
Frequently asked questions#
What is AI brand recall, and how was it measured?
AI brand recall is whether a model names a brand without being prompted with that brand's name. In this study, Gemma 4 received two unbranded prompts per product category and six completions per prompt. A brand counted as recalled if it appeared in any of the twelve completions.
How often did Gemma 4 choose a different product category?
Gemma 4 chose a category that differed from verified ground truth for 194 of its 981 category commitments, a 19.8% different-category rate. Across the full 1,000-brand cohort, 787 choices matched, 194 differed and 19 produced no category preference. That is roughly one different choice in every five commitments.
What predicts whether an AI model names a brand?
Open-web and social prominence showed the strongest adjusted associations with brand recall. Each standard-deviation increase corresponded to 12.58 points higher recall for Common Crawl prominence and 9.75 points higher recall for social prominence. Repeated third-party mentions and coverage that clearly stated the product category also tracked stronger recall.
Was a company's own website more closely associated with recall or category accuracy?
A company’s own website was more closely associated with category accuracy. Having a help center corresponded to 3.81 points higher adjusted category accuracy per standard deviation among 869 brands. Clear, consistent descriptions across the homepage, product pages, documentation and help content give models stronger category context.
Was social following associated with brand recall or category accuracy?
Social prominence was associated with brand recall far more strongly than category accuracy. Each standard-deviation increase corresponded to 9.75 points higher recall among 909 brands, compared with 0.21 points higher category accuracy among 892 brands. Reach and clear product explanation therefore require different channel plans.
Did customer ratings match the sentiment stored in model weights?
Customer ratings barely matched the sentiment stored in model weights. Across 921 brands with G2 or Trustpilot ratings, the correlation was 0.030, with p = 0.36. Marketers should keep reviews as buyer proof while publishing specific evidence about onboarding, ease of use, reliability, support and documentation.
What is the difference between model weights and web retrieval?
Model weights hold patterns learned during training, including associations between a brand, its category and related concepts. Web retrieval adds current pages while an answer is being generated. Stored brand knowledge changes with model updates; retrieval-backed answers can change whenever assistants find newer or stronger sources.
Were ordinary-word brand names harder for AI models to place?
Ordinary-word brand names were associated with lower recall and category accuracy. After adjustment, the differences were 3.99 points lower recall and 9.42 points lower accuracy per standard deviation. Pair an ambiguous brand name with a plain product-category description wherever the name appears without context.
How should marketers improve AI brand recall and category accuracy?
Improve recall by earning repeated, credible mentions across trade media, directories, social channels, Reddit, YouTube and relevant technical communities. Improve category accuracy by repeating one plain product description across owned pages and third-party profiles. Keep pricing, integrations and product capabilities current in crawlable reference pages.
More research#
Continue with the other Discovered Labs studies on the retrieval and citation layers:
- What actually drives AI citations: a statistical analysis of 2M AI citations across 10K pages. A six-month analysis of the page and authority signals associated with AI citations.
- Reddit and LLM citations: 144,284 AI citations analyzed. An analysis of 144,284 AI citations showing how Reddit appears during search and in visible citations.