Named vs described: an entity can be 89 percent citable and still invisible

I run a fixed set of questions against answer engines every morning and record whether one specific entity gets cited. Today's run produced the cleanest split I have seen so far, and it is not the number I usually report.

The headline was 28.6 percent. That number is close to useless.

The setup

Sixteen questions, frozen wording, two engines (OpenAI's search model and grounded Gemini), one run per day at 04:00 UTC. Thirty-two datapoints per run. The questions are grouped into categories before the run, not after, so the grouping cannot be fitted to the result.

Today, run 1d1b24d7, snapshot 04:07:09 UTC:

Per engine: OpenAI search 27.1 percent, Gemini 30.2 percent. The two engines land within three points of each other, which is worth noting on its own, because it suggests the split below is not one vendor's quirk.

What the average hides

The overall 28.6 percent reads like a middling result you could improve by a few points with more work. The breakdown says something different. There are two separate regimes here.

When the question already contains the entity name, the engines find it almost every time. Nearly nine in ten. Retrieval is not the problem, indexing is not the problem, and the knowledge graph work has clearly landed.

When the question describes a need instead of naming a thing, the same entity is cited in roughly one in twelve answers, and in the two hardest categories it is zero out of eight. Those are the questions a real person actually asks. Nobody types the name of something they have not heard of.

So the average is the arithmetic mean of a solved problem and an unsolved one. Moving it from 28.6 to 32 could mean the described-need case improved, or it could mean the named case got slightly better at something it was already doing. Reported as one number, you cannot tell, and you will spend your effort on whichever is easier to move.

Why I think this generalises

Not because of my n. Because of what the two regimes are made of.

The named case is a lookup. The engine has a string, it resolves the string, it answers. Everything that helps there is entity infrastructure: a stable identifier, consistent naming, a canonical page that says plainly what the thing is.

The described case is a selection. The engine has a slot with room for three or four candidates and a population of thousands competing for it. Nothing about your own pages decides that, because the comparison happens somewhere else entirely, in whatever material the engine is drawing on to build its shortlist. That is why the same work that took the named case to 89 percent did nothing at all for the other one. They are not the same task wearing different hats.

Limits, honestly

Small n per category, between 2 and 12 datapoints. One run, one day, one entity, two engines. I am not claiming 8.3 percent is a stable value, and by tomorrow it will have moved. The finding I am willing to defend is the shape, not the numbers: the gap between the named and the described case is far too large to be sampling noise, it has been there in every run I have looked at, and both engines show it independently.

The practical version, if you measure this for anything: split your prompt set by whether the prompt contains the name, and report the two lines separately. If you report one average, you are averaging over the only distinction that tells you what to do next.

Live figures: marin-t-kael.de/en/research/dashboard