Why Does an AI Care More About Relevance Than Style?

Because when a language model decides which source to believe, it doesn't evaluate tone, references, or perceived authority — it evaluates how directly the text answers the question it was asked. A University of California, Berkeley study (ACL 2024) measured this by comparing 2,208 real evidence paragraphs against 238 controversial questions across 144 categories: stylistic changes — adding scientific references, sounding more objective, adding more information — had a neutral or negative effect on how convincing the text appeared to the model. Changes that increased direct relevance to the question, on the other hand, substantially improved that persuasiveness.

Of the eight criteria Nostos measures, this is the one that most directly decides whether a model cites you when someone asks something you can answer.

What they actually measured

For 238 contentious questions across 144 categories, they compared pairs of real web paragraphs — one answering "yes" and one answering "no" — and measured which one convinced five different models more (GPT-4, Claude, LLaMA-2, Vicuna, WizardLM). Semantic similarity between the question and the paragraph strongly predicted the outcome across nearly every model. Text readability and the number of unique words predicted nothing.

What effect style had

Neutral or negative. Adding scientific references, adding unrelated information, sounding more objective or more confident — none of those stylistic changes improved the odds that the model would pick that paragraph as the correct answer. What did work was a relevance change: prefixing the paragraph with a sentence that restated the question — "The following text is about the question: [question]" — was enough to substantially improve its win-rate against the opposing paragraph, while barely changing the content.

What this means for your content

That the section answering your own question needs to say so in the first lines, using the question's own words, before giving context or background. That's exactly what this criterion measures in Nostos: if a section's heading asks something, the answer belongs in that section's first paragraph, not the third.

What you can do today

Check every H2 on your site and ask whether the first sentence below it answers the heading above it. If not, move the answer to the top and push the context down. Three places where this fails most often:

• The article or page introduction, which often opens with an anecdote or a color detail instead of the answer.
• The first line of any question-shaped section (H2 or H3), where the answer only shows up in the second or third paragraph.
• The meta description, which often describes the topic instead of stating the answer upfront.