Confidence, sources and conflicts

Every story carries a score out of 100 and a one-word verdict beside it. ErmisAI calls it the Signal, and it is the first thing on a feed card.

It measures coverage, not truth. How many outlets filed on the event, how those outlets are weighted, how similarly they worded their headlines, and how recently the newest report landed. Nothing in the score has read the article. Nothing has checked a fact against a source.

A high score means an event is widely and freshly reported. It does not mean the reporting is right, and a low score does not mean the reporting is wrong.

The four labels

"Verified signal" is a band name, not a verification claim. Nothing has been verified.

Where the score appears

The n/100 and n% renderings are the same single number displayed two ways. There is no second score.

The feed filter bar has All categories, All pipeline, All editorial, All priority, Breaking only and a search box. There is no confidence filter and no way to sort by score.

What the score is built from

Four factors, fixed weights, no settings:

Two of these deserve to be read slowly.

Headline agreement is word overlap, nothing more. It compares the words in the headlines to each other. It does not compare figures, names, dates or claims. Two outlets that report contradictory death tolls under near-identical headlines score as high agreement. Two outlets that agree completely but headline the story differently score as low agreement.

Tier is a weight, not a quality certificate. It comes from the source catalog, and any feed you add yourself is always Tier 3, regardless of the outlet. A newsroom built mostly on its own custom feeds will see systematically lower scores. See Choosing and adding sources.

Why no story ever scores 0 or 100

The score is clamped to a floor of 40 and a ceiling of 99. Low signal in practice means 40 to 59. Nothing you see will ever read 100/100.

The floor matters more than the ceiling: a story with one thin source from a low-weighted outlet still shows 40-something, not zero. The score cannot express "there is almost nothing here".

A worked example, because the arithmetic is not obvious. A single wire report, no corroboration, published fifteen minutes ago:

• one source out of the five needed for full marks: 6 of 30
• top tier weight: 25 of 25
• a single-article story is given a fixed middling agreement value: 11 of 25
• under 30 minutes old: 20 of 20

Total: 62 — Developing signal. A well-sourced wire story minutes old sits in the third band, purely because nobody has corroborated it yet. Corroboration is the single biggest lever on the score.

When the score is set, and when it stops moving

The score is calculated once, when ErmisAI writes the source story, before any article exists.

The score only moves while the story is still an untouched Draft. Stories that arrive already In review — anything the pipeline scored below 85, or flagged sensitive — are frozen from the moment they land, before any compose has happened. Only stories scored 85 or above and not flagged sensitive arrive as Draft and can still absorb new coverage; they too freeze at first compose, and never move again no matter how many more outlets file, how the article is edited, or what a reviewer decides.

So the score on an article you approved on Tuesday describes the coverage as it stood when you first pressed compose. It is a snapshot, not a live reading.

Nobody can edit it. There is no control anywhere in the product that sets, adjusts or overrides a confidence score.

What the score actually changes

Very little, and this is deliberate.

• It gates nothing. No story is blocked, held, fast-tracked or auto-approved because of its score. Every story — 99 or 40 — needs a human to compose it, submit it and approve it before it can leave the workspace. See Editorial review and approval.
• It does not affect your plan, quota or daily delivery allowance.
• It sets alert severity. A story with a score under 80, or one flagged sensitive, raises alerts at warning instead of info. Breaking stories are always critical.
• It decides which stories your alert rules look at. Rules are only run against stories ErmisAI scored 85 or above and did not flag sensitive, plus every story an editor approves. A low-scoring story never triggers a rule until somebody approves it. See Alert rules and notifications.
• It travels into the compose step as a bare number alongside the source material, with nothing attached explaining what it means.

The score is also included in the CMS and WordPress export, so a downstream system can read it. Conflicts, the sensitive flag and claim provenance are not exported. See Getting approved stories out.

Where a story's sources appear

Three surfaces, showing three different things.

The feed card shows a source count and the favicon of the primary outlet. No outlet is named in text, and no conflict or sensitive marker appears on the card at all.

The Reporting brief in the story workspace shows the source story, the source count, the angle count, an amber count of open issues if there are any, the angles to preserve, the open issues themselves, and a collapsed Source attribution list of outlet names.

The Editorial review screen shows the claim provenance panel, described below.

No per-story screen shows a source's tier, its URL, or when it published. If you need to know what an outlet actually said, open the outlet.

Claim provenance, and what the three badges mean

The Claim provenance panel appears on the Editorial review screen only — the screen you reach through Open review on the Editorial board. It is not in the story workspace, so the person writing the article never sees it. Owners and Admins are the only workspace roles that can open it, and only while the article is in review: once it is approved or rejected the Editorial review screen is no longer reachable for that story.

It lists the first eight paragraphs of the article, badges each one, and lists one or two outlet names underneath. The badges mean exactly this:

That is the whole rule. None of the three badges means a claim was checked against anything. Every paragraph on a two-source story reads Supported, whether or not any source supports it.

Three more things the panel will not tell you unless you know:

• One conflict makes every paragraph "Disputed". Any conflict at all turns every row's badge red. The conflict notes are then handed out by position — note 1 to paragraph 1, note 2 to paragraph 2, and so on — and every paragraph past the end of the list repeats the first note. Nothing links a note to the paragraph it actually belongs to.
• The outlet names under each paragraph are assigned by position, working round the list in order. Paragraph three gets the third and fourth names on the list. There is no link between a sentence and the outlet that reported it.
• Only the first eight paragraphs get rows at all. A longer article's later paragraphs have no entry, and nothing on screen says so.

Empty state: "No claim-level provenance available."

The panel is recalculated from whatever the article says at the moment you open it, so it follows your edits. On a story nobody has composed yet, it dutifully analyses the placeholder text.

Read the panel as a prompt to go and check. Never as a check that has been done.

Conflicts and open issues

When ErmisAI writes a source story it is asked to note points where the outlets reported the event differently, and to attribute each side. Those notes are what you see as open issues in the story workspace and Conflict signal on the Editorial review screen. There can be up to eight, each a short line of free text.

Two limits to hold in mind.

There is no cross-outlet comparison. Nothing in ErmisAI compares the numbers, names or dates in one report against another. The conflict notes are what ErmisAI happened to notice while reading the source material, written as free text. They carry no structure — no "outlet A said X, outlet B said Y" fields, and no record of which source took which side.

No conflicts listed does not mean no conflict exists. An empty list is what you get whenever ErmisAI judges the sources to agree. If it missed a discrepancy, nothing else in the product will catch it, and the story will show no open issues and a clean provenance panel. Silence here is not evidence.

Reading a draft properly

What the product supports, in the order it is worth doing:

And what to ignore: a Verified signal badge as a reason to skip any of the above. ErmisAI's published AI use policy says the same thing in its own words — customers are expected to review attribution context, conflicts, confidence signals and any sensitive flags before publication. The full text is covered in Sensitive coverage and editorial limits.