Evaluation notes · September 2026

What an offline citation checker caught in 23 published papers

Peer review and copyediting are good at catching a lot of things, but not every in-text citation that doesn't match its reference. We ran our checker over published articles to find out what slips through.

What we did

We took open-access, peer-reviewed articles from psychology and cognition journals and extracted each paper's body text and reference list. Then we ran Cite Check, which matches every author–date citation to the reference list and every reference back to the text. It matches by surname and year, entirely in the browser.

Published papers are mostly correct, so every flag is either a real error or a mistake by our parser. That made the exercise a useful precision test, and it drove nine parser fixes before release. We don't name the papers or their authors: the aim is to show the kinds of error, not to embarrass anyone.

The errors that got through

About twelve findings were real errors in the published versions. The examples below are fictional and only show the pattern of each error type.

1. A misspelled co-author

This was the most common real error, and the hardest to see by eye: a doubled consonant dropped, a missing vowel, one letter swapped. We found three separate cases.

2. The wrong first author

Twice, the citation named a different person as first author: once for a multi-author "et al." work, and once for a two-author work. The reference is right there in the list; only the name in the text is wrong.

3. A citation with no reference

One work appears to be cited in the text with no entry in the reference list, so a reader can't find it.

4. Two works that shorten to the same "et al."

When two works share a first author and year but have different co-authors, both shorten to the same "Surname et al. (year)". A reader can't tell which one is meant. Style guides handle this by listing more authors, or adding a and b to the year. In the cases we found, neither had been done.

What the other flags were

We also excluded five papers: three because our own text extraction glued author names together, and two whose reference lists used a book format we didn't parse at the time. These are small, hand-judged numbers. They show that the errors exist, not what percentage of errors any tool catches.

What this means before you submit

• Spelling of co-authors is where eyes fail. A checker compares strings; a tired reader sees the name they expect.
• Every one of these errors survived authors, reviewers and production. A mechanical pass takes seconds and catches what attention misses.
• A reference manager doesn't prevent this if any citations were typed by hand or pasted from another draft.