Every checking tool has a dial, whether or not it shows you. Turn it one way and it reports more, catching more real findings and more noise. Turn it the other and it reports less, with fewer false alarms and more misses. Understand that trade and you can judge a tool's settings, and know which direction suits your task.
The two measures
| Measure | Question | When it matters |
|---|---|---|
| Precision | Of what it reported, how much was real | Daily use; a noisy tool is abandoned |
| Recall | Of what was real, how much did it report | Final checks; a miss reaches print |
What it looks like in fiction
- High recall, low precision: a name-similarity report that lists every pair of vaguely similar words. You'll find every variant, in a long list.
- High precision, low recall: a report listing only near-identical pairs. Everything it says is worth reading, and it misses the variant that differs by two letters.
- The final check wants recall, with the noise accepted, because a missed variant is permanent.
- The weekly check wants precision, because you will run it many times.
- Counts help both, by letting you sort rather than filter.
Point five is the practical resolution. A tool that reports generously but sorts by frequency gives you high recall with the noise pushed to the bottom, where it costs nothing.
Why narrow scope allows both
The trade is forced by ambiguity, and ambiguity comes from scope. A tool trying to judge whether a scene works has to be probabilistic; a tool counting occurrences of a string does not. That's why deterministic tools can be exact within their scope and why their scope is narrow.
The Novel Kit sits on both sides of that line, and it's worth separating the two halves. Counting has no dial to set: every term the rules pick up is listed, and its count and first appearance are the real ones, because there is nothing to decide.
Matching is different. Deciding that two spellings are one name is a judgement about strings, so the similar-name and alias report has a threshold like every other comparison tool, and ours is set tight. A single character's difference counts for names under six letters, two for names of ordinary length, and more only for very long ones. That buys precision at the cost of recall: a variant three edits from its parent is not paired, and short names never cross-match, which is the point of the six-letter rule. The counts printed beside each pair are what make the trade workable, because they let you sort a generous list rather than trust a short one.
So the honest version is that scope removes the trade from the counting reports and does not remove it from the matching ones. Anyone telling you their checker has no threshold is describing the counting half and forgetting the rest.
A practical next step
Decide which you need for your current pass. A final check before publication wants everything, noise included; a routine check wants a short list you will read.
Sources checked
- False positives and why they matter
- How fuzzy name matching works
- Deterministic versus probabilistic manuscript checks
Frequently asked questions
What do the terms mean?
Precision is how much of what a tool reports is real. Recall is how much of what is real the tool reports. Improving one usually costs the other.
Which matters more for a manuscript?
Recall, for a final consistency check, since a missed variant reaches print. Precision, for anything you will use repeatedly.
Can a tool have both?
Only where nothing is being judged. Counting occurrences of a string sets no threshold, so it can be complete. The moment a tool decides that two different spellings are the same name, a threshold exists and the trade comes back with it.