Sanctions Screening / Learning brief
Testing and tuning a screening system
Your notes
In simple terms / 01
What this means in plain language
A screening system must catch true sanctions matches without burying analysts in false alerts. This article explains how synthetic test cases and careful threshold tuning balance those aims, and why each change is documented and re-tested.
A screening system is only useful if it catches the entries it is meant to catch while raising a manageable number of false alerts. Testing checks the first half of that promise. Analysts feed the system synthetic cases — invented records built to look like real sanctioned entities, including misspellings, reordered names, and transliterations — and confirm that each one produces an alert. Tuning addresses the balance. The match threshold is the dial that decides how close a resemblance must be before the system flags it. Setting it higher reduces alerts but risks missing a genuine match written in an unusual way. Setting it lower catches more variants but produces more noise for analysts to clear. The right setting keeps detection strong while keeping the alert volume reviewable. Because either move changes how the control behaves, every adjustment is documented and the system is re-tested afterwards, so the institution can show that a change improved efficiency without weakening detection.
Complete lesson / 02
Understand the full idea, step by step
A screening system that never raises an alert looks efficient. It might also be catching nothing at all. You cannot tell which by watching the alert count alone — you have to feed the system cases whose right answer you already know, and see what it does with them. That is the difference between testing a control and trusting it.
What a good test pack covers
- Transliteration
- A surname carried across from another script
- Name order
- First and last name reversed
- Partial data
- A missing or incomplete date of birth
- Aliases
- A known alias rather than the primary name
- Misspellings
- Common spelling variants of a listed name
Recall and precision — how much the system catches, and how clean its alerts are
Recall is the share of true matches the system actually flags. Precision is the share of its alerts that turn out to be real. Raising the threshold improves precision — fewer, cleaner alerts — but lowers recall, because a genuine match written unusually may now fall below the line. Lowering it does the reverse. Tuning is the act of choosing where that line sits, and the controlling principle is that recall comes first: reduce noise, never suppress detection.
| Failure | What it is | Why it matters |
|---|---|---|
| False positive | An alert on an innocent party | Costs analyst time, but is caught in review |
| False negative | A genuine match the system never flags | Passes silently and defeats the control entirely |
| True sanctioned entities seeded in the test pack | 40 |
|---|---|
| Correctly flagged by the system | 38 |
| Recall (share of true matches caught) | 38 / 40 = 95% |
| Missed — genuine matches that did not fire | 2 false negatives |
These are synthetic test-pack counts, not live statistics. Two seeded matches slipping through is the signal that matters far more than the alert volume: a threshold change that lifts precision but drops even one true match below the line has weakened the control, whatever it does to the queue.
Test on both sides of the line
To set the threshold well, the team tests above it and below it. Above-the-line testing samples alerts that did fire, to see how many were plainly false — that measures precision. Below-the-line testing samples near-misses that scored just under the threshold, to confirm the system is not quietly discarding genuine matches. If a true match is found sitting below the line, that is strong evidence the threshold is too high, whatever raising it did to volume. Live customer records are never repurposed as test targets; the data is always synthetic and labelled as such.
WHAT IF — Below-the-line testing finds a genuine seeded match scoring just under the proposed threshold
What happens: The proposed change is rejected. A tuning move that lowers alert counts but also lowers detection is a failure, not an efficiency.
How it is handled: The finding is documented with the before-and-after test results, and the threshold is left where detection holds or moved only where evidence shows recall is preserved. Reducing noise so analysts can concentrate is the aim; suppressing alerts for their own sake is not.
COMMON CONFUSION
“A lower alert count after tuning proves the system got better.”
A lower count could mean cleaner alerts — or a hidden loss of coverage. The two look identical on a dashboard. Only the documented before-and-after test results tell them apart, which is why every tuning change is measured against a known-answer pack rather than judged by volume.
STRICTLY SPEAKING
Strictly speaking, a tuning change is never finished when the new number is entered. The rationale, the test results before and after, the approver, and the date are all recorded, and the configuration and the test pack are kept under version control so the institution can reconstruct exactly how the system behaved at any past moment. After any change, the full detection test is re-run — not just the part that prompted it — because fixing one gap can quietly reopen another. This regression discipline is what connects tuning back to governance.
FOR NOW, REMEMBER
- Detection testing uses synthetic, known-answer cases to prove the system flags the hard variations it must catch.
- Recall — catching true matches — comes first; a false negative defeats the control silently, unlike a false positive.
- Tuning tests both above and below the threshold; a true match found below the line means the threshold is too high.
- Every change is documented, version-controlled, and followed by a full regression test, so efficiency can be shown not to have cost detection.
TRY IT YOURSELF
After a proposed threshold rise, Meridian's below-the-line testing finds a seeded true match now scoring just under the new line, while the overall alert count drops by a fifth. What should the team conclude?
Testing and tuning assumed the target was a sanctions match — a hard legal stop. The next lesson turns to a different kind of hit that the same engine can raise: a politically exposed person, where the outcome is a closer look, not a block.
KEEP GOINGKey takeaways / 03
Three things to remember
- 01
Testing uses synthetic sanctioned records to confirm the system alerts on name variants and misspellings.
- 02
The match threshold trades alert volume against the risk of missing a genuine match.
- 03
Every tuning change is documented and re-tested so preserved detection can be demonstrated.
Practical use cases / 04
Where you would use this
A tuning team seeds synthetic test entities before go-live to confirm the system detects known variants.
An analyst samples alerts just below the threshold to check the line is not hiding true matches.
A model owner records each threshold change and its re-test results for later review by audit.
Worked example / 05
Put the idea into a real situation
Illustrative example: a fictional bank, Cedar Point Bank, tests a proposed threshold change on a synthetic list containing 200 invented sanctioned entities, such as an individual named Marcus Venn with three recorded spelling variants. At a threshold of 90 out of 100, the system detects 184 of the 200 test entities and produces 5,000 alerts a day. Lowering the threshold to 85 detects all 200 but raises daily alerts to 8,000. The tuning team keeps 85, because missing 16 of 200 known targets is unacceptable, and records that the extra 3,000 alerts are the accepted cost of full detection. The change, its test results, and its approval are stored together and scheduled for re-test after six months.
Evidence & review / 07
Evidence & review
Testing and threshold tuning of a name-screening system; the specific metrics engines expose and thresholds used vary by vendor and configuration.
What this brief simplifies: Recall and precision are introduced in a clean form; the test-pack counts are synthetic illustrations, not real performance figures. Real programmes use larger packs and more matching controls than a single threshold.
Sources for this brief2
- Market practice
Wolfsberg Group Sanctions Screening Guidance ↗ — The Wolfsberg Group · Screening testing and tuning
Wolfsberg guidance is industry market practice, not law; institutions vary in how they apply it.
- Simplified educational illustration
Payments Signal editorial teaching models — Payments Signal
Used wherever diagrams, scenarios, figures, or example values are didactic constructions rather than sourced facts; every such use carries a simplifications disclosure. All people, companies, banks, and list entries in examples are fictional.