How Many Steward Reviews Will a Bank's Customer Clean-Up Create? Sizing the Review Queue Before You Buy (2026)
TL;DR: More than a demo suggests, and only your own data can say how many. In our illustrative bank, two million customer records create about 40,000 steward cases, roughly 117 reviewer-weeks. Count the pairs between your auto-link and auto-reject lines on a grouped sample, collapse them into cases, time real decisions and divide by productive hours.
Book a Demo · Try Tilores Studio (Free)
On this page
- Why is the review queue the cost that decides a customer clean-up?
- Why can no vendor quote your review rate in advance?
- How do you size the review queue from your own data before you buy?
- What does the count look like for a bank with two million customer records?
- How do you shrink the queue without buying wrong merges?
- What happens to the queue once the back book is cleared?
- What should a bank take into the PoC and the vendor conversation?
- Frequently asked questions
- Sources
A vendor demonstration ends on one merge screen: two customer records side by side, a score, a steward confirming the link in seconds. Then whoever owns the budget asks how many of those screens the team will clear before the new customer master can go live.
The demonstration cannot answer, because the number belongs to your data, not to the product. For an invented bank with two million records it comes to about 40,000 decisions, more than two reviewer-years of work.
We think that queue, more than the licence, decides whether a clean-up gets funded. So count it on your own records before you choose a vendor, with arithmetic any finance reviewer can rerun, instead of discovering it after go-live.
Why is the review queue the cost that decides a customer clean-up?
Every pair the matcher cannot settle on its own becomes a decision for a person, and those decisions, not the record count, set the staffing line. Fellegi and Sunter set out three decisions in their 1969 record linkage theory: link, do not link, and hold for a person. The US Census Bureau calls the held band the clerical review region, and the UK Office for National Statistics confirms pairs between its thresholds by clerical review.
A steward review is one person resolving one held item, the outcome Fellegi and Sunter flagged as needing expensive manual work. Everything else links or stays apart unseen, so the cost of the clean-up sits in the width of the middle band. In the invented bank below, that band is about 40,000 cases.
Why can no vendor quote your review rate in advance?
The width of the band follows the completeness and consistency of your source data, which no vendor has seen. One Tasmanian linkage unit reported clerically reviewing 6% and 8% of records for two clean hospital datasets, 30% for perinatal data and 78% for a small ambulance cohort, the last two with incomplete name and address fields: a thirteenfold spread inside one unit. So no review rate belongs in a sales conversation for any product, ours included.
A short queue proves little either. UK Government guidance on linkage quality notes that pairs with substantial missing or inconsistent data are hidden among the non-matches and never submitted for review. A small number can mean a clean database or a matcher that never surfaced the hard pairs, so measure missed links against labelled examples as a separate job. Our review queue design guide covers the reviewer’s side.
So the rate has to be measured, and it can be, before anything is bought.
How do you size the review queue from your own data before you buy?
Take one sample of your own records, drawn so that likely duplicates stay together, and turn it into reviewer-weeks in six steps.
- Choose the unit of work: a case, not a pair. A reviewer decides about a customer, not every pair: three records of one person form three pairs but need one or two decisions. Agree the definition with operations and record the pairs per case in the sample. The example below uses 1.5.
- Draw the sample by group, not by row. A random 5% of rows keeps both records of a pair only 0.25% of the time (0.05 multiplied by 0.05). Group records that could be duplicates instead, for example every record sharing a postcode area and surname initial, and draw 5% of the groups, so each pair arrives whole. Duplicates split across groups are missed, so read the result as a floor and repeat the draw; if the counts differ by more than a fifth, enlarge it. A 5% draw from two million records is 100,000 records, the free local limit of our Tilores Studio.
- Count the pairs in each band by rule. This step needs a matcher, and ours does the counting. Our match rules either link two records or do not, and each link carries the ID of its rule. Our search rules are separate: a search returns candidates with a closeness score, hitScore, between 0 and 1, without creating a link (API documentation). From those parts a bank can express three bands: strict rules that link, a wider search rule set that surfaces candidates for a person, and everything else. Count rule-based edges per rule ID for the auto-link band, and ignore our STATIC edges, which join any records submitted together whatever the rules say. Count the review band from search hits instead, because a search links nothing: search the sample with the wider rule set, and the pairs it finds that no match rule linked are your review band. Each hit carries the ID of the rule that found it, so the band can be counted per rule. The bands are the bank’s own counting method, built from these parts, not a built-in feature.
- Scale the counts and collapse pairs into cases. Multiply each count by the inverse of the sampling fraction (20 for a 5% draw), then divide review pairs by pairs per case.
- Time real decisions on the screen you will use. Have the future reviewers clear 100 sampled cases on the evidence layout they will use in production, and record the median minutes. Reviewers’ decisions depend on which identifiers they can see, so a spreadsheet test does not predict a purpose-built screen.
- Convert cases into reviewer-weeks and add a second look. Multiply cases by minutes and add a second review of a sample of decisions: the ICO’s human review guidance for AI systems expects sampling, a log of overrides and a manageable caseload. Then divide the hours by productive review hours per week.
Here is what those six steps produce for one bank.
New posts on entity resolution, identity data and AI. No spam, unsubscribe any time.
What does the count look like for a bank with two million customer records?
For an invented bank with two million records, the six steps produce about 40,000 cases and 117 reviewer-weeks: roughly fifteen weeks for a team of eight. The bank and every input are illustrative assumptions, not our result.
Its data governance lead has records in four source systems, a sponsor who liked the demo, and a draft business case with no line for reviewers. Three weeks before the steering committee, the lead draws 5% of the groups and finds a quarter of the pairs above the review floor in the band. Operations clocks a hundred cases at four minutes each. The business case now has a reviewer line whose every input the committee can challenge.
Illustrative review queue sizing for an invented bank
| Step | Value | Basis |
|---|---|---|
| Customer records across four source systems | 2,000,000 | Assumption |
| Sample: 5% of groups | 100,000 records | Assumption |
| Pairs above the review floor on the sample | 12,000 | Assumed sample count |
| Of which above the auto-link line | 9,000 | Assumed sample count |
| Of which in the review band | 3,000 (25%) | Assumed sample count |
| Review pairs for the whole base (multiply by 20) | 60,000 | Arithmetic |
| Pairs per case | 1.5 | Assumption |
| Review cases | 40,000 | 60,000 / 1.5 |
| Minutes per case | 4 | Assumption, to be timed |
| Review hours | 2,667 | 40,000 x 4 / 60 |
| Second look at 10% of cases | 267 hours | Assumption |
| Total hours | 2,933 | Arithmetic |
| Productive review hours per reviewer-week | 25 | Assumption |
| Reviewer-weeks | 117 | 2,933 / 25; about 2.5 reviewer-years at 46 working weeks a year |
| Elapsed time with eight reviewers | About 15 weeks | 117 / 8 |

Which inputs move the answer most?
The review band share and the minutes per case; the grid holds the rest at the values above.
Reviewer-weeks for the illustrative bank under different assumptions
| Review band share of pairs above the floor | 2 minutes per case | 4 minutes per case | 8 minutes per case |
|---|---|---|---|
| 15% (24,000 cases) | 35 reviewer-weeks | 70 reviewer-weeks | 141 reviewer-weeks |
| 25% (40,000 cases) | 59 reviewer-weeks | 117 reviewer-weeks | 235 reviewer-weeks |
| 35% (56,000 cases) | 82 reviewer-weeks | 164 reviewer-weeks | 329 reviewer-weeks |
A spread from 35 to 329 reviewer-weeks is the earlier point about quoted rates, now in money. A number that size also invites the obvious response: make the queue smaller.
How do you shrink the queue without buying wrong merges?
Fix the data before the decision, and leave the lines alone until you have checked recall. Each lever below has a check that stops it trading reviewer time for errors.
Levers for a smaller review queue
| Lever | Assumed effect on the illustrative band | What to check first |
|---|---|---|
| Normalise names, addresses and dates before scoring | Removes 20% of review pairs that differ only in format | No true match is lost, and originals stay on the record. Better scoring is a separate route: Palfy and colleagues cut the volume sent to clerical review by improving name similarity scores. |
| Add consistency rules, such as a date of birth that must agree across an entity | Removes 10% of the remaining pairs by turning a class of them into non-links | Pairs with a missing date of birth stay in the queue, because a missing value can neither confirm nor block. See our same-name consistency rules guide. |
| Raise the lower line of the band | Moves pairs from review to separate | Recall on labelled examples. Missed links never reach the queue, so this lever hides its own cost. |
Apply the first two in sequence and the illustrative queue falls from 117 to about 84 reviewer-weeks (117 multiplied by 0.8, then by 0.9). The tempting shortcut is lowering the auto-link line so more pairs merge unreviewed. The saving returns as wrong merges, which put one person’s data inside another’s profile and can become a reportable personal data incident.
Should the queue itself be sampled to save time?
Sample it to measure quality, never to skip decisions. In a study by NHS England and UCL researchers on 50,000 records and about 478,000 candidate pairs, reviewing about 7% of pairs instead of about 23% degraded the estimate: the global linkage error estimate rose from an accurate 4.2% to 9.3%, with the largest band-level errors in the mid-score bands.
What happens to the queue once the back book is cleared?
It shrinks from a wave to a trickle, and the trickle can be measured the same way before go-live. Staff the wave as a project and the trickle with a standing team.
In the illustrative bank, 30,000 new or changed records a month with an assumed 1.2% needing a case produce 360 cases, about 26 hours including the second look, or roughly one reviewer-week. Measure the rate by replaying a month of past arrivals through our submitWithPreview with dryRun set to true, which returns the entities the records would form, and the rules that would link them, without storing anything. Submit the records one at a time, since we join records sent together with STATIC edges.
Do sanctions, PEP and adverse-media alerts share this queue?
No. Screening alerts form a second queue with its own thresholds and reviewers, sized by the same arithmetic. OpenSanctions’ matching guidance says a result is not immediately cause for alarm, because the PEP data contains many people with common names, and advises tuning the alerting threshold to the use case before setting up human review. Our resolution layer sits next to screening without replacing it, so the screening system receives one customer with its name variants and their provenance.
Wave, trickle and screening queue together make the staffing line, and the last step is to use it in the buying decision.
What should a bank take into the PoC and the vendor conversation?
Take the count itself. The manual review rate and time per review are the staffing lines a finance committee asks about, and a champion with no budget line for entity resolution needs them before the committee meets.
We agree precision and recall targets with the bank before testing starts, work from a sample extract, run at least two refinement cycles, and hand over the rule configuration in an exportable form, so the bank can rerun this count. Our guides to agreeing matching rules before a PoC and the seven PoC inputs cover both. We are not a full MDM suite, and a bank can keep its mastered data and its stewardship process in its own systems.
We are not the right answer in every case. If every customer already carries one trusted identifier and a key check and a short script clear the rare duplicates, the queue fits in a spreadsheet and an engine adds cost without removing work. That stops being true once names, addresses and dates vary, because the script’s misses are invisible.
Ways to handle a customer clean-up queue
| Option | Fits when | Falls short when |
|---|---|---|
| Case-management or MDM workflow the bank already owns | The bank needs assignment and escalation built in | It routes only what the matcher surfaces |
| In-house matcher and queue | A dedicated matching team has time to build and maintain it | Nobody can show a supervisor why two records linked |
| Resolution engine with exportable rules (our approach) | The bank wants band counts from its own data before signing | The bank still needs reviewers and an owner for the decision policy |
Whichever route you take, put six questions to every vendor in writing:
- Will you count our bands on a sample of our own records before we sign, and show the count by rule?
- What is a review case in your tool: a pair, a cluster or a customer?
- Which part of the band comes from missing data, and so belongs in data repair instead of a reviewer’s queue?
- Is every decision logged and reversible, with the reason and the rule version that raised it?
- Can we export the rule configuration and rerun the sizing when rules change?
- Who carries the reviewer cost in the price, and what happens to a held pair nobody reviews?
A vendor that answers with counts from your sample has sized your queue. One that answers with a screen has shown you the demo again.
Frequently asked questions
How many manual reviews will a bank’s customer clean-up create?
Count it on your own data: the pairs between your auto-link and auto-reject lines on a grouped sample, divided by pairs per case. In our illustrative bank, two million records create about 40,000 cases. Published reviewed shares run from 6% of records for clean data to 78% for poor data.
How long does one steward review take?
Time it with the people and the screen you will use. The worked example assumes four minutes, and a range of two to eight minutes moves the total fourfold.
Can a bank shrink the queue by merging more pairs automatically?
It can, but the cost moves from reviewer time to wrong merges, which put one person’s data inside another’s profile. Cut the queue with better normalisation and consistency rules first, and check recall on labelled examples before moving any line.
How can a bank count review bands in our Tilores Studio before buying?
Load a sample into our Tilores Studio, which runs locally on up to 100,000 records, count rule-based edges per rule ID for the auto-link band, ignoring STATIC edges, and count the hits of our wider search rule set that no match rule linked as the review band. Each hit carries a hitScore and the ID of the rule that found it. For arriving records, our submitWithPreview with dryRun set to true shows what each would join before anything is stored.
Sources
- Tilores documentation: API (submit, submitWithPreview, search and hitScore), Tilores.
- Tilores documentation: Rules Configuration (edges, rule sets and search), Tilores.
- Download Tilores Studio, Tilores.
- Entity Resolution Review Queues: How to Design Human Decisions for Ambiguous Matches in 2026, Tilores.
- How a Bank and Its Entity Resolution Vendor Agree on Matching Rules Before a PoC, Tilores.
- Entity Resolution Proof of Concept: 7 PoC Inputs a Bank Needs in 2026, Tilores.
- Can Entity Resolution Tell Two Customers With the Same Name Apart? Consistency Rules for Banks, Tilores.
- Is a Wrong Customer Match a Reportable GDPR Incident, and What Must a Bank Document?, Tilores.
- A Fundable Customer Master Business Case for a Smaller Bank, Tilores.
- A Theory for Record Linkage, Ivan P. Fellegi and Alan B. Sunter, Journal of the American Statistical Association, Vol. 64, No. 328 (1969).
- Overview of Record Linkage and Current Research Directions, William E. Winkler, US Census Bureau Research Report Series (2006).
- Developing standard tools for data linkage, Office for National Statistics working paper (February 2021).
- Quality assessment in data linkage, James Doidge, Peter Christen and Katie Harron, UK Government, Joined up data in government: the future of data linking methods.
- The impact of administrative dataset characteristics on clerical review: A Tasmanian case study, Nadine Wiggins and Brian Stokes, International Journal of Population Data Science, Vol. 1, No. 1 (2017).
- Improving name comparison similarity scores to reduce number of records for clerical review, Miro Palfy, Stacy Ann Vasquez and Alexandre Franco Garcia, International Journal of Population Data Science, Vol. 3, No. 4 (2018).
- Designing An Ambiguity-Aware Clerical Review Strategy, Joseph Lam and colleagues, International Journal of Population Data Science, Vol. 11, No. 5 (2026).
- Diversity of record pairs and mental models in clerical review: What reviewers use to decide matches, Yashithi Dharmawimala and colleagues, International Journal of Population Data Science, Vol. 11, No. 5 (2026).
- Human review, Artificial intelligence toolkit, Information Commissioner’s Office.
- The matching API, OpenSanctions.
See what resolved entity data does for your business — and your AI.