What Changes When an AI Agent Works the Queue

The ScalePoint Research map shows which processes suit an AI agent, and the reconciliation example shows how one works a case. Neither says what happens to the team, the controls and the numbers once an agent is working the queue every day. That's the operating model, and it's where a pilot that worked can stall on the way to rollout.

I'll keep using reconciliation breaks as the example. The same points apply to chargebacks, alert review and payment repairs.

Analysts become reviewers

Today an analyst opens a break, pulls the settlement file, searches the ledger, works out the cause and books the fix. With an agent, the analyst opens a break that's already been traced, valued and explained, with a drafted journal waiting. The job becomes checking the explanation against the records, approving, editing or rejecting it, and taking the cases the agent escalates.

That's a different skill. Reviewing well means spotting a plausible explanation that's wrong, which is harder than it sounds when most of them are right. People who know the process best make the best reviewers, so the analysts doing the work today are the right people to review it.

There's also a new job: owning the agent. Someone has to maintain its instructions, decide which tools it gets, keep a set of test cases and look at why it got things wrong. I'd put that role in the finance or operations team and give it a named owner. Left with IT, the agent drifts away from how the process actually runs.

Approval points are a design decision

Every agent needs a line between what it may finish on its own and what a person approves. Draw that line before the pilot, using three things:

• Amount. Below a threshold the firm sets, the agent's fix might go straight through. Above it, a person approves.
• Type of action. Some actions should always need a person whatever the amount, such as writing off a balance, sending a claim to a provider or contacting a customer.
• Sampling. Whatever goes through without a person gets sampled, for example a fixed share each week, and the results are reported.

Start strict and loosen the thresholds as the numbers come in. The record of what was sampled and what reviewers found is also what you'll show the second line and the auditors when you ask to loosen them.

The three lines of defence still apply

Payments firms already run three lines of defence, and an agent fits into them without a new framework.

• First line: the operations or finance team owns the agent and its results, the same way it owns the process today.
• Second line: risk and compliance set the limits, meaning what the agent may read, what it may change and where approvals sit. They also test it before go-live and review the sampling.
• Third line: internal audit checks that the controls work, using the agent's audit trail of every lookup, result and draft.

For FCA-regulated firms, the FCA has said it won't add AI-specific rules and will rely on its existing frameworks, including the Consumer Duty and the Senior Managers and Certification Regime (SM&CR). In practice, the senior manager responsible for the function stays accountable for what the agent does. What changes is what that senior manager needs to see each month, which is the next point.

The numbers you report change

Hours spent and cases closed no longer show how the process is doing. These are the measures I'd put in front of the CFO and the senior manager each month:

• Cost per case: model fees plus reviewer time, per case closed.
• Money got wrong: the value of fixes approved that turned out to be wrong, found through sampling and later corrections.
• Override rate: how often reviewers change or reject the agent's draft. A rising rate is the earliest sign that something has drifted.
• Escalation rate: the share of cases the agent hands to a person.
• Time to close: from break to booked fix.

Set the baseline before the pilot. Without today's cost per case and time to close, nobody can say whether the agent improved anything.

Plan for changing model or vendor

Models change every few months, and vendors reprice. The protection is to keep three things in-house whichever way you buy or build: the instructions that describe how your process works, the list of tools and data the agent may use, and a test set of past cases with known answers. With those, you can rerun the test set on a new model or a new vendor and compare the results in money before you switch. Without them, every change means rebuilding the instructions and tests from scratch, which weakens your position at renewal.

What happens to headcount

I'd be careful promising cuts. The time an agent frees up shows first as backlog cleared and volume absorbed without hiring. Whether that becomes a smaller team is a decision to make in advance, and to tell the team about before the pilot starts. If people think the pilot is about cutting their jobs, that worry will shape how they review the agent's work.

In post-merger work I've merged back-office teams across operations and compliance, and the same rule held there: decide the target roles and the sign-offs first, then move the work.

What I'd put in place before the pilot

• A named owner for the agent in the finance or operations team.
• Approval points by amount and by type of action, agreed with the second line.
• Today's baseline: cost per case, time to close and the error rate.
• A test set of past cases with known answers, kept by you.
• A clear message to the team about what their work becomes.