← Writing

Models · October 2026

Nine Skills for Testing AI Models in Payments and Finance Operations

S1 asked mostly for the two easiest skills, so it says little about which model to trust with harder work. Here is the full list, and where each test fits.

In brief

In S1, 34 AI models read UK card acquiring statements and had to list every fee with its amount and type. With the same plain prompt for every model, Grok 4.7 and Claude Opus 5.5 got almost every fee right, and models costing 1 to 3 cents a statement, such as Gemini 3.8 Flash and GLM 5.3, got under 0.5% of fees wrong.

That's a useful result for anyone reading statements at volume. It doesn't say much about which model to trust with harder work, because the task didn't ask for much. Reading a statement is mostly finding the lines and putting each one in the right box.

The nine skills

Payments and finance work asks a model for a mix of skills. ScalePoint Research uses nine to plan the tests, from the most mechanical to the ones that need the most judgement.

SkillWhat it means in payments workTested in
1. ExtractionFinding the data in a document, such as the fee lines and amounts on a statementS1, S2
2. ClassificationPutting each item under the right type, such as a non-UK card surcharge rather than a penaltyS1
3. Rule applicationApplying a contract term or scheme rule, such as checking a fee against the contracted rateLater tests
4. Reasoning across sourcesTying several documents together, such as a settlement file, the bank lines and the ledgerS2
5. Calculation set-upBuilding the right calculation, such as the blended rate on a pricing dealLater tests
6. JudgementDeciding what matters, such as which difference is a real error and which will clear by itselfLater tests
7. Long contextWorking across a lot of material at once, such as a month of filesS2
8. Tool use and planningLooking things up step by step in other systemsAgent tests, after the one-step tests
9. Staying within authorityNot changing what it isn’t allowed to changeAgent tests, after the one-step tests

The last two matter most for AI agents, which work a case step by step in a firm's own systems. ScalePoint Research is running a first round of one-step tests, where the model gets all the files at once and no tools, and plans to test agents end to end after that. Those two skills come in with the agent tests.

What S1 showed about the skills

Every usable model found the fee lines and read the amounts correctly, so extraction wasn't the problem. The mistakes were classification, mostly higher charges on non-UK and commercial cards filed as penalties. A payments analyst learns those calls on the job, and the models that made them correctly were the ones that stood out.

Two of the statements were faulty: one had a subtotal that didn't add up. Spotting that is a small piece of judgement, because the model has to check the statement's own sums rather than copy them out. Claude Opus 5.5, Grok 4.7, Gemini 3.8 Flash, GLM 5.3 and DeepSeek V4.1 Flash flagged both faulty statements. Mistral Large 3 and GPT-5.5 flagged neither.

What changes in S2

S2 gives each model a month of an invented merchant group's settlement files, bank lines, ledger and acquirer contract. The model has to find and value every difference. That adds reasoning across sources and long context. It also asks the model to leave timing differences alone, such as a card sale on 31 October that the acquirer settles on 2 November, which is a first step towards judgement.

Later tests move further down the list, to applying contract terms, setting up a pricing calculation and deciding which alerts are real.

Evidence so far

Each ScalePoint Research test adds a row to this table, with what that test shows for the point of this article.

TestWhat it shows
S1
Card statements
Extraction and classification. Every usable model read the amounts correctly, and the mistakes were classification.
S2
Reconciliation
Reasoning across sources and long context. Most cheaper models fell well behind.

2 tests published so far. A new row is added when the next test comes out. Full results are on the tests page.

What this means when choosing a model

A model that does well on a reading task has shown it can read. It hasn't shown it can apply a rule or make a call. Hence a business should test each model on the skills its own task needs, and a price gap measured on an easy task shouldn't be assumed to hold on a harder one.

Last reviewed 21 October 2026, with results up to S2.

Part of ScalePoint Research. The tests:

martin@scalepointpartners.com