The ScalePoint Research Agenda

AI agents are showing up in payments and finance operations, both in vendor products and in tools firms build themselves. Either way, an AI model underneath does the reading and makes the calls. Model developers publish scores on maths, coding and general knowledge, and those say little about whether a model can tell a settlement that's late from one that's missing.

A wrong answer in this work has a price. It might be a fee overpaid for months, a chargeback lost on a missed deadline or a payment booked to the wrong customer. Hence ScalePoint Research tests models on the tasks payments and finance teams actually do, and measures accuracy in money.

The two questions

Every test answers two questions about each model.

1. How well does it do a given payments task, and what do its mistakes cost? Each test has a leaderboard that ranks models on the money they get wrong.
2. What does it cost to run at a firm's own volume, and where can it run with zero data retention? The cost per task sits next to the accuracy score, and the hosting page shows which routes keep no copy of the data.

The second question matters as much as the first for regulated firms. In the EU, firms work under GDPR and, for financial firms, DORA. UK firms work under UK GDPR and the operational resilience rules. Where a model runs, and whether the host keeps a copy of the files, can decide whether a model is usable at all.

What the research covers

The research covers payments and the finance operations around them. It starts from a map of 51 processes across six types of firm: merchants, platforms and marketplaces, acquirers and PSPs, issuers and banks, EMIs and cross-border firms, and stablecoin firms. Lending, wealth and insurance are outside it.

The 51 processes group into 11 clusters whose processes share the same steps. Each cluster gets one model test in the first series.

The process map shows each process and where the tests sit.

What a test looks like

Each test takes one task from one cluster and gives it to every model in the same way: the same files, the same plain prompt and the same output budget. Each model runs three times through OpenRouter on routes with zero data retention.

The files come from an invented market that ScalePoint Research builds in code, with planted errors that cost firms money in practice and decoys that look like errors and aren't. Every task has an exact right answer, so code marks each answer to the penny and no AI model does any marking.

The tests aim at the steps a product would actually give to a model. In real products, rules and machine learning do the matching, the sums and the lookups, and a model handles the reading and judgement they leave. How ScalePoint Research decides what to test explains that design rule, and the method page covers how the tests are built and scored.

What the public tests leave out

• Apps built on models. Tools such as Microsoft Copilot add their own instructions, so their results say more about the app than the model.
• Vendors' products. A vendor's software adds its own data handling and checks around a model. Testing one on a firm's own files is what a private evaluation is for.
• Multi-step agents. The first series tests a model answering in one reply. Agents that use tools and take several steps come later.

After the first 11 tests

Once all 11 tests are out, they form a benchmark for payments and finance operations, and ScalePoint Research will publish a full write-up of what they show. The leaderboards stay live after that. New flagship models are run as they come out, and other new models are added in quarterly batches.

The next stage goes deeper on where models can run, with more open-weight models, models a firm can run on its own hardware and EU-only hosting. After that come multi-step agent workflows, where a model works a case through several systems. Agent tests cost far more to build and run, so ScalePoint Research plans to run them with partners.

Partners can help design later tests under fixed rules. No vendor pays to be included, results aren't tuned, and a partner gets early sight of the cases and a say on whether they're realistic.

How the research is paid for

The tests, leaderboards and articles are free to use. No model maker pays for a test or sees the private cases. ScalePoint Research is paid for custom work, such as testing models or a vendor's product on a firm's own files, and advisory work on what to build or buy.