Why I started benchmarking AI on payments work

Public AI benchmarks such as Vals AI and Artificial Analysis cover finance at a general level. From what I can tell, none drills into payments and finance operations. Yet almost every finance and payments vendor now claims it has an AI agent.

Hence I started ScalePoint Research. It benchmarks AI models on payments and finance tasks and scores them on the money they get wrong, with the cost per run alongside.

My starting assumption is that the right model depends on the task. Simple tasks shouldn't need an expensive frontier model. Tasks that need judgement should, and putting a cheap model on them adds the cost of checking and fixing its work. The first two tests point that way.

Test 1, reading card statements, is mostly extraction, and Grok 4.7 matched Claude Opus 5.5 with almost no fees wrong, at under a third of the price: 4.6 cents a statement against 15.5 cents.

Test 2, reconciling settlements, needs judgement. Grok 4.7 still did well but missed $162 a run, against $13 for Opus. The cheapest models fell much further behind: GLM 5.3 missed $1,394 a run and DeepSeek V4.1 Flash $965.

Of course, some steps, such as sums and matching on amount and reference, don't need a model at all and are cheaper and safer done by simple rules.