AI Economics From the Top Down: Five Questions About an AI Portfolio

There are two ways into the economics of AI. The first starts at the top. A CFO or CIO looks at everything the business spends on AI, across finance, operations, customer service and IT, and asks whether the money is well spent. Industry analysts and the large consultancies mostly work this way, and their clients are the people who sign off the budget.

The second starts at the bottom, with one task and one model. How often does the model get the answer wrong, what do those mistakes cost, and what does each run cost? ScalePoint Research works this way.

The two need each other. A portfolio view without task-level evidence rests on vendor claims and estimates. Task-level results without a portfolio view don't tell a CFO what to fund. This article sets out the five questions behind the top-down view, and where test results fit into each one.

1. Which AI use cases should we fund first?

The budget is finite and the list of candidate use cases is long. Every function has a pilot in mind and every vendor has a product to sell. Prioritising means ranking use cases on value, cost, risk and how ready the firm is to adopt them, then funding a few properly.

In payments and finance operations, the hard part is knowing which tasks a model can do well enough. Settlement reconciliation can look like a strong candidate on volume and staff cost, and still be a poor one if models miss too much money on it.

Where the tests fit. The process map lists 51 processes in 11 clusters, and each test shows how well current models handle one of them. For the processes tested so far, that turns "could AI do this?" into a measured answer.

2. What does AI cost in full?

The cost of an AI use case has two parts that behave differently. The one-off investment covers building or buying, integration, data work, testing and training staff. The recurring cost covers model fees, hosting, monitoring, maintenance and the staff who check the work. Business cases tend to show the one-off cost in detail and understate the recurring cost, which grows with volume and moves when model prices change.

Where the tests fit. Each test reports the cost per task for every model, and the test pages add the staff time to check each answer and the cost of the mistakes that get through. The unit economics of AI in payments and finance operations sets out the four parts of cost per case. The tests don't cover the one-off cost, which depends on a firm's own systems.

3. Is the value showing up?

A business case forecasts benefits: fewer hours, faster processing, fewer losses. Value realisation asks whether those benefits turned up once the use case went live, and how long payback took. Forecasts are often based on what a vendor or a pilot reported. Measured outcomes tend to come in lower, because adoption is partial, staff still check the work and exceptions take longer than planned.

Where the tests fit. A measured error rate and a measured cost per task give a forecast a firmer starting point than a vendor's claim. A business case built on them is also easier to check later against what the process delivers.

4. When should we build, buy, scale, redesign or stop?

Portfolio decisions don't end at approval. Each use case needs a decision on whether to build it, buy it or combine the two, and then, once it runs, whether to scale it, change it or stop it. Stopping is the hardest call, since the money already spent makes it tempting to carry on.

Where the tests fit. A vendor's product can be tested on a firm's own files against the models underneath it, which shows what the vendor's layer adds for its price. Buying payments and finance operations software with AI inside covers what the usual buying process misses. The leaderboards also show when a newer or cheaper model would change the answer, which is often the point to redesign a use case.

5. What moves cost and value at scale?

The same use case can pay back in one firm and lose money in another. The difference usually lies in how far staff adopt it, how the workflow changes around it, how good the output is, how much checking it needs and how much risk the firm will accept. Vendor pricing and commercial terms matter too. A price per task that looks small at pilot volume can be large at a million transactions a year, and terms on data use and switching decide how easy it is to move to another model or vendor later.

Where the tests fit. Output quality and the checking it needs are what the tests measure. The hosting page tracks the price of each model, its hosting routes and its data retention terms, which bear on both cost at scale and risk.

What changes in the research agenda

The research agenda now has two parts.

The testing programme. Every test answers three questions about each model.

1. How well does each model do a given payments task, and what do its mistakes cost?
2. What does each model cost per task at a firm's own volume, including the staff time to check its work?
3. Where can each model run with zero data retention?

The wider agenda. Four questions about the use case as a whole, drawn from the five above.

• Which AI use cases should a payments or finance team fund first?
• What does a use case cost in full, from the first build to running at scale?
• Should a firm build, buy or combine, and when should it scale, redesign or stop?
• Is the value showing up?

The wider agenda sits outside the testing programme, since the answers depend on a firm's own volumes, systems and staff. ScalePoint Research covers it in articles like this one, and for a given firm through custom research and advisory: private evaluations on a firm's own files, and the business case built on them.