SAP B1 blog

This series continuously examines individual AI basic terms and methods such as the process reward model. previous episode has addressed a sober finding: generating is cheap, verification is the actual bottleneck. A system can generate thousands of candidate answers in a short space of time; the real work only begins afterwards, when the correct answer must be found amongst these candidates. This episode delves deeper into this verification step. And it poses the obvious follow-up question: on what basis does a system actually decide which of several generated answers is the correct one? Is merely looking at the end result sufficient, or does it require a closer look at the path taken to get there?

Read more: Process Reward Models: Warum ein richtiges Ergebnis noch keinen richtigen Weg beweist

The first episode of this series used the approval process as an example, this episode chooses a second, related example.

When the final result of a calculation, an invoice or a painstaking account reconciliation is correct, the matter is usually considered settled in a medium-sized business. One closes the laptop with a sigh of relief and turns one's attention to the next pile.

That is the ideal world. In lived practice, however, a correct final figure proves absolutely nothing about how one got there. It is not uncommon for two major calculation errors in the middle of a calculation to simply cancel each other out, leaving the correct sum in the box at the end anyway. Anyone who checks only the final result systematically overlooks such errors. This applies to mental arithmetic at school just as much as to an artificial intelligence (AI) system.

The danger of the „convincingly false answer“

In verification research, a specific term has become established for this tricky case: the „convincing wrong answer“. This refers to a solution which, although it produces the correct figure at the end, contains an error in its calculation at least once. A traditional verification mechanism, which only checks the final result at the end of the working, cannot distinguish these cases from genuine correct answers. It considers both to be correct because it checks only the final result.

For companies and CFOs considering AI-supported analyses of their own financial data, this is far more than an academic subtlety. In the end, an AI might name the correct metric without having taken the correct path to get there. Whether one can still trust the result on the next, slightly different task is decided precisely by this. Anyone who relies solely on the final result is relying on chance.

Two ways of quality control: goal or process

The first generation of AI assessment mechanisms, known as outcome-based reward models, evaluate only the final result. The process is quickly explained: a trained language model generates many different attempted solutions to a task all at once – in some cases, several hundred. Each of these attempts is automatically compared with the known, correct final result, is assigned a simple label such as „correct“ or „incorrect“, and a dedicated verifier learns to assess the probability of success. When applied, the system then selects the attempt with the highest final result score.

Be that as it may, this method systematically overlooks „convincing wrong answers“, as it only assesses performance at a single point. This is where modern process-based reward models come in. Instead of looking solely at the target, they evaluate each individual step of a solution path separately. To this end, human evaluators rate each intermediate step as correct, incorrect or neutral. To save effort, priority is given to checking precisely those solutions where the final result is correct, but the path to it contains errors. The model learns from these step labels. Its overall assessment is derived from the combination of all individual step assessments, not from a single value at the end.

The effect is noticeable: according to the research paper, alongside better generalisation to new tasks, increases „Let’s Verify Step by Step“ data efficiency per annotation by a factor of 2.6. Any incorrect intermediate step is immediately spotted where it occurs. This dataset from OpenAI, comprising 8,500 primary school maths problems (GSM8K named) from 2021 demonstrates this effectiveness in multi-step computing.

What procurement has been demonstrating for a long time along the audit trail

Interestingly, SAP Business One a function that has long since embodied this exact difference in everyday business operations—completely without Artificial Intelligence. We are talking about the document chain in purchasing, which leads from the framework agreement through the purchase requisition, the supplier enquiry and the order to the goods receipt and the incoming invoice.

At every single chain step, the system compares the planned quantity minus the processed quantity in the „Open quantity“ field. The overall document remains „Open“ as long as even a single item has not yet been fully processed. Each step thus delivers its own verifiable intermediate result.

This contrasts with the credit limit check carried out when creating a sales document. Here, SAP Business One adds together the outstanding balance, any outstanding deliveries and the value of the new document. At the end of the document creation process, the system compares this total with the defined Credit limit. This check – much like an outcome-based reward model – looks only at a single value at the end. It does not take into account how this value is made up of the individual items on the document. Both mechanisms are currently purely rule-based and do not learn from decisions.

Thought experiments SAP B1 to Process Reward Models

In theory, an extension that goes beyond a simple quantity comparison within the document chain is conceivable. A trained criterion could also analyse patterns across many past document chains and check for combinations of supplier, item and deviation metric that stood out in the past, even though the quantities were technically correct. Such a criterion would complement, rather than replace, the strict quantity rules.

Here are five practical scenarios from the ERP– and supply chain scenarios where the document chain is formally valid, but a pattern-based check reveals anomalies:

1. Systematically pushing tolerance limits to the maximum (Micro-Creep)

• Scenario: For a supplier of bulk goods or goods sold by the metre, the automatic quantity tolerance is ±5%.
• Pattern Over 40 Document chains Furthermore, for standard items, the supplier delivers within a range of exactly +4.8% to +4.9%.
• Findings: Technically, automatic approval applies to every individual booking. Overall, the supplier systematically exploits the tolerance for margin optimisation at the buyer's expense.

2. Hidden scrap despite receipt of identical quantities of goods

• Scenario: A machining company buys blanks. an order1,000 units, goods receipt: 1,000 units, purchase invoice: 1,000 units.
• Pattern When combining Supplier A and Material B Historically, in over 30 % of cases, manual confirmations of production scrap are recorded within 14 days of posting.
• Findings: The 3-way match is green. However, the statistical criterion is still blocking the immediate invoice approval and requires a re-confirmation from QA/the production line.

3. Artificial splitting of receipts to circumvent audit thresholds

• Scenario: An order of €50,000 requires a dual-control approval from the department head for individual invoice values starting from €20,000.
• Pattern A supplier provides for a single Goods Receipt simultaneously issued three partial invoices for €16,500, €17,000 and €16,500 respectively.
• Findings: Each individual invoice formally matches the partial delivery and remains below the review threshold. However, the overall pattern across the document chain signals a deliberate bypassing of approval hierarchies.

4. Opportunistic timing regarding upcoming price tiers

• Scenario: For item X, a contractually agreed price reduction of 15 % has been entered into the system at the start of the new quarter.
• Pattern Delivers two days before the deadline Supplier C indicate the maximum over-delivery tolerance for blanket orders – historically, the same supplier otherwise delivers this item strictly Just-in-time or with slight undercuts.
• Findings: The quantities match the open purchase order backlogs, but the timing leads to unnecessary capital commitment at legacy conditions.

5. Creeping rise in unstructured ancillary costs

• Scenario: The item positions match precisely in terms of quantity and price.
• Pattern With Supplier D do freight, packaging or small-order surcharges gradually appear on the invoices that were not explicitly maintained on the original purchase order, but individually remain below the document overall tolerance (e.g. €20)?.
• Findings: Isolated vouchers do not trigger a query case; significant, unplanned procurement ancillary costs accumulate at the item level.

The catch is that such a system also learns from past poor decisions. If a flawed combination was previously „dumped“ and waved through unexamined out of sheer time pressure, this shapes the learned criterion just as much. The system cannot distinguish on its own between correct historical decisions and those that were mistakenly waved through.

Keeping your eyes on the road remains crucial

In the end, this realisation remains: anyone who only checks the final result has not yet satisfied themselves that the path taken to get there was also correct. Whether a human, a rigid rule, or a trained model is watching makes little difference. This applies just as much to the reasoning paths of complex language models as it does to documentation in procurement. Without checking the path, a correct final result remains a product of chance – and in solid [business] one does not rely on that. Medium-sized companies better not, as is well known.