What Happens When the AI Reads an Invoice Wrong?
Every Invoice capture demo reaches the same moment. Someone in the room, usually the AP lead who will end up owning this, asks the only question they care about: what happens when it gets a number wrong?
Fair question, and the honest answer is that it will get numbers wrong sometimes. No extraction model reads every vendor layout perfectly. What matters is what the system does with the documents where it isn’t sure, and what catches the ones where it’s sure and still wrong.
The first gate: confidence scores, field by field
Invoice capture for Dynamics 365 Finance runs each document through an AI document extraction model, which returns the extracted value along with a confidence score for every field.
Those scores get checked against thresholds you define in the configuration group. A score below the threshold raises a warning or an error depending on how it’s configured. Errors hold the invoice for review. Everything that passes transfers to D365 F&O without anyone touching it.
The common mistake is setting one global threshold. A wrong vendor account sends money to the wrong company. A wrong line description costs nothing and is obvious to whoever reads the posting later.
Thresholds should be tiered by consequence: strict on vendor account, invoice number, total, currency and date, looser on descriptive fields where an error is visible and cheap to fix.
Confident and still wrong
A model can be wrong and confident at the same time, and it usually happens when a layout looks familiar. An invoice carrying both a gross and a net total in the spot where the total normally sits.
A credit note formatted like an invoice. A date the vendor wrote day-first that gets read month-first. Confidence describes how cleanly the document matched what the model expected to find, not whether the number is correct.
That’s why threshold tuning on its own isn’t the whole control.
The checks that catch confident errors are deterministic rather than statistical: total sales tax against the calculated tax, currency code against the vendor default, line quantities and units of measure against the purchase order, and the duplicate check on vendor plus invoice number.
These run after extraction and they don’t care how sure the model was.
What the reviewer actually sees
Held invoices open in the side-by-side viewer, document image on one side and extracted fields on the other, with the flagged fields marked. The clerk confirms or corrects, completes the review, and the invoice transfers.
The design goal is that the clerk verifies instead of re-keying. A review that requires reading the whole document is barely faster than manual entry. A review that surfaces three flagged fields and points at where they sit on the page takes seconds.
The part most teams assume wrong
Recognition quality is not fixed at go-live, and the improvement comes from several places rather than one.
Corrections made in the side-by-side viewer are captured as data: which field, what the model returned, what the clerk changed it to, for which vendor. That record is what tells you where the real problems are, and it feeds the work that follows.
For vendors whose layout the standard model reads badly, key-value pair mapping points the extraction at the right region of their document, so the next invoice from that vendor lands correctly.
Derivation rules cover legal entity and vendor account, which are the fields that decide whether an invoice can transfer at all. Thresholds themselves get adjusted per field once there’s enough history to justify the change.
And in those cases where a vendor’s document format is genuinely unusual and the volume of work justifies the cost, a dedicated document processing model can be trained for exactly those documents.
The pattern worth setting expectations around is that the gains are vendor by vendor rather than global. Your highest-volume suppliers are where the tuning pays back fastest.
Tuning it with numbers
Start strict. Then track two things per field and per vendor: how often a field gets flagged, and how often the flagged value turned out to be right anyway.
A field flagged constantly and corrected rarely means the threshold is too high, and you can lower it with evidence behind the decision. A field that passes silently and gets fixed later during matching, or by a vendor calling about a payment, means the threshold is too low.
Touchless rate is the number leadership will ask for. Correction rate by vendor is the number that tells you what to fix next.
The question worth asking at a demo
No extraction model is perfect, and any AP automation built on the assumption that one is will disappoint the team running it. What matters is everything sitting around the model.
Thresholds are set field by field, so the system won’t accept the values that carry real consequence on its own judgement. The validations run regardless of how confident the extraction was.
When an invoice does get held, the review screen lets a clerk settle it in seconds instead of reading the document from the top. And the mapping and training work keeps lifting recognition quality on the vendors you deal with most.
That’s also the thread running through everything we’ve covered on Invoice capture, from voiding and approval routing to three-way matching and line-heavy manufacturing invoices.
The automation is worth having because of how it handles the exceptions, not because the happy path is fast.