Large language models — ChatGPT, Claude, Gemini and their peers — don't actually "know" facts; they predict plausible text. And sometimes the most plausible answer is an invented one: a line item that isn't on the invoice, a "rounded" total, an HS code that appears nowhere in the document. That's a hallucination. For a casual chatbot it's a minor glitch; for accounting and import operations it's real money — a wrong amount on a payment, an error in a customs declaration, a fine at the border.
Why AI makes things up — and why it's not a bug someone will fix soon
Hallucinations are a property of the technology itself, not a flaw in one particular model. A model is trained to always produce an answer, even when the data isn't there. If a digit on a scanned invoice is smudged, it won't say "I can't read this" — it will fill in the most likely value. You can't eliminate this completely. What you can do is design your workflows so an invented number never makes it into a payment.
Where hallucinations hit an importer's money
- Invoice amounts and quantities. The model "completes" a poor scan and turns 1,480 into 14,800 — confidently, with no warning.
- HS codes and duty rates. The AI names a code that looks right but doesn't exist anywhere in your documents.
- Supplier terms. Summarizing a long email, the model can "remember" a discount or a deadline that was never offered.
- Bank details. A mixed-up account number in a response means a payment sent to the wrong place.
Five workflow rules that keep AI from inventing a total
1. Extract, don't summarize
When working with documents, frame the task as "extract these fields from this file," not "tell me what's in the invoice." An extracted field can be verified by a machine; a free-form summary can't.
2. A source behind every number
Every figure must come with a pointer to where it was taken from: the file, the page, the line. If the system can't show the source, the number isn't accepted. That's the essence of the RAG approach: the answer is grounded in your document, not in the model's "memory."
3. Arithmetic checks run by code, not by AI
Quantity times price equals the line total; line totals add up to the invoice total; the invoice total matches the proforma. These checks are performed by plain program code, which is incapable of making things up. If anything doesn't reconcile, the document goes to a human.
4. A human at every money decision
The AI prepares, a human approves. A payment, a customs declaration, an order to a supplier — always with explicit confirmation. Raise autonomy gradually: at first the AI produces drafts only — a payment draft, never the payment itself — later it takes over the routine, but money still gets your sign-off.
5. A log and an honest "not sure"
Every extraction is logged: which document, what was extracted, which checks it passed. If the system isn't confident, it must say so explicitly and stop — not paper over the gap with a guess.
A simple test for your workflow
Take a real invoice, corrupt one digit in it, and run it through your AI workflow. A good workflow catches the discrepancy at the reconciliation step or hands the document to a human with a flag. A bad one lets it through silently. The test takes ten minutes and costs far less than your first real payment incident.
The takeaway: hallucinations aren't cured by a "smarter model" — they're cured by workflow architecture. Extraction instead of summarizing, a source behind every number, checks run by code, a human on every payment, a log of every operation. For more practical playbooks, see our articles and FAQ, and to estimate how much time manual document checking eats up, try the calculator.
These are exactly the principles HORUVIA is built on: every figure pulled from supplier emails and invoices is verified and keeps a link to its source document before it ever lands in your reports.