Cited bookkeeping

The question

A bookkeeper posting a supplier invoice makes two claims at once. The first is that the cost belongs in a particular account. The second is that Swedish law puts it there. Models are already competent at the first claim, and are almost never asked for the second — which is the one an auditor needs.

This evaluation asks for both: the booking, and the paragraph it rests on.

Why the citation is the hard part

An account code can be guessed from a vendor name and still be right often enough to look impressive. A paragraph reference cannot. It either supports the booking or it does not, and a reader can check it in seconds.

Requiring the citation turns a classification score into something closer to an audit. A wrong booking defended with a confident citation is a different failure from a wrong booking that admits uncertainty, and the two should not score the same.

The invoice set

Real Swedish supplier invoices, each booked by a person before a model sees it. The account, the VAT treatment and the supporting paragraph are recorded separately, so a model can be right about the account and wrong about why.

The set is weighted toward the cases where a plausible booking is most often the wrong one, rather than toward the volume cases a simple vendor-to-account mapping already handles.

The harness

Each invoice is presented once, with no retry and no access to the human booking. The model returns an account, a VAT treatment and a citation. Grading is split: the booking is checked against the verified entry, and the citation is checked for whether it supports that entry — not for whether it matches the one the bookkeeper happened to write down.

That split matters. Two paragraphs can both justify the same posting, and a harness that only accepts the recorded one measures agreement with a bookkeeper rather than correctness.

What gets reported

The harness has not been run at scale yet. Defining the measures before there are numbers to report is deliberate — it is the part of an evaluation most easily adjusted after the fact.

Booking accuracy

Share of invoices where the account and the VAT treatment both match the verified entry.

Citation support

Share of correct bookings where the cited paragraph actually supports the posting, judged independently of which paragraph the bookkeeper recorded.

Confident error rate

Share of wrong bookings returned with a citation and no stated uncertainty. The failure that costs the most to catch downstream.

Abstention

Share of invoices the model declines to book. Counted separately from errors, because declining is the correct answer for some invoices.

What we expect to be hard

Three things, in advance of the results. Costs that are only partly deductible, where the account is right and the proportion is not. Costs that belong to a period other than the invoice date. And reverse-charge services, where the invoice itself carries no VAT and the obligation sits with the buyer.

In each of these a confident, fluent, wrong answer is the likely failure, which is the reason for asking for the citation in the first place.

References

• SFS 1999:1078Bokföringslagen
• SFS 2023:200Mervärdesskattelagen
• BASBAS-kontoplanen, the Swedish standard chart of accounts