I Replaced my Accountant by Claude

Before I get into the weeds, a bit of background. I’m by no means a professional accountant or bookkeeper. That said, I know a thing or two about accounting and bookkeeping. I did a financial oriented studies, I founded a company where I was responsible for finances for 8 years next to my other responsibilities and I still have 2 small business where I still do the bookkeeping myself. Also I have roughly 12 years of submitted annual reports and bookkeeping as a “golden set” to rely upon when making this agent more reliable.

Disclaimer; I know a thing or two about bookkeeping myself. I was responsible for the finances for 8 years in an earlier venture; also I did a financial study. So although I’m not an account, I kind know how to navigate in this area.

So what are we building? My goal was to build an agentic system that can process invoices, match invoices to bank transactions, flag any missing invoices, align the balance sheet accordingly and write a similar annual report as my accountant did for the past few years.

Building agents is usually

Digital twin

One of the core principles is that I wanted a digital twin of my bookkeeping, for 2 very specific reasons:

  • it allows me to dry-run changes
  • it allows me to swap out my actual bookkeeping software as the backend of this digital twin when I run LLM evals

Evals

The idea is to create task based benchmarks - not a single benchmark for the entire tool. Each vendor will have different quirks, different API abstractions. Something are really easy for an LLM at one vendor, while downloading a PDF might be really hard at another.

In our example Moneybird had some quirks around downloading PDFs and inspecting these. So, that is a good candidate for a tiny benchmark.

  • Create a tiny benchmark that gages how good a model is at extracting all details from an invoice. download 30 PDFs and receipt and control and load them into a notebook.
  • Updating journal posts based on different

Ideally you want to run benchmarks that you can feed initially with real live data, but are not dependent on the actual live environment. Because running benchmark against a live account would be a bad idea.

I build a tiny benchmark where each case was failed if either one of the following items was incorrectly generated by the agent.

  • company name
  • currency
  • amount
  • VAT %
  • VAT amount I also tracked the amount of tokens used, the total costs, the amount of tool calls executed and time consumed. I ran the full benchmark 3 times to get an proper variance score.

I used a remote sandbox environment to run the benchmark in full isolation with no way to cheat the actual scores.

All results are stored in the runs directory with a config, results.jsonl and summary.json file. The scripts in run.py and score.py before the actual scoring.

Just by asking “what do we need to do to submit the Q2 VAT submission”, it give a list of

  • unprocessed bank transactions
  • unprocessed purchase invoices
  • unprocessed receipts
  • journal entries that required attention

To reduce risk I created multiple API keys so I can isolate which agent does what. For example if an agent can update invoices I did not want it to also update the balance sheet.

The MCP server cannot link bank transactions to purchase invoices - so I tried the CLI, which is just a wrapper around the API. It is a pretty dangerous action, because X TODO - so, it’s not something i would recommend doing if you do not know how agents work and what the risks are.

It actually can also download unprocessed receipts, analyse the actual downloaded PDF or images an update the purchase receipt or invoice accordingly. This is a slow process however, and it consumes lots of tokens. Be aware of this when using Fable 5.

Quarterly update & VAT

Annual report

My accountant always asked for the same set of documents, so I also provided this to Claude when i wanted to

Skill

Connecting bank transactions to purchase invoices

moneybird-cli financial_mutations link_booking <mutation_id> \
  --booking_type SalesInvoice \
  --booking_id <invoice_id> \
  --price <amount> \
  --price_base <amount>
  1. first detect missing invoices or receipts by attach all bank transactions
  2. identity any unprocessed bank transactions; notify the user about each transaction, including details so the user can identify it clearly

Whats next?

I would not be surprised if more bookkeepers actually use this approach. The most dangerous thing I did use 1 api key for everything. I think it is more secure to use multiple api keys with different scopes for different use cases.

I want to see how Haiku 4.5 will perform exactly like this

I want to run this each month, so it emails me each month with any