Contents10
- 01What the test measures, and why it changes things
- 02The leaderboard: nine assignments out of ten fail
- 03Eight areas, from individuals to disputes
- 04What a file contains: a firm’s real mess
- 05Who built the files
- 06The separating couple: an assumption never checked
- 07Three errors any tax professional recognises
- 08Bringing doubt into the machine: the role of a decision model
- 09What it changes for an accounting firm, right now
- 10Frequently asked questions
Anna wants to sell her condo. She moved out in August 2024 and asks whether closing in October 2026 or January 2027 would be better. The AI agent compares the two dates and correctly concludes that both meet the residence requirement that excludes up to $250,000 of gain.
Yet it misses the main point. The rule requires having lived in the home for two of the five years before the sale: from the end of August 2027, Anna no longer meets it. If the sale slips, the benefit is gone, and nobody warned her.
That is the case the data company Surge AI opens DAYJOB: Tax with, published on October 27, 2026. The test hands AI agents real tax files, documents in a jumble, and checks whether they return work a tax professional would sign. We read it closely, because it finally answers the question accounting firms keep asking us: how far can a file be handed to AI?
In short
50 real tax files, 1,789 documents, built and reviewed three times by tax professionals from large firms.
The best agent passes 10.8% of assignments, the tenth 1.6%, at a 95% criteria threshold.
Failures are not about arithmetic but judgment: what the client never asked, what needed checking, what is missing from the file.
No agent judges consistently: the one that spots Anna’s deadline misses another client’s marital status, and vice versa.
For a firm, the right answer is not to wait, but to separate what the agent prepares from what the accountant decides.
What the test measures, and why it changes things
Tax tests published so far posed a clean problem: a legal question, a return to calculate, a missing piece of information to request. DAYJOB: Tax starts from a client request and a file as it actually reaches the firm: incomplete, contradictory, with decisive issues nobody flagged. The agent must decide what to examine, settle conflicting evidence, see when information is insufficient, then deliver: a return, a workpaper, a technical review or written advice.
The three existing tax tests and what DAYJOB: Tax also asks: owning the entire responsibility of the job, not one clean question. Table published by Surge AI on October 27, 2026.
The difference fits in one line from the publisher: the real question is not “does the agent know tax?”, but “does the agent take ownership of the file?”. That is exactly what a firm expects from an accountant.
The leaderboard: nine assignments out of ten fail
An assignment passes when the agent meets at least 95% of the criteria set by the tax professionals: calculations, supporting evidence, decisive issues raised, complete deliverable. Each agent had one attempt per assignment for the score.
The top ten agents, as published by Surge AI on October 27, 2026 (pass in one attempt, 95% threshold).
Share of assignments passed, by agent
On a scale of 100, it is plain to see: even the leader fails nine times out of ten. Gaps between vendors exist, but they matter less than this shared ceiling. These figures come from the test’s publisher itself, which also sells training data to AI labs: it is not an independent evaluation, and the full dataset is only available on request.
What an attempt costs
Surge AI also measured the average cost per attempt, which matters directly to a firm that bills per file.
Pass rate and mean cost per attempt, log scale. Chart published by Surge AI on October 27, 2026.
Read off the chart, the gap is one to sixty: about $0.25 per attempt for the cheapest model on the frontier, around $14 for the leader. And paying more guarantees nothing: GPT 6 Astra costs about as much as the leader for less than half its score. Several open models, which a firm could run in-house, stay below 1%. The lesson for a firm is the same as everywhere: the right model is chosen task by task, not once and for all.
Eight areas, from individuals to disputes
The test covers eight areas of tax practice, each with its own rules and records:
areas: individuals and families, small businesses, partnerships, property and investments, estates, gifts and trusts, international, corporate transactions, review and disputes
assignments, from full filing packages to advice on decisions the client has not yet made
Some assignments require a complete filing package with workpapers. Others ask to investigate a transaction, review a filed return, or advise before a decision. The files are American, but a Swiss, Belgian or Luxembourg firm will recognise every one of these situations.
What a file contains: a firm’s real mess
documents in total across the 50 files
PDFs, 117 Excel workbooks and 38 Word documents, plus emails, photos, CSVs and exports
documents per file at the median
documents in the heaviest file
They include filed and draft returns, wage and income statements, tax authority notices, general ledgers, payroll records, contracts and correspondence. And above all the less standard evidence an accountant works with every day.
A handwritten receipt for merchandise: the agent must read the items and amounts, reconcile them with recorded spending, then work out the effect on inventory and cost of goods sold. Document from a DAYJOB: Tax file, published by Surge AI.
The second example is even more telling. An industrial silo-cleaning company believes it only operates in one state, Alabama. Its dispatch log says otherwise.
The dispatch log shows crews sent to five states: locations, headcount and days on site bear on the company’s tax obligations outside its home state. Document from a DAYJOB: Tax file, published by Surge AI.
The client asked nothing about it. The evidence is in the file, on a page nobody flagged. A third file mixes the payslips and property of a household living between two countries, to be reconciled with cryptocurrency exchange exports and wallet histories. Each of these cases looks like an ordinary Tuesday at an accounting firm.
Who built the files
The assignments start from real professional situations. Tax professionals turned them into files, with documents, expected outcomes and grading criteria, and each assignment was reviewed three times: correctness, completeness, and whether it can actually be solved. Among them, according to the publisher: a tax attorney from PwC, Deloitte and Exxon, a Baker Tilly tax director formerly at PwC and Moss Adams, and a real estate and trusts specialist from Deloitte and EY. In other words, people who know what a partner refuses to sign.
The separating couple: an assumption never checked
A separating couple asks how to divide two retirement accounts. One of the top-ranked agents answers bluntly that the couple is not married, and recommends keeping the accounts separate with a cash payment to even things out.
You aren’t legally married, so neither account can be split without triggering taxes and penalties.
Yet the couple’s file raises doubts. They now live in Florida, where new common-law marriages are generally not recognised, but they lived together in Texas, which recognises informal marriage under conditions, and held a ceremony in Colorado. A marriage validly formed elsewhere may remain valid in Florida. And that status changes the whole tax treatment of the split.
Across five attempts, this agent built all five recommendations on the assumption that the couple was unmarried, even when it acknowledged uncertainty. A competitor, ranked lower overall, saw the problem in four attempts out of five and answered what a tax professional would have:
We first need to confirm your legal marital status.
The most striking part lies elsewhere. In Anna’s case it is the reverse: that same competitor never saw the deadline in five attempts, while the couple’s agent saw it five times out of five. No model judges consistently from one file to the next. That is the best reason not to look for “the right model” to hand everything to, but to build a circuit where every doubt goes back to a person.
Key takeaway
The job is not in the arithmetic
Agents can perform each step in isolation. What they lack, according to the test’s authors, is the ability to exercise professional judgment and reliably carry a real file from start to finish. Their conclusion fits in one line: agents haven’t yet passed the audit.
Three errors any tax professional recognises
- Missing the question the client never asked.
Anna’s deadline, the work in five states: the decisive issue was not in the request, it was in the file.
- Reasoning on an unverified fact.
The couple’s marital status was doubtful and the documents said so; the agent decided without checking.
- Treating a missing document as present.
In another file, a studio’s ledger refers to an attached receipt that does not exist; the agent still treats the reimbursement as supported.
These three errors share one trait: they do not show in the deliverable. The calculation is right, the memo is well written, and that is exactly what makes them dangerous for a firm that carries the liability.
Bringing doubt into the machine: the role of a decision model
If the problem is judgment, the answer is not a bigger language model. It is to ask the file, before any recommendation, the questions a partner would ask, and let a person decide whenever there is doubt.
That is where decision models such as Jev, from TypeSafe AI, make sense. Jev writes nothing: it answers closed questions, a choice, a score or a true-false, with a probability, in under a second. We introduced it in our article on AI decision models. Applied to a tax file, it gives checkpoints like these:
A checkpoint before every recommendation
- The agent prepares the draft.
It has read the documents, run the calculations and written a first memo.
- The decision model questions the file.
“Does a fact the recommendation depends on rest on an assumption without a document?” “Could a deadline in the next twelve months change the chosen option?” “Does every deducted amount have its receipt?”
- The code applies a threshold.
Above it, the file moves on; below it, or in doubt, it goes to the accountant with the precise question.
- The accountant decides and signs.
They do not reread everything: they start with the flagged doubts.
The value of a calibrated probability is simple: a model that says 90% should be wrong about one time in ten, which lets the threshold follow the risk of the file. An international estate does not get the same threshold as an individual return.
Two precautions we apply ourselves. Jev was not evaluated on DAYJOB: Tax: this circuit is our proposal, not a test result, and it is measured on the firm’s files before being tuned. And a decision model does not replace written rules: anything that fits a simple rule (“no receipt, no deduction”) stays a rule, and the model only handles the fuzzy questions, those that require reading and understanding a situation.
What it changes for an accounting firm, right now
Our view: this test does not say “wait”, it says “put AI in the right place”. What an agent does well today is everything that does not require a signature.
Hand to the agent
Receiving, naming and filing the client’s documents
Reading receipts, even handwritten, and reconciling them with the ledger
Spotting missing documents and unsupported amounts
Drafting the first version of the workpaper
Keep with the accountant
Validating the facts the file rests on
Asking the client the questions they missed
Choosing between tax options
Signing and sending the deliverable
That is exactly the ground covered by our bookkeeping agent on Telegram, which we run internally. You send it a photo of a receipt; it extracts the supplier, amount, VAT, date and category, checks for duplicates and consistent amounts, and only records the entry when its confidence clears a threshold. Below it, or for any sign-off, one human action: a button in the messaging app. It is the logic the test calls for, applied to bookkeeping: the agent does the volume, flags its doubts, and a person validates.
The step-by-step version of this split is in where the accountant sits when AI preps the file. Document collection itself can run over a messaging app, as with accounting bots in a firm.
For the overall approach, see how to integrate AI in a business. Two topics remain that are specific to firms. Confidentiality first: a tax file is among a client’s most sensitive data, and the agent can run on a model hosted by the firm, as explained in the four levels of private AI. Cost next: between $0.25 and $14 per attempt depending on the model, choosing task by task makes all the difference over a tax season, and that is the principle of our AI at the right cost offer.
Frequently asked questions
Can an AI agent prepare a tax return on its own?
Not end to end today. On real files, the best agent tested passes 10.8% of assignments at a tax professional’s standard. It is useful for preparing, sorting and checking, not for signing.
Which AI model should a firm choose for tax work?
The test shows none judges consistently: each spots issues the others miss. A circuit that sends doubts back to the accountant beats a bet on a single model, with the model chosen per task according to its cost.
Do these results apply to European tax?
The files are American. But the errors described do not depend on one country’s law: checking facts, seeing deadlines, flagging missing documents. They will show up in any tax system.
What does a decision model like Jev add?
It asks the file closed questions before any recommendation and returns a probability, so exactly the doubtful cases go to the accountant. It writes nothing and does not replace written rules; it is measured on the firm’s files before being tuned.
Where should a firm start?
With repetitive tasks that involve no decision: collecting and filing documents, deadline reminders, consistency checks. An audit sprint finds them in your own files and puts a figure on the time saved.
Jim · founder of Brain Plus AI
Also in business ai
Practical guide4 min
Accounting firms: where the accountant sits when AI preps the file
An AI agent can sort documents, reconcile accounts and draft a workpaper. It does not yet know when to doubt. Here is how to split a tax file between the agent and the accountant, without losing time or rigour.
Understand7 min
Accounting firms: one bot for receipts, one for client files
One bot records the receipts clients send in, another answers accountants' questions about a client file: two different uses, and both can run on the firm's own servers without losing quality.
Understand29 min
AI decision models: Jev, Clef, uses and costs
Jev, Clef and their rivals write nothing: they decide, with a probability attached. What these decision models are used for in each line of work, how to put them in place, what they really cost and what we make of them.