The Hidden Risks of Building Your Own AI Agents

What accounting firms need to know before they put a homegrown agent into production.

AI is changing accounting fast: reconciliations, financial statement review, tax research, client document processing. More and more firms tell me they are taking matters into their own hands and building proprietary AI agents.

That is not a mistake. These firms have already proven the value of AI in their workflows, and they are usually the most forward-thinking firms in the market. They want control, less dependence on software vendors, and tools shaped around their own processes.

But one question gets far less attention than it deserves: what happens to these agents when the model underneath them is retired, replaced, or leapfrogged? And who maintains, secures, tests, and updates them for the next five years?

This is not theoretical. OpenAI and Anthropic, the two companies whose models power most of today’s agents, have published guidance on exactly these challenges. Here is what they say, and why it matters for accounting.

1. AI models have an expiration date

Every agent depends on a large language model such as GPT or Claude to read instructions, reason, call tools, and act. Those models are not permanent infrastructure. OpenAI maintains an official deprecations page listing models being retired, their replacements, and shutdown dates.

Picture a firm that builds an agent to reconcile bank statements against its accounting software. The prompts, tool calls, and output formats are tuned to one model’s behavior. Twelve months later, that model is retired.

Now the firm has to:

• Rewrite prompts and tool integrations for the replacement model.
• Revalidate the accounting logic and outputs.
• Test whether the new model reads instructions differently.
• Update security controls and error handling.

From what we see at Taxxa, revalidating one production agent after a model change takes roughly 2 to 3 engineer-weeks. A firm running 10 agents faces that every time a model is retired, which has happened 20 times in the past 12 months.

Swapping a model does not mean rebuilding the agent. But the tighter the coupling, the bigger the bill. For a firm without a dedicated AI engineering team, that is an operational dependency it may not be able to sustain.

2. Complexity is a liability, not a moat

In December 2024, Anthropic published Building Effective Agents, based on its work with dozens of teams building agents. Its central finding: the most successful implementations use simple, composable patterns rather than complex frameworks. Anthropic recommends starting with the simplest solution and adding complexity only when it measurably improves results. It also warns that frameworks add abstraction layers that make systems harder to understand and debug.

Now consider a firm building a multi-agent system for financial statement review: one agent retrieves data, another classifies transactions, a third reviews accounting treatment, a fourth checks compliance, a fifth writes the report.

Much of that orchestration exists to work around what today’s models cannot yet do reliably. When the next generation can handle several of those steps in one pass, a large part of that engineering becomes dead weight that still has to be maintained.

The real risk is not that the agent stops working. It is that the firm spends its budget engineering around limitations that will disappear on their own.

The lesson: design for change. Keep components modular, keep the model swappable, and put your investment in what will not change with the next model release: your accounting knowledge, rules, and quality standards.

3. An agent with access to financial systems is a new attack surface

Agents become valuable when they can act inside business systems: ERP and bookkeeping software, bank transactions, payroll, client financial statements, email and document management. That same access is what makes them a target.

In March 2026, OpenAI published Designing AI agents to resist prompt injection. Two points stand out. First, the most effective real-world attacks now look like social engineering, not obvious hidden commands, so filters that try to spot malicious input miss most of them. Second, OpenAI’s answer is to design the system so that the damage stays limited even when an attack succeeds, in the same way a human employee’s permissions are limited.

Translate that to accounting. A firm builds an agent that reads supplier invoices and prepares payment proposals. An invoice arrives with instructions buried in it: a “corrected” bank account, an urgent note, a request to forward the ledger. If the agent cannot reliably tell document content from legitimate instructions, and if it has the permissions to act, the attacker is now steering a payment workflow.

The attack is the same in both paths. What decides the outcome is whether the agent can pay without a human signing off.

For an accounting firm this is worse than for most businesses, because one agent may touch confidential data from dozens of clients at once.

Security has to be designed into the architecture: least-privilege access, strict separation of client data, human approval for payments and disclosures, logging, and monitoring. Building an agent that works is one job. Building one you can trust with client money and client data is a much bigger one.

4. The cost nobody budgets for: testing that never stops

An agent that works today is not guaranteed to work next quarter. Traditional software follows fixed instructions. An LLM-powered agent’s behavior shifts with the model, the context it receives, and the tools it calls.

In January 2026, Anthropic published Demystifying evals for AI agents. It explains that agents are hard to evaluate because they work over many steps, call tools, and change course based on intermediate results, so a mistake early on compounds. Anthropic’s point is that without a systematic evaluation suite, teams end up catching failures in production, where fixing one problem creates others.

Take an agent that reviews VAT transactions. It passes your test cases at launch. Then four things move under it:

• The model changes. A newer version reads instructions or uses tools differently.
• The rules change. Tax legislation, reporting requirements, and official guidance are updated constantly.
• The integrations change. ERP vendors modify APIs, permissions, and data structures.
• The workflows change. New document formats, approval steps, and client processes appear.

None of these changes are under the firm’s control, and each one can break an agent that passed every test at launch.

Each change means another round of testing and validation. Multiply that by dozens of agents across many clients and the maintenance load quickly outgrows the original build cost.

An AI agent is not a one-off automation project. It is a software product that needs an owner, a test suite, and a budget for as long as it runs.

5. Accountability cannot be delegated to a model

In December 2023, OpenAI published Practices for Governing Agentic AI Systems. Its recommendations include limiting what agents can do, requiring approval for consequential actions, making agent behavior understandable, monitoring operations, and keeping humans accountable.

Accounting firms do not need OpenAI to tell them this. Their own rulebook already does:

• ISQM 1. Audit and assurance firms must manage the quality risks of the resources they rely on, including technology. A homegrown agent used in an engagement is a resource the firm has to evaluate, document, and monitor.
• GDPR. Client financial data flowing through an agent needs a lawful basis, a clear processor chain, and control over where the data is processed and stored, including when it is sent to a model provider outside the EU.
• The EU AI Act. Firms that deploy AI systems take on obligations around transparency, human oversight, and record-keeping, which are being phased in over the coming years.

The firm remains responsible for its work whether a human or an agent produced it. So before an agent posts a journal entry, flags a VAT discrepancy, or recommends an accounting treatment, the firm needs answers to four questions. Who is accountable when the output is wrong? Who can authorize the agent to change financial records? How is each action documented? And how will the firm investigate an error discovered months later?

These are not technical questions. They are questions of governance, professional responsibility, and risk, and a firm that builds its own agents owns every one of them.

6. “Isn’t buying just swapping one dependency for another?”

Yes, it is a dependency. The question is which one you would rather carry.

If you build, you depend on a model provider’s roadmap, and you alone absorb every migration, security patch, regulatory change, and regression test. If you buy, you depend on a vendor, but that vendor spreads those same costs across hundreds of firms. It has to move to new models faster than any single firm, because its business depends on it.

The changes are the same either way. The difference is how many firms share the cost of absorbing them.

That changes what you should demand from a vendor, not whether you should use one. Ask four things:

• Model independence. Can it switch between model providers without you rebuilding anything?
• Data portability. Can you take your rules, prompts, workflows, and data with you if you leave?
• Transparency. Can you see what the system did, why, and on which source?
• Security and data residency. Where is client data processed, who can access it, and what can the system do without approval?

A vendor that cannot answer these well is a real lock-in risk. A vendor that can is taking on the work you would otherwise have to staff.

7. Build vs. buy: five questions to answer before production

There are good reasons to build. Some firms have processes no product supports, and some have engineering teams that can run software for years. For them, building can be the right call.

For most small and mid-sized firms, the decision deserves a proper business case. Before an agent moves from experiment to production, leadership should be able to answer these five questions:

1. What happens when the model is retired? Can the agent move to another model without major rework, and who pays for that?
2. Who owns security? Does the firm have the expertise to protect client data and control what the agent can do in financial systems?
3. Who maintains it? Is there a named team responsible for model updates, integrations, testing, monitoring, and incidents?
4. How is accuracy validated? Is there an evaluation suite that catches wrong outputs before a client or an auditor does?
5. Is this the best use of our people? Could a specialized platform deliver the same outcome while the firm focuses on client work and accounting expertise?

If the honest answer to questions 2 to 4 is “we’ll figure it out,” the agent is not ready for production.

Conclusion: building the agent is the easy part

The firms building their own agents have already done the hard part: they have shown that AI belongs in their workflows. The open question is whether they also want to run the infrastructure underneath it.

The companies building the models are clear about what that means. Models get retired. Complexity becomes a burden. Agents connected to financial systems become targets. Reliability has to be tested again and again. Accountability stays with the firm.

Those are not warnings from AI skeptics. They come from the companies making AI agents possible.

So before you invest further in proprietary agents, ask one question:

Are we building a lasting competitive advantage, or taking on a software maintenance job that will grow more complex and expensive every year?

Your advantage is your accounting expertise, your client relationships, and your judgment. Put your investment there, and let the infrastructure be someone else’s full-time job.

References

1. OpenAI. API Deprecations. Official documentation on model retirement and replacement. platform.openai.com/docs/deprecations
2. Anthropic (December 19, 2024). Building Effective Agents. anthropic.com/engineering/building-effective-agents
3. OpenAI (March 11, 2026). Designing AI agents to resist prompt injection. openai.com/index/designing-agents-to-resist-prompt-injection
4. Anthropic (January 9, 2026). Demystifying evals for AI agents. anthropic.com/engineering/demystifying-evals-for-ai-agents
5. OpenAI (December 14, 2023). Practices for Governing Agentic AI Systems. openai.com/index/practices-for-governing-agentic-ai-systems