Enterprise Software Still Needs Human Expertise
Ask an AI model for a stock-tracking app and you will get something that runs within minutes. Ask it to replace the way a manufacturer with three plants, two currencies and a decade of customer-specific pricing runs its operations, and the picture changes. Enterprise software is not mainly a coding problem. It is a problem of understanding how a business really works, deciding what the system must guarantee, and keeping those guarantees for years. That is where human expertise still makes the difference, even as AI changes how the code itself gets written.
This is not an argument against AI. Our own teams use AI tools every day. It is an argument about where responsibility sits, and why companies buying or building business-critical systems should look for experienced people behind the tools.
Code Is the Easy Part of Enterprise Software
In a typical enterprise project, writing code is a minority of the effort. The harder work happens before and around it:
- Discovering the real process: the documented process and the process people actually follow are rarely the same. Exceptions, workarounds and informal approvals have to be found by talking to people on the shop floor, in the warehouse and in finance.
- Making trade-offs: should a sales order reserve stock immediately or at confirmation? Should credit limits block orders or just warn? These are business decisions with financial consequences, not technical details.
- Handling data that already exists: years of item codes, customer records and open balances must be cleaned, mapped and migrated without breaking reporting or audits.
- Managing change: users need to trust the new system. Rollout plans, training and parallel runs decide whether a project succeeds.
AI can help with each of these, summarising interviews or drafting test cases for example, but it cannot take responsibility for the decisions.
ERP and CRM: Where Context Matters Most
ERP and CRM systems sit at the centre of money, stock and customer relationships. A small misunderstanding there has outsized effects. Consider a few examples that appear in real projects:
- Costing: a change in how scrap or overhead is allocated changes product margins and, through them, pricing decisions.
- Units and conversions: buying in rolls, producing in metres and selling in pieces requires conversion rules that must be exact everywhere.
- Tax and e-invoicing: rules differ by country and change over time. The UAE, Türkiye and the UK each have their own requirements and timelines.
- Customer-specific terms: price lists, discounts, payment terms and delivery rules often depend on contracts that live in people's heads or in email threads.
Generated code can implement any rule it is given. The hard part is knowing which rules are right, which ones conflict, and which will still be right after the next regulatory change. That knowledge comes from people who have seen many implementations.
Architecture Decisions Outlive the Code
Architecture is the set of decisions that are expensive to change later: how data is modelled, how systems talk to each other, where business logic lives, how the system is deployed and how it fails. A model can propose an architecture, but it does not carry the consequences when that architecture meets reality three years later.
- Data model: whether an order line can carry several lots, or a product can have several units of measure, shapes every screen and report.
- Integration style: real-time APIs, scheduled syncs or event queues each fit different needs. Choosing wrongly creates either fragility or stale data.
- Boundaries: deciding what stays in the ERP and what lives in custom modules keeps upgrades possible and avoids a tangle of customisations.
- Failure handling: what happens when the e-invoice portal is down, or a marketplace rejects a stock update, must be designed, not discovered.
Security Is a Human Responsibility
Enterprise systems hold personal data, prices, supplier terms and financial records. Research shows why generated code needs expert review: in Veracode's 2025 study of more than 100 large language models across 80 coding tasks, about 45 percent of the AI-generated code samples introduced security flaws. Models were also weak at preventing cross-site scripting, failing in 86 percent of the relevant cases.
The point is not that AI is careless and humans are perfect. It is that security requires threat modelling, knowledge of how an attacker thinks, careful access control and a habit of questioning assumptions. In enterprise software, a human must decide who can see costs, who can approve payments, and what an AI agent connected to the ERP is allowed to do. We covered code review practices in more detail in our article on AI-written code; here the key message is ownership: someone must be accountable.
Scalability and Reliability Need Experience
A prototype that works with ten orders can fail with ten thousand. Scalability problems often appear only under real load: slow reports at month-end, locking issues when many users post transactions, integrations that time out during peak season. Experienced engineers anticipate these patterns because they have seen them before. They design indexes, background jobs, caching and monitoring from the start, and they know which shortcuts are safe and which will come back as outages.
Reliability also covers the boring things that matter most: backups that are actually tested, logs that make problems traceable, alerts that wake the right person, and deployment processes that can roll back safely.
Where AI Genuinely Helps Enterprise Software Teams
None of this means enterprise teams should avoid AI. Used well, it removes a lot of routine work:
- Generating boilerplate code, data transfer objects and simple screens.
- Writing first drafts of unit tests and test data.
- Explaining legacy code and suggesting refactoring steps.
- Drafting documentation, migration scripts and API clients.
- Summarising requirements workshops and turning them into user stories for review.
The pattern that works is AI-assisted and human-led: engineers use AI to move faster on well-understood tasks, and they keep design, review, security and sign-off in human hands.
A Practical Checklist for Enterprise Projects in the AI Era
If you are planning an ERP integration, a CRM rollout or a custom operations system, these checks help keep AI's speed without losing control:
- Name an accountable architect or technical lead for the project, not just a team.
- Document key business rules, such as costing, units, credit limits and tax, and have the business owner sign them off.
- Require code review and automated tests for all code, including AI-generated code.
- Run security scanning and dependency checks in the build pipeline.
- Test with realistic data volumes before go-live, especially month-end and peak-season scenarios.
- Define what any AI agent connected to your systems may read, write and approve, and log its actions.
- Keep a rollback plan and test your backups before the first production release.
Accountability, Compliance and Regulation
Enterprises answer to auditors, tax authorities, customers and, increasingly, AI regulation. In the EU, the AI Act's high-risk obligations for stand-alone systems now apply from 2 December 2027, after the Digital Omnibus postponed the original August 2026 date. Most enterprise systems are not high-risk AI systems, but many companies will still need to show how automated decisions are made and controlled. That requires documented design decisions and people who can explain them.
Frequently Asked Questions
Can AI build enterprise software on its own?
Not responsibly today. AI can generate large parts of the code, but enterprise software depends on understanding processes, making business trade-offs, designing architecture, securing data and taking accountability. Those still need experienced people.
Why does ERP integration need human expertise?
ERP integration touches money, stock and tax. Small misunderstandings about units, costing, credit limits or e-invoicing rules can create financial errors. Experts know which rules matter and how to test them.
Is AI-generated code secure enough for enterprise use?
Not without review. Veracode's 2025 research found that about 45 percent of AI-generated code samples introduced security flaws. Generated code should go through the same review, testing and security checks as any other code.
What should humans keep control of in enterprise software projects?
Requirements and process decisions, architecture, data model, security and access design, production deployments and final sign-off. AI can assist with all of these but should not own them.
Does AI make enterprise software cheaper?
It can reduce effort on routine coding and testing. The savings are smaller on discovery, integration, data migration and change management, which are often the largest parts of an enterprise project.
How do we choose a software partner in the AI era?
Ask how they use AI, how generated code is reviewed, who is accountable for architecture and security, and what experience they have in your sector. Look for an AI-assisted, human-led process.
How much access should AI agents have in enterprise software?
Only what the task needs. An agent can read stock or order data and prepare suggestions, but payments, price changes and deletions should require human approval. Log every agent action and review permissions regularly.
Conclusion
AI is changing how enterprise software is written, but not what it must guarantee: correct numbers, secure data, systems that scale and decisions someone can explain. Those guarantees still come from experienced people who understand both the technology and the business. Codefacture builds custom software for manufacturing, supply chain, automotive, wholesale and packaging companies in the UK, Australia, New Zealand, the Gulf and Türkiye. Our teams use AI where it speeds up the work and keep human expertise where it protects your operations.