AI Agents on Top of Legacy: How the Mittelstand Modernises Without Ripping Out the ERP

Walk into any Mittelstand manufacturer, wholesaler, or services firm in Germany and you will find the same thing under the hood: an ERP that was installed when smartphones did not exist, a dozen satellite systems bolted on over two decades, and a room full of Sachbearbeiter who know the workarounds the software does not. This is the operating reality of the companies that generate most of Germany’s economic output¹.

For years, the advice has been the same: modernise first, then do AI. Rip out ECC, migrate to S/4HANA, consolidate the satellites, clean up the data, then think about agents. That advice is wrong. Gartner projects that nearly half of all SAP ECC customers will still be running it past the 2027 mainstream support deadline³. Roughly 75 percent of ERP-related AI projects stall at the integration boundary between the legacy system and the new AI capability⁹. The companies trying to do it in order are the ones getting stuck.

This guide lays out the opposite approach. You deploy AI agents as a layer on top of what you already run. No rip-and-replace. No 36-month migration before anyone sees value. You get ROI from your legacy stack while the modernisation decision is still being weighed - and when you eventually do migrate, the agent layer survives the transition and actually de-risks it.

The Mittelstand Legacy Reality

Legacy is not a niche problem. It is the operating state of German mid-market IT. Any honest plan for AI has to start from that premise, not from a greenfield fantasy.

• Pre-2005 ERP is common - 41 percent of German and Italian industrial firms still rely on custom-built ERP or MRP systems developed before 2005, many without modern APIs¹.
• SAP dominates, ECC persists - More than 75 percent of German enterprises run core SAP systems. Gartner estimates 70 percent of ECC customers have not yet migrated to S/4HANA as of early 2026^3,14.
• Satellite sprawl is real - A typical Mittelstand company of 300-1,500 employees runs 20-80 business systems alongside the core ERP: MES, WMS, CRM, PLM, CAD, custom Access databases, industry-specific tools, and shared drives full of spreadsheets.
• Technical debt eats the budget - Enterprises burn roughly 40 percent of their IT budgets just maintaining legacy systems. CIOs divert 10-20 percent of their new-product budget to tech debt^13,14.
• Developer time is trapped - Internal IT spends 33 percent of its time on technical debt activities rather than building new capabilities¹⁴.
• The digitalisation headache is now a management problem - Bitkom reports that for the first time, a majority of German companies say they struggle to steer their own digitalisation. 53 percent identify steering as the main challenge, ahead of budget or technology^1,16.
• The IT talent gap makes it worse - Germany has roughly 149,000 open IT positions; Mittelstand firms rarely win the competition for ABAP, RPG, or integration specialists¹⁶.

Why Rip-and-Replace Fails the Mittelstand

The traditional playbook says: modernise the core system first, then layer AI on top. For most Mittelstand companies that advice is not just slow - it is structurally broken. Three things go wrong.

1. The timeline kills the case

• Migration projects run long - A mid-sized S/4HANA migration typically takes 12-36 months from kick-off to stable production.
• Headcount is committed for years - Your best process and IT people are locked into the migration, unavailable for anything else - including AI.
• Value lands at the end - The business sees no operational improvement until cutover, which means years of invoice without return.
• Priorities shift mid-flight - A supply chain crisis, a new regulation, or a leadership change mid-migration regularly derails the project or forces re-scope.
• The AI race does not pause - While you migrate, competitors who skipped the rip-and-replace step deploy agents and widen the efficiency gap.

2. The risk concentration is extreme

• Single-point-of-failure project - A rip-and-replace project bets the operational core of the business on one cutover event.
• Data migration is where projects die - Decades of incomplete, inconsistent, or undocumented data have to be cleaned and mapped before cutover. Nobody owns all of it.
• Change fatigue is real - The company goes through a multi-year change programme just to end up where it already was functionally, only with a new UI.
• Cost overruns are the norm - Every dollar of maintenance deferred today costs roughly four dollars when forced to address it later, and greenfield ERP projects regularly run 50-100 percent over budget¹⁴.
• Vendor leverage flips - Mid-migration you are in the weakest commercial position possible with your ERP vendor and system integrator.

3. The AI outcome is still not guaranteed

• New ERP, same process - If the goal is AI, you have to redesign the processes anyway; the migration does not do that for you.
• New ERP, old data habits - A fresh system does not automatically produce clean data; it just gives you new places to store the same mess.
• AI after migration is a second project - Post-migration your team is exhausted, budget is drained, and the AI effort starts from zero, not from an ahead-of-schedule position.
• Annie Hodgkins of Gartner put it bluntly - simply bolting AI on top of an old existing stack does not magically make an application intelligent or future-ready⁸.

The Agent-Layer Alternative

The agent layer is not a product; it is an architectural pattern. The idea is simple: an AI agent sits above your existing systems, reads and writes through documented interfaces, keeps its own state, and does the multi-step work a human previously did. Your ECC, your AS/400, your custom .NET app, your Access database - all stay in place.

What the pattern actually looks like

• Interface layer - the agent consumes whatever your systems already expose: REST APIs, SOAP services, SAP BAPI/RFC, IBM i Db2 SQL, file drops, email, or a thin adapter built specifically for the agent.
• Agent runtime - the reasoning and tool-use logic that decides what to do next, hosted in your cloud tenant or on-prem environment.
• Agent state and memory - the agent’s own store for pending tasks, conversation context, and learned patterns, entirely separate from the ERP.
• Human-in-the-loop checkpoints - critical actions (large invoices, contract changes, exceptions) route to a reviewer before being written back.
• Audit trail - every agent action is logged with inputs, outputs, and the reasoning chain, making governance and compliance straightforward.
• Observability - dashboards show agent throughput, accuracy, and escalation rate so the business team owns the quality metric, not just IT.

Why this wins for the Mittelstand

1. You use what you have - no new ERP, no new platform, no rip-and-replace. The agent becomes additive capacity, not another system to learn.
2. You start narrow - one process, one system pair, one team. The blast radius of a mistake is contained and the success case is measurable.
3. You build in weeks - typical first use case goes to production in 8-12 weeks, not 8-12 months. McKinsey reports about a quarter of organisations already scaling agentic AI in at least one function with similar cycle times⁵.
4. You accumulate wins - use case two reuses the integrations and governance from use case one, so the second deployment is cheaper and faster than the first.
5. You keep optionality - the agent layer does not lock you into any migration path. It improves your current system and pre-computes which data and flows matter most, which is exactly what a future migration needs.

5 Integration Patterns That Actually Work

The agent layer stands or falls on how well it talks to the legacy systems underneath. Here are the five patterns that keep showing up in successful Mittelstand deployments, ordered roughly from cleanest to scrappiest.

1. Documented API or service interface

• What it is - the legacy system exposes REST, SOAP, OData, or GraphQL endpoints that the agent calls directly.
• Where it shows up - newer SAP releases, modern CRM and service desks, cloud-era satellites, vendor-provided integration APIs.
• When to pick it - whenever the endpoint exists and is stable, this is always the first choice.
• Watchouts - rate limits, authentication expiry, versioning; treat the legacy API like a contract and monitor it.

2. ERP-specific protocols (BAPI, RFC, IDoc)

• What it is - SAP and other enterprise ERPs expose remote function calls long before they have REST endpoints. The agent uses the SDK or a middleware translator.
• Where it shows up - SAP ECC, SAP S/4HANA on-prem, many IBM mainframe workloads.
• When to pick it - when the standard API surface does not cover the function you need; BAPI/RFC often go further than REST.
• Watchouts - licensing (indirect access), transactional integrity, rollback semantics if the agent fails mid-workflow.

3. Data pipeline or event stream

• What it is - the agent reads from a data warehouse, change data capture stream, or message queue that already mirrors the legacy system.
• Where it shows up - companies with a data lake, Kafka backbone, or ETL tooling already in place.
• When to pick it - high-volume read use cases (analytics, forecasting, anomaly detection) where real-time writes are not required.
• Watchouts - freshness lag, schema drift, lineage; the pipeline becomes part of the agent’s blast radius.

4. RPA bridge or UI automation

• What it is - the agent drives a UI Path, Blue Prism, or custom Playwright bot that clicks through the legacy UI on its behalf.
• Where it shows up - green-screen terminals, old Windows client apps, portals without APIs.
• When to pick it - when the legacy system truly has no programmable surface and an adapter is too expensive.
• Watchouts - brittle against UI changes; treat the RPA bridge as temporary even if it lasts for years.

5. Thin adapter service

• What it is - a small service (often a few hundred lines of code) that reads from files, database tables, or queues the legacy system already uses, and exposes a clean API to the agent.
• Where it shows up - custom-built ERPs, Access databases, shared drives, old AS/400 programs with no public interface.
• When to pick it - when none of the above apply but you can still get to the data through some indirect channel.
• Watchouts - keep the adapter read-mostly at first; writes can be added once confidence is high.

The SAP ECC Question: Migrate Now, Layer Now, or Both?

Every German Mittelstand board is having the same conversation: what do we do about SAP ECC before the 2027 deadline? The answer is not binary. Agent layering and S/4HANA migration are not competing choices; they are complementary decisions with different timelines.

What the data says

• Mainstream support ends 31 December 2027 - after that, extended support is available at a premium until 2030, and third-party support is an option for even longer²¹.
• Most customers will miss the deadline - Gartner projects about 17,000 ECC holdouts at the 2027 cutoff, roughly half the customer base³.
• Only 39 percent of ECC customers had licensed S/4HANA as of end-2024³.
• Migration alone does not deliver AI value - it changes the platform, not the process. Without a deliberate process redesign, S/4HANA becomes a more expensive ECC.
• Agent layering works on ECC today - BAPI, RFC, and IDoc interfaces are stable, documented, and support the vast majority of process automation scenarios.

How to sequence the two decisions

1. Deploy the agent layer now - on whichever system you already run, with a use case that pays back inside 9 months.
2. Let the agent surface the real process - the mapping exercise documents what your business actually does, which is the single most expensive deliverable in any S/4HANA project.
3. Use the agent’s data clarity to scope the migration - you know exactly which data flows, master data elements, and customisations matter because the agent has been using them in production.
4. Migrate when the business case closes - with the agent doing the operational work, the migration timeline stops being on the critical path for AI value.
5. Swap the connector at cutover - agent logic, training, governance, and workflows stay; only the integration endpoints change from ECC to S/4HANA.

AS/400, Custom ERP, and the Other Legacy Monsters

Not every Mittelstand company runs SAP. Many have grown up around IBM i on Power hardware, a custom-built ERP from a regional integrator, or a constellation of MS Access databases held together by one retiring developer. Each has its own path into the agent layer.

IBM i / AS/400

• It is not as stuck as it looks - IBM i exposes data through Db2 SQL, stored procedures, Integrated Web Services, and increasingly through MCP servers that translate legacy operations into tool calls an agent can use^11,12.
• RPG business logic can stay - expose the right programs through IWS or a thin REST adapter; the agent calls them like any other tool without RPG rewrites.
• Access patterns matter - prefer read-only access first, then add approved write paths once confidence is high.
• Licensing and capacity - additional interactive workload on IBM i has historical licensing quirks; coordinate with your IBM partner before going live.

Custom regional ERP

• APIs are usually thin or absent - integrator-built ERPs often expose the database or file drops but not a proper service interface.
• Adapter pattern wins - a small service you control, sitting next to the ERP, exposes a clean API to the agent and isolates future ERP changes.
• Keep the vendor informed - many custom ERP vendors are open to API work; some will co-invest to add AI-friendly surfaces.
• Document as you go - the mapping you do to build the adapter is a capital asset, not a throwaway artefact; it becomes the spec for any future replacement.

Access, Excel, and the shadow stack

• Yes, it counts as legacy - shadow systems often run processes the ERP cannot. Agents can pick up those processes too.
• File-based ingestion is fine - an agent that reads from a shared folder or Excel file is still a valid pattern when nothing better exists.
• Make the shadow official - the agent project is a good moment to move critical shadow-system data into a governed store, rather than ripping it out.
• Cover the human process, not just the file - the agent has to understand who edits the file, when, and why; automating the file without the process does not deliver ROI.

Risk Management and Governance on Legacy

Layering an agent on top of a 20-year-old system is not reckless - it is actually one of the safer AI deployment patterns, because the core system stays untouched. The risks that do exist are well known and manageable.

Six risks to plan for

1. Unintended writes - an agent that can change data in the legacy system can do damage at scale if it misbehaves. Start read-only, promote to write one action at a time, with approval checkpoints for high-impact writes.
2. Load and stability - legacy systems were sized for human traffic. An agent running thousands of queries per hour can destabilise performance if not rate-limited.
3. Authentication and secrets - credentials for legacy systems are often shared or poorly rotated. The agent should use scoped service accounts with per-environment secrets stored in a secure vault.
4. Data leakage - sensitive data (HR, finance, customer PII) must stay inside the boundary. Avoid sending raw records to external LLMs; redact, summarise, or keep inference in-tenant.
5. Prompt injection via legacy content - user-entered fields (customer notes, supplier messages) can contain instructions that manipulate the agent. Treat legacy content as untrusted input and validate accordingly.
6. Dependency on people with legacy knowledge - the agent project reveals how much relies on a single RPG or ABAP specialist. Pair them with the agent team explicitly and document what they know.

Compliance in practice

• DSGVO - data stays inside your infrastructure, processing purposes documented, legal basis mapped per use case. Agents do not create new compliance categories, they inherit the ones the underlying process already had.
• EU AI Act - most operational agents on legacy systems fall into minimal or limited risk. Transparency notices for customer-facing agents, Article 4 training for employees, conformity assessment only if the use case is high-risk (credit scoring, hiring, safety critical)¹⁸.
• Sector regulations - automotive, medical, pharma, and financial services add their own audit requirements. The agent’s structured log is usually easier to audit than the human trail it replaces.
• Works council (Betriebsrat) - co-determination rules apply to any process that touches employee work. Involve the Betriebsrat early; in practice, the agent layer is often easier to approve than a full ERP migration because the changes are narrower and more visible.

The 90-Day Agent-on-Legacy Playbook

The scariest part of the agent layer is the legacy system underneath. The playbook addresses that head-on. Here is how a successful first deployment actually runs.

Phase 1: Discovery and architecture (Weeks 1-3)

1. Week 1: Use case selection - pick one process that costs real hours, involves at least two systems, and has a clear business owner. Avoid the most sensitive process first; pick the most expensive routine one.
2. Week 2: System inventory - document every system involved, every interface available, and every manual workaround the team uses today. Pay special attention to the workarounds - that is where most legacy value hides.
3. Week 3: Integration design - choose the integration pattern for each system touched. Confirm access, auth, rate limits, and environments. Get IT security sign-off on the boundary before writing code.

Phase 2: Build and test (Weeks 4-8)

4. Week 4-5: Connectors and adapters - build the thin integration layer. Keep adapters narrow; one use case does not need the whole system mapped.
5. Week 6: Agent logic - assemble reasoning, tool use, and human-in-the-loop checkpoints. Run against historical data first.
6. Week 7: Shadow mode - agent runs alongside the existing process without acting. Team reviews agent decisions and flags the ones they disagree with.
7. Week 8: Refinement - agent adjusts based on shadow-mode feedback. Edge cases documented, escalation paths validated.

Phase 3: Soft launch and measure (Weeks 9-12)

9. Week 9: Read-only go-live - agent writes nothing yet. It produces recommendations the team chooses to execute or override.
10. Week 10: Write-enabled on low-risk actions - agent starts writing for the clearly safe actions (create draft, suggest category, route ticket).
11. Week 11: Full scope - agent handles the full process within defined thresholds; anything above threshold escalates.
12. Week 12: Measurement and next use case - compare KPIs to the baseline, present to leadership, scope the next use case reusing the same integration layer.

How Superkind Fits

Superkind builds custom AI agents that sit as a single layer on top of your existing tech stack. For Mittelstand companies, this is the whole point: no rip-and-replace, no new platform to learn, and no waiting for the ERP migration to finish before AI delivers value.

• Process-first, system-second - we start with the workflow your team actually runs, not with the system architecture. The mapping surfaces the real rules, including the ones nobody wrote down.
• Works on whatever you run - SAP ECC, SAP S/4HANA, IBM i, custom regional ERPs, Salesforce, HubSpot, Access, file shares, email. The agent layer is system-agnostic by design.
• Clean boundary with your legacy - agents talk to your systems through documented interfaces or thin adapters we build. Nothing invasive, nothing that creates new technical debt.
• Live in 8-12 weeks - first production use case inside a quarter, not inside a decade. Your team gives feedback from day one and the agent gets sharper with every interaction.
• Outcomes, not licenses - pricing is per use case with measurable ROI defined before the build starts. No per-seat contracts, no multi-year lock-ins.
• Migration-safe - when you eventually move to a new ERP, the agent logic, the workflows, and the team training all survive. We swap the connector, not the whole stack.
• Governance built in - audit logs, human-in-the-loop checkpoints, scoped access, DSGVO-aligned architecture, Article 4 literacy materials ready for the EU AI Act.
• Continuous partnership - we do not deliver and disappear. Use case one funds use case two, and the integration layer compounds across departments.

Decision Framework: Is the Agent Layer Right for You?

Not every company needs the agent layer first. Here is a framework for deciding.

Frequently Asked Questions

Further Reading

• AI Agents for the Mittelstand - the broader playbook for deploying AI agents in German SMEs.
• The 12-Month AI Strategy Roadmap for the Mittelstand - how to sequence AI across four 90-day phases.
• RPA vs AI Agents - when RPA is still the right answer and when it is not.
• Fix Your Processes Before You Add AI - why process clarity beats new technology.
• Your AI Is Only as Good as Your Data - the data prerequisites for successful agent deployment.
• AI Agents vs Microsoft Copilot - when custom agents beat off-the-shelf AI.
• AI Agents in Manufacturing - use cases and integration patterns for production.

Henri Jung

Co-founder of Superkind, where he helps SMEs and enterprises deploy custom AI agents that actually fit how their teams work. Henri is passionate about closing the gap between what AI can do and the value it creates in real companies. He believes the Mittelstand has everything it needs to lead in AI - it just needs the right approach.