How Enterprises Are Structuring Business AI Programs

Business AI programs grow when companies build shared infrastructure beneath their agents instead of buying each agent as a separate piece of software. The tools should fit the work, but the operating layer must provide memory, identity, governance, and oversight across them.

Why enterprises hit a limit when they treat AI deployment as a procurement problem rather than an infrastructure problem

Enterprise agent adoption is accelerating, yet Gartner predicts that over 40% of agentic AI projects will be cancelled by 2027 due to insufficient governance. An agent can perform well in a demonstration while the system around it lacks the controls needed for production. Loops, hallucinated tool calls, and silent failures expose that gap. Buying a more capable model does not give a company identity, integration depth, or auditability.

What separates an enterprise agent from a consumer assistant? It is less the intelligence than everything that surrounds it. Alenezi (2026) describes the move from stateless prompt-response calls to goal-directed systems as a “consequential reconfiguration”. The model becomes a cognitive kernel inside a wider control architecture that has persistent state, typed tool interfaces, and runtime policy enforcement.

That distinction also answers the reasonable objection that some agents already work well in production. Small scope can conceal missing infrastructure for a time. One agent working with clean data does not expose the same scheduling, permission, health monitoring, state management, and auditability problems as hundreds operating across an enterprise. When buying software, companies ask which product can do the task. Infrastructure must keep that performance dependable as the scope grows.

What the layered architecture of a production enterprise agent actually looks like

A production enterprise agent program is a five-to-seven layer system, with each layer representing a separate architectural decision. In the seven-layer view, foundation models supply intelligence, protocols provide interoperability, and orchestration frameworks coordinate work. Tools and actuators connect decisions to actions. Memory preserves state, evaluation and governance assess and constrain behavior, and applications expose the resulting capabilities to users. These are distinct responsibilities, even when one platform packages several together.

The separation makes failures easier to locate. A reasoning error is different from an expired credential, a stale memory, or a tool execution that violates policy. Each belongs to a different part of the design and raises a different ownership question. Treating all of them as one “AI platform” decision hides those distinctions. It also makes a procurement comparison stand in for engineering decisions that still need explicit answers.

Alenezi argues that “agency is an architectural capability, not anthropomorphic intent”. That framing explains why production systems are converging on hybrid designs. LLMs handle high-level decomposition, while symbolic constraints govern tool execution. The model can adapt its plan without gaining unrestricted authority to act. Clean boundaries between cognition, execution, state, and policy are what make that balance possible.

How the model, protocol, and orchestration layers are consolidating — and what that means for teams building now

The foundation model layer is narrowing around a small set of frontier families, including GPT, Claude, and Gemini. The 2025 AI Agent Index found that 9 of the 30 documented agents explicitly let users choose among providers, with most of those being enterprise agents. The model is increasingly a replaceable component. That weakens the case for treating model choice as the defining infrastructure commitment.

Protocols are settling into distinct roles as well. Model Context Protocol supports tool and resource access, while A2A supports agent-to-agent calls. AG-UI has streaming interfaces, and A2UI lets agents use declarative agent-driven UI generation. These protocols have moved from experimental to default in under two years. Meanwhile, orchestration is converging around standardized agent loops, registries, and auditable controls.

Implementation choices remain within those shared boundaries. vLLM provides GPU-optimized serving through PagedAttention and continuous batching, while SGLang uses RadixAttention and prefix reuse for lower latency on agent workloads. For teams building now, the lesson is to shift attention upward. Memory, governance, and lifecycle management remain unresolved even when model access and tool communication are working.

Why memory is the most underbuilt layer in most enterprise agent programs

An LLM does not remember from one API call to the next. Persistence therefore has to be designed outside the model. Without that design, an agent may appear capable in isolation but lose the continuity needed for multi-step, multi-agent work. A longer prompt gives a call working context without deciding what is kept afterward.

Production memory has four distinct tiers, namely in-context working memory, external key-value storage, episodic logs, and semantic vector stores. They have different costs and persistence characteristics. Enterprise queries often use 50,000 to 100,000 tokens before reasoning begins, so working memory has the highest compute cost of any context even though it has zero persistence. Alenezi links these choices directly to how the system behaves. Cache policies affect context retention under load, vector database latency constrains perception cycles, and context-window budgets shape both cost and effective reasoning.

Memory also differs from long-term business context. Remembering a conversation does not establish which policy, business definition, or data source is currently trusted. An enterprise agent needs both forms of continuity, with different rules for each. A purpose-built agent operating system can make persistence, expiry, and propagation across agents shared runtime responsibilities.

How context orchestration at the knowledge layer maps to the same scaling crisis containers created a decade ago

Virtual machines needed VMware, and containers needed Kubernetes. Mouzouni (2026) uses that progression to explain why agents need an orchestration layer for organizational knowledge. Each computing generation encounters the same five problems, which are scheduling, permissions, health monitoring, state management, and auditability. Improving the underlying primitive does not resolve them. For agents, the shared resource is knowledge delivered with the right permissions and freshness.

The proposed Context Kubernetes architecture makes that delivery declarative. It combines six core abstractions with a structured configuration manifest for knowledge-architecture-as-code, a reconciliation loop, and a three-tier permission model. Agent authority is always a strict subset of human authority. Granting access once differs from keeping control over what knowledge an agent may receive. Reconciliation gives that design a way to detect when the available knowledge no longer matches its declared state.

Mouzouni evaluates ungoverned RAG, ACL-filtered retrieval, routing governed by role-based access controls, and the full three-tier model. Only the three-tier model blocks all five tested attack scenarios. The prototype also detects staleness in under one millisecond with reconciliation, while stale and deleted content is served silently without freshness monitoring. These are experimental findings. They still show that the infrastructure needs governed, versioned, permission-aware context routing as well as tool access.

What enterprise governance of AI agents actually requires — and where most programs are still improvising

Agent identity is often borrowed from a developer. That shortcut gives agents more access than they need and can break their operation when the developer leaves. The correct design provisions native agent identities through the organization’s identity provider, inherits document-level access controls, and defines provisioning, credential rotation, and decommissioning. Otherwise, the agent’s authority depends on a person whose role and access may change independently.

Observability has a similar boundary problem. Telemetry spread across four or more vendor dashboards does not give a company one audit trail across its agents, tools, and data sources. A kill switch covering only one vendor’s platform also leaves the broader fleet outside its reach. Governance must be able to account for the operation as a whole.

Lifecycle and cost controls complete that picture. Agents can be created quickly and persist beyond their creators, so a registry needs to track every agent, its owner, and its retirement. Clean revocation is part of retirement. Token spending also compounds with retrieval volume. Per-agent cost telemetry and usage caps therefore belong in governance, because a user-based budget alone does not capture how an agent consumes resources.

How named enterprise programs have actually structured their agent deployments

The recurring production pattern is a shared operating layer beneath specialized agents. A Fortune 100 consumer goods enterprise, for example, uses a triage layer to route employee requests across HR, finance, and operations agents. Each specialist runs its own sequential workflow under unified governance, identity, and audit. Coordination sits above the individual workflows. A Fortune 100 technology company applies that principle at a different scale, using more than 200 interconnected agents across its corporate venture capital division for investment sourcing and evaluation.

Fiserv’s deployment spans core banking, payments, issuer processing, and servicing, with policy controls, auditability, and human oversight embedded in the design. PwC agent OS serves large enterprises, while EY.ai Agentic Platform supports EY’s own workforce. Amdocs aOS addresses telecom operators, and Experian’s Agent Operating System serves lenders and financial services.

Incident investigation provides a more specific comparison. Augment Cosmos’s Incident Investigator responds to PagerDuty alerts, correlates Datadog metrics, Sentry errors, GitLab changes, and code context, and posts an initial root cause analysis with a recommended remediation before human review. Harness’s Human-Aware Change Agent instead listens to team dialogue and connects observations with deployments, feature flags, configuration, infrastructure changes, and ITSM records. The entry points differ, even within one workflow category. That is why a useful shortlist begins with the work and its required integrations, while the shared infrastructure remains a separate decision.

Why the pilot-to-production gap is a structural problem, not a maturity problem — and the specific failure modes that cause it

A pilot that uses clean data and covers only a narrow set of tasks can succeed without testing the conditions it will face in production. Broader deployment then exposes connector limitations, permission mismatches, and quality gaps. These failures are predictable consequences of the test boundary. They show that the pilot tested how well the agent could reason while leaving assumptions about how it would operate untested.

Poor data makes the gap especially costly. Automation does not merely encounter flawed information, but can turn it into flawed outputs at scale. More agents can amplify the same problem. When agents proliferate without a registry or clear ownership, the coordination burden resembles microservices sprawl with non-deterministic behavior added.

A dashboard can display activity without establishing who owns an agent, what authority it has, or how it should be retired. The lesson for a stalled program is to structure pilots around defined go/no-go decisions. Those decisions must test governance, identity, and observability as well as reasoning quality.

How should engineering teams implement business AI with the operating layer built in?

For engineering teams, the practical agenda begins with the layers most often improvised, namely memory, governance, and observability. A registry should precede proliferation, with every agent passing through it and retaining a named owner. Agent identities should come from the organization’s identity provider. Auditability should be designed before compliance evidence is requested. Memory tiers need explicit design, and token budgets need treatment as governance controls from the start.

Those requirements point toward a purpose-built agent operating system. Its role is to provide a shared runtime that hides coordination complexity behind a clean interface while preserving the programmability production engineering requires. General-purpose cloud tooling does not supply autonomous agent lifecycle management by default. The architectural decision is therefore not simply where agents run. It is whether their common operating needs are built into a platform or repeatedly retrofitted around individual deployments.

Alenezi places the next phase of agentic AI on a path similar to the maturation of web services, “relying on shared protocols, typed contracts, and layered governance structures to support scalable and composable autonomy”. That makes the operating layer an enduring infrastructure investment. Each new agent should receive established memory, identity, policy, and audit capabilities.