From Copilots to Agents

The first wave of enterprise generative AI focused primarily on assistance. Copilots helped employees search for information, summarize documents, draft communications, analyze data and generate content. In most cases, the user remained responsible for reviewing the output and taking the next action.

AI agents change this relationship. An agent may be able to retrieve information from multiple systems, interpret a request, develop a plan, select tools, execute a workflow, update an enterprise application, communicate with another agent, monitor the result, and escalate an exception.

The shift may initially appear incremental. A copilot recommends an action; an agent executes it. Architecturally and operationally, however, the difference is significant. When AI can act, the enterprise is no longer governing only what a model says. It must govern what a system is allowed to do. That changes identity, access, integration, risk, accountability and operating-model requirements.

A copilot supports work. An agent participates in work.

The distinction between copilots and agents should not be based on product naming. It should be based on authority.

A copilot generally supports a person by providing information or generating a response. The person reviews the output and remains the primary decision-maker. An agent can be given a goal, determine intermediate steps, and use tools or systems to complete part of the work.

Consider a customer-service scenario.

A copilot may summarize the customer's history, retrieve the relevant policy, recommend a response, and draft a communication.

An agent may validate the customer's identity, determine eligibility, update the case, initiate a refund, schedule follow-up activity, send the communication, and close the request.

The business value may be considerably greater, but so is the consequence of error. The architecture must therefore recognize a central principle:

Autonomy is not a binary product feature. It is an enterprise-controlled level of authority.

The enterprise autonomy continuum

Organizations should classify AI capabilities according to the authority they receive.

Level	Description	Human role
1. Inform	The AI retrieves, summarizes or explains information.	User interprets and acts.
2. Recommend	The AI evaluates context and proposes a decision or next step.	User approves or rejects.
3. Prepare	The AI prepares a transaction but requires explicit human approval before execution.	Human authorizes.
4. Execute within boundaries	The agent completes approved actions autonomously within defined policies and limits.	Human handles exceptions.
5. Coordinate	Multiple agents or an orchestrating agent plan and execute across systems, functions or organizations.	Human governs the ecosystem.

Level 1: Inform

Search enterprise knowledge, summarize a service case, explain a policy, draft a meeting brief. The user remains responsible for interpretation and action.

Level 2: Recommend

Recommend a case category, identify the likely cause of an equipment issue, suggest a customer-service response, prioritize sales opportunities. The user approves or rejects the recommendation.

Level 3: Prepare an action

Prepare an order change, draft a claim decision, create a customer record, assemble a purchase request, prepare a maintenance work order. The human remains the authorization point.

Level 4: Execute within boundaries

Resolve low-risk service requests, schedule approved maintenance, update routine customer information, reorder inventory within thresholds, execute standard employee onboarding tasks. Human intervention is required only for exceptions or activity outside the approved boundary.

Level 5: Coordinate across processes

Coordinate supply-chain exceptions, manage portions of a claims process, respond to equipment disruption, execute multi-step financial operations, coordinate customer, product and service actions. At this level, the enterprise is governing an ecosystem of identities, decisions, tools and interactions — not an isolated application.

The risk moves from output to outcome

With a copilot, the immediate risk is usually associated with the quality of the output. Was the answer correct? Was confidential information exposed? Was the content biased, misleading or inappropriate?

These risks remain relevant for agents, but additional risks emerge:

• Was the agent authorized to take the action?
• Did it act for the correct user or business entity?
• Did it use an approved tool?
• Did it access only the required information?
• Did it interpret the policy correctly?
• Did it remain within financial or operational limits?
• Can the action be reversed?
• Can the enterprise reconstruct what happened?
• Who is accountable for the result?

The risk is no longer limited to an inaccurate response. It may become an incorrect transaction, an unauthorized change or a chain of actions across multiple systems.

NIST describes AI agents as systems capable of planning and taking autonomous actions that affect real-world systems or environments. Its work on agent security highlights that established cybersecurity practices remain important but require adaptation for agent-specific threats and authority.

Six architecture capabilities agents require

1. A governed identity for every agent

An enterprise would not permit an unidentified employee, contractor or application to access critical systems. Agents should not be treated differently. Every production agent should have a distinct identity.

The enterprise should be able to determine which agent is acting, who owns it, who sponsors it, which user or process it represents, which systems it can access, which actions it can perform, how long its authority remains valid, when its access was reviewed, and how it can be disabled.

Shared service accounts are particularly problematic for agents because they weaken traceability and can provide broader access than the agent requires. Agent identity should support both scenarios:

Agent acting as itself

The agent performs an enterprise function with its own defined permissions.

Agent acting on behalf of a user

The agent performs an action using delegated authority while preserving the identity and permissions of the person who initiated it.

Microsoft's Agent ID guidance applies identity, lifecycle management, access policies and audit logging to agents, while Google similarly describes unique cryptographic identities for agents acting independently or on behalf of users. The important principle is vendor-neutral:

2. A controlled tool and integration layer

Agents create business value by using tools. A tool may allow the agent to retrieve a customer record, update an order, create a case, initiate a payment, schedule maintenance, send a communication, query production information or invoke another agent.

These tools should be exposed through governed APIs and integration services — not through unrestricted direct access to databases or enterprise applications. Each tool should define its business purpose, required identity, permitted inputs, allowed actions, transaction limits, validation requirements, approval rules, logging requirements, and error and reversal processes.

AWS's enterprise agentic-AI reference architecture similarly separates controlled model access, enterprise knowledge and securely executable tools, with authorization and observability operating across the architecture.

For many organizations, this means their existing API and integration strategy will directly affect how quickly they can scale agents. A fragmented integration estate will become an agentic-AI constraint.

3. Least-privilege authority

An agent should receive only the minimum authority required to complete its approved purpose. This must include more than system access. The enterprise should also restrict transaction type, financial value, data category, customer or employee population, geographic boundary, operating period, frequency of action, tools available, and the ability to delegate to another agent.

For example, an agent may be allowed to issue a standard credit below a defined threshold but required to obtain human authorization above it. A maintenance agent may be permitted to create a work order but not shut down equipment. A claims agent may collect documents and prepare a recommendation but not deny a complex claim.

These boundaries should be enforceable at runtime, not simply documented in a policy.

4. Policy-aware orchestration

An agent should not independently decide which enterprise rules apply. The orchestration layer should enforce approved workflow sequences, mandatory validation, human approvals, separation of duties, restricted actions, exception routing, retry limits, escalation conditions, and timeouts and termination rules.

This becomes especially important when multiple agents collaborate. An agent receiving instructions from another agent should not automatically trust them. The receiving agent must verify identity, authority, context and requested action.

Emerging agent protocols can improve interoperability, but technical connectivity does not create business authorization.

5. Comprehensive observability

Traditional application monitoring asks whether a system is available and performing as expected. Agent observability must answer more detailed questions:

• What goal was the agent assigned?
• Which information did it retrieve?
• Which reasoning path or plan did it follow?
• Which model was used?
• Which tools did it invoke?
• Which permissions were applied?
• Which actions were completed?
• Which actions failed?
• When did a human intervene?
• What did the process cost?
• Did the result achieve the intended outcome?

Logs must connect model interactions, agent decisions, tool calls and business transactions into a traceable sequence. The objective is not to expose every internal model token. It is to provide sufficient evidence to investigate decisions, understand behaviour, support audit and improve performance.

Microsoft's guidance for agent deployments similarly emphasizes trust, traceability, auditable actions and visibility into data handling and decision-making.

6. Runtime control and intervention

An enterprise must be able to intervene while an agent is operating. Useful controls include pause, stop, revoke access, reduce authority, require human approval, block a tool, route to a fallback process, isolate an agent, reverse an action, and suspend an entire agent category.

This is the agent equivalent of operational resilience. If an agent begins behaving unexpectedly, the organization should not need to wait for a code release or vendor response before restricting it.

Human oversight must be designed around consequence

"Human in the loop" is often used as a universal answer to agentic risk. But requiring human approval for every action can eliminate much of the value of automation. Removing human review entirely can create unacceptable exposure. The right level of oversight depends on consequence.

Business consequence	Appropriate oversight
Low	Autonomous execution with monitoring
Moderate	Execution within limits; human review for exceptions
High	Human approval before consequential action
Critical or irreversible	Multiple controls, separation of duties and explicit authorization

The organization should consider financial impact, customer impact, safety implications, regulatory exposure, reversibility, data sensitivity, brand and reputational impact, and the potential for actions to cascade.

Human review should be applied at the most consequential decision point, not added indiscriminately to every step.

Agent accountability requires more than a technical owner

Every enterprise agent should have accountable ownership across five dimensions.

Business owner

Owns the outcome, process and value case.

Agent product owner

Owns the roadmap, performance and continuous improvement of the agent.

Platform owner

Owns the underlying agent, model, integration and observability services.

Risk and control owner

Defines the risk classification, control requirements and approval conditions.

Human sponsor

Remains accountable for the agent's purpose, access and lifecycle.

Microsoft's current guidance for an enterprise agent centre of excellence similarly identifies responsibilities spanning portfolio ownership, risk, security, data governance, platform operations and architecture.

An enterprise agent control record

Before an agent moves into production, the enterprise should be able to document:

Control dimension	Required definition
Purpose	The business outcome the agent is authorized to support
Owner	Accountable business and technology owners
Identity	The unique identity used by the agent
Autonomy	The approved level of independent action
Data	Information the agent can access and retain
Tools	Approved APIs, applications and other agents
Permissions	Specific actions and limits
Human oversight	Decisions requiring approval or review
Evaluation	Quality, security and behavioural tests
Monitoring	Production metrics and risk indicators
Economics	Cost limits and expected business value
Incident response	Pause, revoke, investigate and recovery process
Lifecycle	Review, renewal and retirement dates

This should become part of the agent's product lifecycle, not a document created only for audit.

Common failure patterns

Treating an agent as an advanced chatbot

A conversational interface does not reveal the full authority operating behind it. Agents must be classified according to the actions they can take.

Giving an agent a broad service account

This increases exposure and weakens attribution. Every agent should have a governed identity and minimum necessary access.

Connecting agents directly to core systems

Agents should use approved APIs and integration services with explicit transaction boundaries.

Trusting all instructions in retrieved content

Documents, messages and external sources may contain malicious or misleading instructions. Agents must distinguish trusted system policy from untrusted content.

Testing only the happy path

Agent testing must include incomplete data, conflicting instructions, unavailable tools, repeated attacks, unexpected user behaviour and attempted policy violations.

Assuming a human can always intervene

An agent may complete actions faster than a person can identify a problem. Boundaries must be technically enforced.

Launching without an exit mechanism

Every production agent requires a way to pause activity, revoke permissions and route work to a safe alternative.

Where enterprises should begin

Organizations do not need to start with highly autonomous, cross-enterprise agents. A more responsible progression is:

Begin with bounded processes

Select work with clear rules, defined data, measurable volume and reversible actions.

Introduce explicit approval points

Allow the agent to prepare work before gradually extending execution authority.

Establish identity and tool controls early

Do not build pilots using access patterns that cannot be retained in production.

Measure both value and behaviour

Track operational outcomes alongside agent accuracy, exceptions, interventions and policy violations.

Expand autonomy only after evidence

Increased authority should be earned through demonstrated performance and control maturity.

Questions executives should ask before approving an agent

1. What business outcome does the agent own or support?
2. What autonomy level is being requested?
3. Which actions can it take without human approval?
4. What is the consequence of an incorrect action?
5. Which identity will the agent use?
6. Who sponsors and owns its lifecycle?
7. Which systems, data and tools can it access?
8. How are financial and operational limits enforced?
9. Can every action be traced to its source and authority?
10. How will the agent be tested against misuse and unexpected conditions?
11. How can access be revoked immediately?
12. Which actions can be reversed?
13. Who handles exceptions and incidents?
14. How will performance and value be measured?
15. What evidence would justify increasing — or reducing — its autonomy?

The executive takeaway

Copilots largely extend human capability. Agents begin to extend organizational authority.

That is why agentic AI should not be treated simply as the next software feature. It represents a new category of enterprise participant — one that can interpret context, use tools and initiate business activity.

Enterprises should welcome the productivity and service opportunities agents create. But they should do so with a clear principle:

No autonomy without identity. No action without authority. No scale without accountability.

The organizations that succeed with agents will not necessarily be those that deploy the largest number. They will be those that establish a trusted path from assistance to action — and increase autonomy only as architecture, evidence and organizational confidence mature.