For most of the past decade, automation in operations meant bots that followed fixed scripts. When a script broke, an IT team fixed it, and operations managers rarely thought of bots as part of their workforce. AI agents are different. They read unstructured requests, make judgment calls within limits, escalate when unsure and improve or degrade as their inputs change. In a growing number of service desks, finance teams, claims units and customer operations, they now handle a meaningful share of the daily workload.
That changes the job of operations leaders. McKinsey’s State of AI in 2026 found that about 40% of large organizations are scaling AI agents, and Gartner expects that by 2028, 15% of day-to-day work decisions will be made autonomously. Yet most operating models still manage people and automation separately, with different metrics, different owners and different review rhythms. This playbook adapts familiar operational excellence disciplines (work allocation, capacity planning, quality control, daily management and continuous improvement) to a hybrid workforce of people and AI agents.
Why Agents Need to Be Managed Like a Workforce
It is tempting to treat AI agents as software: deploy, monitor uptime, patch when broken. That view misses how agents actually behave in operations.
- Their performance varies with the work mix. An agent that resolves routine requests reliably may struggle when a new product launch changes the type of inquiries it receives.
- They make judgment calls. Unlike rule-based bots, agents decide how to interpret a request and which actions to take, so quality must be measured on outcomes, not just on whether the system ran.
- They create work for people. Every escalation, exception and correction lands in a human queue. Poorly tuned agents can increase total workload even while they appear to handle high volumes.
- They have a variable cost. Model usage, tool calls and retries mean that cost per case can change with volume, complexity and design choices.
- They drift. Changes in data, upstream systems, policies or underlying models can shift behavior without any change to the agent’s own configuration.
These characteristics look much more like the management challenges of a team than of a server. Operations leaders already have the tools to manage them; they need to extend those tools to cover agents explicitly.
Designing the Work: Who Does What
The starting point is work design, not technology. In a 2026 series for Modern Distribution Management, Bart Tessel argued that AI should support decisions while accountability for business outcomes stays with domain experts. That principle translates directly into how work is divided.
Segment the work
Break the process into task types and segment by volume, variability, risk and reversibility. A useful grid has four zones:
- Agent-led: high-volume, low-risk, reversible tasks with clear policies, such as order status inquiries or password resets. Agents complete them end to end with sample-based human review.
- Agent-prepared, human-decided: tasks where the agent gathers information and proposes an action, and a person approves, such as refunds above a threshold or vendor master data changes.
- Human-led, agent-assisted: complex or sensitive work, such as complaints, negotiations or vulnerable customers, where agents retrieve context and draft but people own the interaction.
- Human-only: decisions that set policy, involve significant discretion or carry legal or ethical weight.
Gartner’s advice to match the tool to the need applies here: conventional automation for deterministic steps, assistants for retrieval, and agents only where multi-step decisions add value. Not every task in the agent-led zone needs an agent; some are better handled by simple workflow rules.
Define handoffs explicitly
Most problems in hybrid teams occur at handoffs. Specify what context an agent must pass when it escalates (the request, what it checked, what it concluded and why it stopped), and where escalations land. A person should never have to repeat the agent’s work to understand a case.
Capacity Planning for a Hybrid Team
Workforce management teams forecast demand, convert it into required hours and schedule people to meet service levels. Agents change every step of that calculation.
- Forecast by task type, not total volume. The share of work agents can handle depends on mix. A forecast that predicts a rise in complex cases implies more human capacity even if total volume is flat.
- Model containment, not just deflection. Count a case as contained by an agent only if it does not return as a repeat contact, complaint or correction within a defined window.
- Plan for escalation load. Escalated cases are usually harder than average, so they take longer per case than the historical human average. Capacity plans that use old handling times will understaff.
- Include oversight work. Sample reviews, approvals, exception handling and agent tuning all consume human time and must appear in the plan.
- Plan for agent outages. If an agent is paused because of an incident or a model issue, its volume falls back on people. Keep a documented fallback plan and a view of how much surge capacity exists.
The result is a combined capacity model in which agent throughput, containment rates and escalation volumes are inputs alongside human handling times and shrinkage. Reviewing it weekly, as most operations do for staffing, keeps both sides of the workforce aligned with demand.
Quality Control: Applying Proven Methods to Agents
Operations teams have long used quality sampling, calibration sessions and statistical process control. These methods transfer well to agents, with some adjustments.
Outcome-based quality sampling
Review a statistically meaningful sample of agent-handled cases every week, scored on the same rubric used for people: correct resolution, policy compliance, tone and completeness. Stratify samples by task type and risk, oversampling high-impact categories. Automated evaluation using a second model can increase coverage, but human review should remain the reference standard and should be calibrated regularly.
Control charts for agent behavior
Track key agent metrics, such as escalation rate, policy exception rate, repeat contact rate and cost per case, on control charts. Statistical process control distinguishes normal variation from signals that something has changed, such as a model update, a new upstream data issue or a shift in customer behavior. Agree in advance what happens when a metric breaches its control limits: who investigates, and whether the agent’s autonomy is reduced while they do.
Error taxonomy and root cause
Classify agent errors consistently: misunderstanding the request, retrieving wrong or outdated information, applying the wrong policy, tool failures, and inappropriate actions. A shared taxonomy turns individual corrections into improvement data. The OpEx90 commentary on the future of Lean Six Sigma makes the point that AI workflows themselves should be improved with DMAIC, and an error taxonomy is the measurement foundation for doing so.
Manage agents with the same discipline you apply to people: clear work design, capacity plans, quality sampling, control charts and daily reviews. What gets measured as one workforce gets improved as one workforce.
Metrics for One Workforce
A common failure is to report agent metrics in an IT dashboard and human metrics in an operations dashboard, with no view of the combined outcome. A hybrid scorecard should show the end-to-end process first, then the contribution of each part of the workforce.
- End-to-end outcomes: resolution time, first-contact resolution, customer satisfaction, error and rework rates and cost per case for the whole process
- Agent contribution: share of volume handled end to end, containment rate, escalation rate and reasons, quality score from sampling and cost per agent-handled case
- Human contribution: handling time and quality for escalated and human-led work, approval turnaround and time spent on oversight
- Interaction health: context quality on escalations, rework caused by agent errors and the share of escalations that were avoidable
Cost deserves particular attention. McKinsey’s 2026 survey found that about one in five respondents said AI operating costs, including tokens, had limited their AI use. Tracking cost per case by channel and task type, for both people and agents, shows where agents genuinely lower unit costs and where they merely shift cost into model consumption and human rework.
Daily Management and Roles
Lean daily management provides a ready-made rhythm for hybrid teams. Tiered huddles review yesterday’s performance, today’s risks and open problems. Agents simply need a place on the board.
- Include agent metrics in tier 1 and tier 2 huddles, alongside staffing and backlog, so that frontline leaders see the whole workload
- Review notable agent errors and escalation patterns daily, and route systemic issues to the improvement backlog
- Agree clear authority for team leaders to reduce an agent’s scope, for example routing a task type back to people, when quality signals deteriorate
- Hold a weekly calibration session in which reviewers, team leaders and the agent’s technical owner score sample cases together
New and changing roles
Hybrid operations create roles that many organizations have not yet formalized. An agent operations lead owns the performance of agents in a process, much as a team leader owns a team. Quality analysts extend their sampling to agent-handled work. Knowledge managers become critical, because the policies and knowledge articles agents rely on must be accurate and current. Frontline staff increasingly handle the hardest cases and supervise agent proposals, which calls for different skills, training and, often, recognition in role design and pay.
Change Management: Bringing People Along
Introducing agents into a team changes people’s work in ways that are easy to underestimate. The routine cases that once provided a steady rhythm move to agents, leaving staff with a higher concentration of difficult, emotionally demanding or ambiguous work. Some employees worry about job security; others become frustrated when they have to clean up agent mistakes. Ignoring these dynamics undermines both adoption and quality.
Several practices help:
- Be explicit about what the agent will and will not do, and how capacity freed by agents will be used, whether for backlog reduction, new services, growth or redeployment
- Involve experienced staff in designing agent workflows, writing policies and reviewing samples; their tacit knowledge is often what makes an agent reliable
- Give frontline staff an easy way to flag agent errors and show them how their feedback leads to changes
- Rebalance workloads and targets to reflect the harder mix of cases people now handle, rather than holding them to historical productivity norms
- Recognize new skills, such as agent supervision, exception handling and knowledge management, in role definitions and career paths
Teams that see agents as colleagues that take on tedious work, rather than as a threat or an unreliable tool, are far more likely to surface problems early and help improve the system.
A Worked Example: A Shared Services Finance Team
Consider a generic shared services center handling supplier inquiries and invoice exceptions for a multinational group. It deploys an AI agent to answer supplier payment status questions and to resolve simple invoice mismatches, such as small price variances within tolerance.
In the first months, the agent handles a large share of supplier emails, and the team celebrates. But the combined scorecard tells a more nuanced story. Repeat contacts from suppliers rise because some agent answers are technically correct but incomplete, and escalated invoice exceptions take longer than before because the agent passes little context. Human workload has not fallen as expected.
The team applies operational excellence methods. Quality sampling and an error taxonomy show that most repeat contacts come from two inquiry types where the agent lacks access to remittance details. Access is added and the response template is improved. Escalation handoffs are redesigned so that the agent passes the invoice, purchase order, goods receipt, its comparison and the reason it stopped. Capacity plans are updated to reflect longer handling times for escalated cases, and the agent’s metrics are added to the daily tier board with control limits.
Within a few cycles, repeat contacts fall, escalated cases are resolved faster and the team can redeploy capacity to supplier onboarding, which had been backlogged. The key change was not a better model but treating the agent as part of a managed workforce.
Pitfalls, Outlook and Conclusion
Pitfalls to avoid
- Measuring agent volume handled rather than cases genuinely resolved
- Using historical handling times to staff escalated work, which is harder than average
- Leaving agent oversight work out of capacity plans
- Reporting agent and human performance in separate dashboards with separate owners
- Allowing knowledge bases and policies that agents rely on to go stale
- Expanding agent autonomy based on demos rather than sustained quality evidence
Outlook
As agents become more capable and more common, hybrid workforce management will become a core operations competency, much like workforce management for contact centers became one a generation ago. Expect workforce management and quality tools to add native support for agents, and expect regulators and auditors to ask how organizations supervise the agents acting on their behalf.
The organizations that benefit most will not necessarily have the most advanced agents. They will be the ones that manage people and agents as one workforce, with clear work design, realistic capacity plans, disciplined quality control and a daily rhythm that turns problems into improvements. These are familiar operational excellence disciplines. Applying them to agents is how operations leaders turn AI from a promising pilot into a reliable part of how work gets done.
How Hanoi Technology Can Help
Hanoi Technology helps operations teams design hybrid human and AI agent workflows, build quality and capacity metrics, and run agents as managed services.
Further Reading
- The state of AI in 2026: On the road to ROI – McKinsey & Company
- Gartner: More than 40% of agentic AI projects will be canceled in the next few years – SD Times
- Lean Six Sigma Was Early in Distribution. AI Makes It Real (Part 1 of 4) – Modern Distribution Management
- The Future of Lean Six Sigma: How Automation, AI and Synthetic Agents Are Redefining Process Excellence – Purdue TAP