The Accountability Gap: Why Your Autonomous AI Agents Need a Manager, Not Just a Tech Stack

By the Editorial Desk, adapted from insights by Rick Folgmann, COO at Get Covered
Published in Tech & Enterprise Governance

Main Facts

In the fast-evolving landscape of enterprise artificial intelligence, a silent operational crisis is brewing. While corporate boardrooms obsess over model parameters, compute power, and algorithmic efficiency, they are largely ignoring the most critical vulnerability of the generative AI era: accountability.

According to recent data from Deloitte’s State of AI in Enterprise survey, 23% of companies are already utilizing agentic AI moderately today, and an overwhelming 74% expect to integrate autonomous agents into their core workflows within the next two years. However, this rush toward automation is crashing against a harsh market reality. Research firm Gartner has projected that more than 40% of all agentic AI projects will be outright canceled by the end of 2027.

Why are these promising initiatives failing? According to industry leaders like Rick Folgmann, Chief Operating Officer at insurtech firm Get Covered, the bottleneck is no longer technical. Building functional AI agents is easier than ever; the true challenge lies in determining who these digital workers report to.

In regulated environments—such as property management, finance, and insurance—software mistakes carry real legal and financial consequences. Whether a missed security deposit deadline in California results in a penalty of up to twice the deposit due to "bad faith," or an erroneous compliance filing triggers a regulatory audit, the law does not care whether the infraction was committed by a human employee or a piece of software. Someone, eventually, has to answer for it.

Chronology and Evolution: From Static Scripts to Autonomous Agents

To understand how enterprises arrived at this precipice, it is necessary to trace the evolution of workplace automation over the past two decades.

• Early 2000s (The Rule-Based Era): Enterprise automation was rigidly deterministic. Systems operated strictly on "if-then" logic. If a database field matched a specific string, an automated email was sent. These programs required zero judgment, possessed no contextual awareness, and rarely broke compliance because they could not deviate from their programmed rails.
• The 2010s (The Machine Learning & Underwriting Era): Predictive models began entering core business operations, such as credit scoring, fraud detection, and risk assessment. While these models could process vast amounts of historical data to output probabilities, their black-box nature created friction. Financial institutions quickly learned that every automated credit or insurance decision ultimately had to be explainable to a customer, an auditor, or a regulator. "The model did it" was never an acceptable defense.
• The 2020s (The Generative and Agentic AI Shift): The introduction of Large Language Models (LLMs) and autonomous agents shattered the boundaries of traditional software. Unlike simple calculators or predictive models, modern AI agents can execute multi-step workflows, interact with third-party software APIs, parse unstructured text, and make independent tactical decisions across millions of transactions. Get Covered, for instance, now deploys 16 distinct AI agents managing operations across more than 3 million rental units.
• The 2026–2027 Crossroads: As agentic AI scales from pilot projects to core enterprise infrastructure, organizations are realizing that technical capability has outpaced governance. The impending crash predicted by Gartner for late 2027 marks the inevitable reckoning where companies must transition from treating AI as a "cool tech tool" to managing it as a regulated workforce.

Supporting Data and Industry Metrics

A deep dive into recent enterprise research highlights the stark dichotomy between organizations that successfully harness AI and those that stumble into costly failures:

• Adoption Velocity: Deloitte’s 2026 enterprise insights reveal that nearly a quarter of businesses have already adopted agentic AI, with three-quarters planning deployment in the immediate future. This represents one of the fastest enterprise technology adoption curves in modern history.
• The High Failure Rate: Gartner’s forecast that over 40% of agentic projects will face cancellation by 2027 underscores a severe management deficit. The projects aren’t failing because the underlying neural networks are flawed, but because organizations fail to define business value or establish risk boundaries.
• The Workflow Redesign Premium: Research from McKinsey & Company’s State of AI tracking indicates that only about 6% of surveyed companies are currently seeing significant bottom-line returns from their AI investments. What separates this elite group from the rest? They are nearly three times as likely to have fundamentally redesigned their underlying operational workflows rather than simply pasting AI over broken legacy processes.

According to Folgmann, this success metric comes down to a fundamental rule of thumb: If a new human hire couldn’t follow your description of a workflow on their very first day on the job, an AI agent won’t be able to follow it either.

Official Perspectives and Operational Frameworks

As enterprises grapple with how to manage this new class of digital labor, operational leaders are developing blueprints to bridge the gap between software deployment and corporate governance.

1. Start with Work That Has an Existing Rulebook

The most successful early deployments of agentic AI occur in domains where the operating rules are already explicitly documented outside anyone’s head—such as legal statutes, contractual clauses, or vendor insurance mandates.

For example, property management teams traditionally spend upwards of three hours per property every week chasing down vendor certificates of insurance (COIs). While tedious, the task is completely specified. By delegating this to an agent, organizations gain two major advantages:

• The agent has an objective standard against which to verify its output.
• Internal auditing teams can review the final results without getting bogged down in subjective judgment calls.

2. Calibrate Autonomy by Workflow

A common misconception among technology executives is that "autonomy" is a binary, fixed property of an AI product. In reality, a mature deployment treats autonomy as a sliding scale tailored to individual workflows.

At Get Covered, agents operate under varying levels of freedom depending on the sensitivity of the task. A routine reminder sent to a vendor whose certificate of insurance expires next month is fully automated. Conversely, responding to a resident requesting an emotional support animal—a sensitive, legally nuanced process—is never fully automated and always routes through human review.

Leadership teams are advised to evaluate every automated workflow by asking a simple question: If this automated action goes wrong, how long until we find out, and what will the financial or reputational cost be by then? Quick detection and low cost earn an agent broader autonomy; high-risk, high-cost scenarios demand strict human-in-the-loop controls.

3. Give Every Agent an Owner with a Name

In traditional corporate structures, every budget, department, and software license has a designated human owner. Yet, many organizations allow multiple departments to spin up AI agents in silos, leading to an invisible web of contradicting software actions.

To prevent systemic failures, every operational AI agent must have an assigned human manager. This individual is responsible for reviewing error logs, adjusting autonomy boundaries as capabilities evolve, and directly answering for the agent’s behavior to internal stakeholders or regulators.

Implications for Decision-Makers Heading Toward 2027

The trajectory of enterprise software is clear: foundation models will continue to improve, become cheaper, and handle increasingly complex multi-modal tasks. However, a company’s internal clarity—its understanding of who decides what, and who bears responsibility when things go wrong—will not improve on its own.

Organizations that treat their AI agents like unregulated sandbox experiments will spend the next few years locked in crisis management, explaining to clients, boards, and regulators why their software took an unauthorized action.

Conversely, businesses that treat AI agents like human employees in highly regulated roles—enforcing clear rulebooks, defining strict operational scopes, and assigning named human managers to oversee performance—will unlock unprecedented scale. They will be able to justify expanding their agents’ responsibilities over time because they will possess the audit trails and governance frameworks to prove it.

Ultimately, your AI agents do not just need better code. They need a chief operating officer, a clear mandate, and a manager who checks their work.