A model call is often the least complicated part of an agent's work. The expensive part tends to begin after the model decides what to do next.

Picture an agent asked to find out why yesterday's sales figures do not match the finance dashboard. It reads a pipeline log, looks up a table definition, queries the warehouse, checks a source extract, and asks another service to validate the difference. The model itself may burn only a modest number of tokens along the way. The warehouse query, the repeated lookups and the failed attempts can account for far more of the bill.

Anyone who has operated data platforms for a few years will recognize this. The request a user sees is not the same thing as the work executed underneath it. Autonomous AI widens that gap, because the application decides at runtime which systems to call and how many times to call them. Nobody wrote that execution plan down in advance, so nobody reviewed what it would cost.

The useful economic unit is therefore not a token, and not a tool call. It is a task that reaches an agreed, verifiable result. That idea builds on the unit-economics practice used in cloud financial management, which ties spend to a business measure rather than to raw resource counts [4]. This article lays out a way to account for that cost, shows an illustrative calculation, and describes what I would instrument first.

The Cost Moves Downstream

Token counts remain useful for budgeting model usage. They are not a complete accounting system for an agent that talks to databases, search services, APIs, queues and workflow engines. An agent can write a perfectly valid SQL statement that scans far more data than necessary. A retry can run the same expensive query a second time. A verification step can trigger a separate reconciliation job that nobody budgeted for.

In production data engineering, I look at the query plan and the physical work before I admire a short SQL statement. The same habit belongs in agent design. A shorter prompt is not necessarily a cheaper workflow, and a lower-priced model is not necessarily the cheaper architecture.

Six Places the Bill Accumulates

When I break down the cost of a task, I use six categories, summarized in Figure 1 and Table 1.

Figure 1. Six Cost Layers of an Autonomous AI Task
Figure 1. Six Cost Layers of an Autonomous AI Task

These six categories reflect where the bill typically accumulates in an agentic workflow. Some costs are visible on a provider invoice, while others are buried inside shared infrastructure or downstream service execution. The point is not to mirror vendor billing categories exactly, but to establish a consistent management model for task-level attribution.

Table 1. Cost Categories for an Autonomous Task

Category What It Includes Typical Signal
Inference Input and output tokens, model-serving capacity, and model calls made during planning, execution, and evaluation Tokens, model calls per task
Data access Warehouse compute, object-store requests, search and vector retrieval, metadata lookups, and cross-system data transfer Bytes scanned, compute seconds, request counts
Tool execution Downstream work started by an API or tool, including serverless functions, business services, jobs, and external calls Invocations, duration, provider charges
Verification Reconciliation against authoritative records, deterministic rule checks, secondary evaluation, and necessary human review Checks run, reviewer minutes
Retries and recovery Incremental work after a failed or inconclusive attempt, including re-planning, re-querying, and checkpoint recovery Attempts per task, retry reasons
Orchestration and observability State storage, queues, schedulers, traces, audit logs, monitoring, and shared platform capacity Allocated platform cost, log volume

These are accounting categories, not vendor invoices.

These categories are not necessarily separate line items on anyone's invoice, and they must be mutually exclusive when costs are allocated. A warehouse query run during a retry should be counted once, not under both data access and recovery. One workable convention is to book first-attempt consumption in its resource category and classify only the additional consumption from later attempts as recovery.

A Better Denominator: Verified Outcomes

To address the limitations of inference-only cost accounting, I propose a task-level analytical framework based on Cost per Verified Outcome (CPVO). The framework combines six infrastructure cost categories with an explicit verification requirement, allowing teams to evaluate the total attributable cost of producing an accepted result rather than merely measuring model consumption. For a defined workload and measurement window, CPVO is calculated as follows:

CPVO = Total attributable cost of all task attempts ÷ Number of verified outcomes

The numerator includes unsuccessful attempts. The denominator excludes answers that merely look plausible, and workflows that return HTTP 200 without completing the business task. The acceptance test has to be written before the measurement, not after. For the sales-reconciliation example, it might require identification of the affected tables, a reproducible explanation of the variance, and confirmation against source records.

CPVO is a proposed management metric, not an industry standard. It does not replace latency, safety, correctness, or service-level measures. A cheap answer that arrives too late, or that changes the wrong records, is not an operational success.

An Illustrative Comparison

Consider 10,000 attempts at the same class of task under two hypothetical architectures. The figures are constructed to demonstrate the accounting method. They are not production benchmark results.

Table 2. Hypothetical Cost per 10,000 Task Attempts

Cost or Outcome Architecture A Architecture B
Model inference $1,200 $1,500
Data access $2,000 $900
Tool execution $1,500 $900
Verification $400 $500
Incremental retries $1,000 $300
Orchestration / observability $400 $400
Total attributable cost $6,500 $4,500
Verified outcomes 7,500 9,000
Cost per verified outcome $0.87 $0.50
Figure 2. Why Cost per Verified Outcome Can Reverse an Infrastructure Decision
Figure 2. Why Cost per Verified Outcome Can Reverse an Infrastructure Decision

Hypothetical figures for illustration only. CPVO = total cost divided by verified outcomes ($6,500 ÷ 7,500 ≈ $0.87; $4,500 ÷ 9,000 = $0.50).

Architecture B spends more on inference and still costs less overall. It avoids expensive retrieval and repeated execution, and it completes a larger share of tasks. The example does not show that a larger model always wins. It shows why model price alone cannot settle an infrastructure decision. Workload mix, verification rules, allocated fixed costs, and pricing assumptions must stay consistent on both sides of any real comparison.

Where Data-Platform Experience Matters

None of the following is new. These are controls that data teams already apply to shared platforms. The difference is that an agent can generate load at a pace and in shapes that a human analyst rarely does.

Make Generated Queries Behave Like Production Queries

Agent-written SQL deserves the same controls as any other untrusted workload: read-only roles where appropriate, statement timeouts, scan or compute limits, concurrency isolation, and review of query plans. Most platforms already expose these controls, for example per-query byte limits in BigQuery [5] and statement timeouts in Databricks SQL [6]. A syntactically correct query can still be a poor physical plan. Partition pruning, selective predicates, and sensible joins matter whether the SQL was written by an analyst or generated by a model.

Stop Rediscovering the Same Context

Repeated catalog searches and duplicate retrievals waste both time and money. Cache stable metadata where freshness requirements allow, keep useful intermediate results within the task, and pass compact references instead of copying whole documents into every step. Cache invalidation is not optional, though. A stale schema or an obsolete policy can make a cheap execution wrong.

Treat Retries as an Engineering Decision

Not every failure should be retried. A transient network timeout may justify backoff. A permission denial or a malformed query generally calls for a different response. Use idempotency keys for side-effecting operations, cap the number of attempts, preserve checkpoints, and record why each retry happened. Without that discipline an agent can turn one bad plan into several expensive executions.

Keep Verification Independent Where It Counts

For low-risk information retrieval, lightweight checks may be enough. For changes to business records or financial reconciliation, compare results with authoritative data and require approval where appropriate. A model confirming its own output is not independent validation.

What I Would Instrument First

Start with a task identifier that is propagated through inference calls, retrievals, database queries, tools, and verification. This makes the system economically legible: every downstream action can be joined back to the task that caused it.

Figure 3. Task-Level Observability for Autonomous AI Workflows
Figure 3. Task-Level Observability for Autonomous AI Workflows

Join those events to provider bills or resource-metering data. When precise attribution is not available, document the allocation rule instead of presenting an estimate as a measured charge.

Table 3. Minimum Capture for Task-Level Cost Accounting

Capture Why It Matters
Task / trace ID and workflow version Connects all work to one attempt and makes comparisons reproducible
Model usage and latency Separates inference cost from downstream delay
Query ID, scanned bytes and compute time Surfaces inefficient warehouse and storage access
Tool name, status and duration Shows expensive or unreliable integrations
Retry reason and attempt number Distinguishes transient failures from repeated bad plans
Verification result and acceptance rule Defines the outcome used in CPVO
Metered or allocated cost and its source Makes the economic calculation auditable

OpenTelemetry's generative-AI semantic conventions describe spans for inference, retrieval, agent workflows, and tool execution, and they are a useful interoperability reference. They are still at development status, so teams should verify the latest state of the specification before depending on attribute names, and avoid collecting sensitive prompt or tool payloads by default [1, 2, 3].

How to Run a Credible Evaluation

Choose one bounded workflow, such as incident triage or a read-only data-quality investigation. Freeze the task set and the acceptance criteria. Record a baseline over enough attempts to expose failures and long-tail latency. Then change one thing at a time: retrieval filtering, query guardrails, tool routing, caching, or retry policy. Compare total attributable cost, verified-outcome rate, p95 latency, and error categories.

Table 4. Common Ways an Evaluation Misleads

Pitfall Safeguard
One architecture receives easier tasks Run both on the same frozen task set
Weaker acceptance rules on one side Define one acceptance test before any run
Shared fixed costs ignored or hidden Disclose the allocation rule and apply it to both
Small sample treated as conclusive Report uncertainty and the number of attempts
Savings claimed before they are measured Claim a saving only after a comparable test confirms it

The Next Optimization Target Is the Whole Task

Inference will keep getting faster and cheaper, and that is worth having. It will not remove the cost of a badly planned warehouse scan, an unnecessary external call, or a loop that keeps retrying a broken operation. Autonomous systems inherit the strengths and weaknesses of the data platforms underneath them.

For enterprise teams, the practical shift is simple to state. Trace the entire execution. Charge resources to the task that caused them. Count a task as successful only when its outcome has been verified. The goal is not to minimize tokens at any cost. It is to deliver reliable work with predictable infrastructure consumption.

References

  1. OpenTelemetry, Semantic Conventions for Generative AI Spans (development status).
  2. OpenTelemetry, Semantic Conventions for GenAI Agent and Framework Spans (development status).
  3. OpenTelemetry Blog, Inside the LLM Call: GenAI Observability with OpenTelemetry, 2026.
  4. FinOps Foundation, FinOps Framework 2025.
  5. Google Cloud, Estimate and control costs, BigQuery documentation.
  6. Databricks, statement_timeout, Databricks SQL language manual.