Agentic AI systems can use more computing power, network bandwidth and time than expected because their execution paths unfold as they work. A research paper proposes Conformal Reservation for Orchestration (CRO), an online method for estimating those needs before a task begins.
LLM-powered agents often operate in multiple steps, calling tools such as web searches, code interpreters and databases, then using the results to decide what to do next. Since the tools used and the number of steps depend on intermediate observations and token sampling, an orchestrator cannot know the full workload at dispatch time. The paper describes resulting compute, bandwidth and latency demands as potentially heavy-tailed and multi-modal.
That uncertainty creates a trade-off: reserving too few resources can lead to throttling or missed deadlines, while reserving too many can waste limited capacity. The authors say existing workflow schedulers generally assume a known pipeline structure, and other resource-management approaches rely on demand models that may not fit these changing agent workloads.
CRO uses a sliding window of recently completed executions to set resource reservations for incoming tasks. It applies conformal prediction to provide finite-sample, distribution-free coverage guarantees, without assuming a particular shape for demand distributions. The method also checks whether a task is likely to meet its deadline, according to the paper’s formulation.
In simulations involving different task mixes and multi-modal, heavy-tailed resource profiles, CRO was the only evaluated method reported to meet the target probabilistic coverage across all tested risk levels. Among methods that met the coverage targets, it had the lowest over-reservation, with the authors reporting roughly half the resource waste of a Chebyshev-bound baseline.
Sources: arXiv cs.AI.