Tool descriptions can explain what an action does without showing whether it is reliable at the moment an agent needs to use it. A new study calls this gap “operational blindness” and tests whether giving models a concise, verified snapshot of system conditions can improve their choices before a tool call is dispatched.

The proposed approach, Agent Harness Awareness, gives the model selected, fresh information about execution conditions while leaving the host system in Control of measurement, enforcement, revalidation and dispatch. The model can use the snapshot to choose an eligible action, but cannot change the telemetry or bypass the host’s safeguards.

In a controlled benchmark called ToolRoute, researchers compared tool choices across 216 decisions involving GPT-5.6 Sol, Claude Sonnet 5 and Gemini 3.7 Flash. The task and candidate tools were held constant while the information provided to the models varied. Against a control with the same telemetry-style interface but neutral values, verified tool-health information cut mean observable policy regret from about 4,252 milliseconds to 131 milliseconds, a 96.9% reduction. Single-dispatch completion rose from 69.4% to 95.8%.

The study also compared the verified-state condition with a task-only baseline. Mean regret fell by 97.3%, from about 4,826 milliseconds to 131 milliseconds, while completion increased from 65.3% to 95.8%. Each model completed 23 of 24 decisions when given verified state. All 69 active-route selections in that condition matched the route judged best using the available operational information; the remaining three decisions chose to wait when both routes were degraded.

A separate 27-decision replication tested the approach through live local HTTP execution, with OpenTelemetry client spans and faults injected using Toxiproxy. In that test, verified state reduced mean regret from about 8,956 milliseconds to zero and completed all nine live actions. The neutral-control condition completed five of nine.

The authors limit their findings to tool and network health in the tested setup. They identify runtime conditions, hardware pressure, costs, recovery after failures and selection among different providers or data sources as possible extensions that still require empirical evaluation. The studies test whether current host telemetry can improve an agent’s initial operational choice; they do not establish results for those broader scenarios.

Sources: arXiv cs.AI.