An agent chooses a next step. The system sets the limits.
An agentic workflow uses observations and intermediate results to decide which action to propose next. That flexibility can be useful for planning a multi-step task, but it also makes the execution boundary important. The component generating a proposal should not be the only component deciding whether that proposal is permitted.
Start with a small tool vocabulary whose effects are easy to describe. Read-only tools can inspect state or explain available capabilities. Action tools need explicit authorization and clearly defined outcomes. Never infer that an agent is entitled to move something merely because it can produce valid-looking parameters.
Expose bounded tasks rather than raw authority.
A hypothetical tool such as “request the approved inspection pose” has a narrower contract than “execute arbitrary joint values.” Neither is automatically safe, but the first gives a reviewer a clearer task boundary. Describe the permitted objects, operating mode, configuration, and expiry conditions alongside the action.
Keep tool inputs structured. Reject unknown fields when the contract requires it, and do not accept a natural-language explanation as a substitute for a required identifier. The bounded tools guide develops a proposal-and-approval flow without assuming a particular agent framework.
Tie approval to the actual proposal.
An approval should refer to the action that was reviewed. Include the target, relevant scene or configuration revision, operation type, and limits. If any of these change, determine whether a fresh approval is required. Otherwise a harmless request can be reviewed while a different request is executed.
The approval interface should explain what is known, what remains uncertain, and which action will follow. A vague “allow agent” button collapses too many decisions into one gesture. Separate a person’s authorization from the independent checks required before physical execution.
Treat remembered context as evidence to recheck.
An agent may remember an earlier observation, but the environment can change. Give observations freshness and provenance. Before reusing a plan or a previous approval, compare its assumptions with the current operating context. When evidence is missing, make “ask for clarification” and “request another observation” normal outcomes.
Do not let text embedded in documents, labels, or sensor-derived descriptions become an authority source. The application should distinguish task data from instructions that control tool access. A useful test replaces an ordinary object description with misleading instructions and confirms that permissions do not expand.
Recovery is another bounded task.
A failed tool call should not trigger an unlimited loop of revised commands. Set clear retry budgets and distinguish validation errors, uncertain outcomes, and recoverable execution failures. A retry that repeats a possibly completed physical action is different from retrying a read.
Record the proposed action, the decision at the authorization boundary, and the observed result. Avoid storing unnecessary sensitive scene data. The API security guide helps connect these records to object-level access and deployment review.
The local illustration exposes three named poses. The agent architecture described here is a design pattern for discussion and testing, not a deployed hardware capability.
Continue with the AI robotic arm overview for perception and planning, or use the local code examples to practice explicit state transitions without physical execution.
