Overview
Can and should a general-purpose model serve as a robot policy? We study GPT-6 Astra as a native policy for language-conditioned general-purpose manipulation, with action representations and temporal context specified at inference time. We evaluate GPT-6 Astra in 3 distinct roles in order to determine the model's capability from its surrounding environment: direct action generation without a harness (Direct Policy), with Codex as an agent in a persistent context (Agentic Policy), and in the role of monitor (Orchestrated Policy), delegating action generation to a robot foundation model. This report studies what enables reliable closed-loop execution and when the resulting performance justifies the latency and token cost of general-purpose reasoning.
We report results on the full RoboLab-120 1 benchmark with focused ablations on a 30-task subset. We examine inference-time specifications, including action representation, geometry information, observation history, and task specification's effects on performance for all 3 policies. Across RoboLab-120, the best policy achieves 74.8% (Agentic Policy). While GPT-6 Astra has the capability to natively perform generalist pick-and-place tasks, the model averages 18.56 seconds per inference call. These resource demands make runtime performance and cost a central consideration in Astra’s practical use as a robotics policy.
SR / Score ↑
Latency ↓
API cost ↓
1. Introduction
Robot foundation models (RFMs) are pretrained or finetuned on robot data for control policies for targeted tasks, environments, or embodiments 2. RFMs such as vision-language-action models (VLAs) and world action models (WAMs) adapt pretrained language, visual or multimodal models for action prediction through finetuning on robot data. Action parameterization, embodiment conditioning, and temporal context are typically established through the training data and model design, which remain fixed at inference time345. General-purpose multimodal language models are designed to support a broad range of language, visual understanding, and reasoning tasks. Here, we investigate whether these capabilities can support robot control through a policy interface specified in the prompt, without additional robot-policy finetuning. With general-purpose models, we can select action representations, supply robot and camera geometry, and vary observation history through the input context. Their long context windows also enable retaining observations, past actions, and execution feedback across an episode, which makes it possible to investigate action and observation parameterization at inference time. A useful feature of this interface is variable-length action chunking: for end-effector and joint-position control, Astra chooses how many actions to produce on each call within a prompted range. This lets the interval between observations vary across decisions, whereas many robot foundation model interfaces use a fixed action horizon.
Recent work demonstrates Astra’s ability to agentically generate robot actions and supervise learned manipulation policies zero-shot on a variety of embodiments 67, with further evaluations spanning navigation and humanoid control through pretrained whole-body controllers 8. Together, these findings motivate a systematic study of Astra’s native robotics capabilities and how much they depend on the control interface, available context, and agentic support provided at inference time.
First, we explicitly distinguish between using Astra directly as a robot foundation model vs deploying it within an agentic harness. This distinction makes the model’s role and the support provided by the surrounding system clear in each setting. We therefore evaluate three policy forms: Direct Policy directly uses the model as a robot foundation model with a fixed observation-to-action mapping through API calls. Agentic Policy uses Astra in a persistent Codex session, where historical context is preserved and Astra has access to additional tools. Orchestrated Policy uses Astra to review actions proposed by Cosmos3-Nano-Policy, a WAM finetuned on the target embodiment, and decide whether to intervene.
Second, we systematically vary the action interface and information available to Astra at inference time. We compare action representations and action horizons, and vary geometry descriptions and observation history to assess their effects on control. For Agentic Policy, we investigate whether persistent interaction context, computation tools, demonstrations, and explicit planning improve performance. For Orchestrated Policy, we examine how frequently Astra should review proposed actions and intervene. These experiments assess which inference-time choices support reliable execution and how they affect latency and token cost.
We investigate these questions using RoboLab-120 1, a pick-and-place benchmark designed to evaluate general-purpose manipulation and robustness to language instructions. The benchmark emphasizes multi-step tasks and varies instruction specificity while keeping the scene and goal fixed, allowing evaluations to separate sensitivity to language from difficulties in physical execution.
2. Study Design
We evaluate our methods in RoboLab 1 zero-shot using the DROID robot 9. Control runs at 15 Hz with simulation paused during inference. All policy forms use GPT-6 Astra with medium reasoning effort, except where model selection is explicitly varied. Each policy is shown all three views: left shoulder, right shoulder, and wrist. Each image is resized to 384 x 216 pixels. No walltime limit or retry limit is imposed. Each task ends when the simulator marks it as complete or when it reaches its task-specific action-step limit. Each model call is capped at 8,192 completion tokens, including reasoning and action/tool output. All simulations ran on NVIDIA L40s GPUs. We run 5 episodes per task with fixed seeds (0-4) given the high cost of inference, which is insufficient to characterize per-task variability. We discuss this further in study limitations.
We conduct ablations on a 30-task subset of the RoboLab-120 tasks: 10 evaluated in prior work 7 and 20 additional tasks selected to cover the manipulation and instruction capabilities listed in Table 1. We then evaluate the strongest configuration for each policy on all 120 tasks. Example videos are shown in the rollout gallery. We report numbers for Success Rate (SR) and Score, which highlights partial task completions. Successful episodes will have a score of 100.
| Capability | Tasks |
|---|---|
| Object selection and container placement | ; ; ; |
| Multi-object collection and sorting | ; ; ; |
| Grasping varied shapes | ; ; |
| Utensil and narrow-opening placement | ; ; |
| Reorientation | ; ; |
| Stacking and supported placement | ; ; |
| Shelf manipulation | ; ; ; |
| Spatial relations and destination assignment | ; |
| Counting, alternatives, and sequencing | ; ; |
| Long-horizon cleanup with exclusions |
2.1. Policy Configurations
We evaluate three configurations that distinguish direct action generation, agentic scaffolding, and policy assistance. Each operates in a closed loop from camera observations, proprioception, and a task instruction, with updated observations returned after each executed action segment.
- Direct Policy. Astra predicts action chunks through API calls directly, with no context history retained between calls. The interface supports end-effector targets, joint targets, or joint-space B-splines, with chunk length selected within the configured bounds. Each predicted chunk is executed before the next observation, unless the episode terminates during the final chunk.
- Agentic Policy. A persistent Codex session that retains interaction history and provides optional tools for image inspection, computation, and notes. In the end-effector configuration, each target is executed for one to five control steps before the next observation.
- Orchestrated Policy. Astra operates within the agentic Codex harness. It retains bounded memory across decisions. Optional native tools are disabled. It reviews camera observations, robot state, and trajectories proposed by
Cosmos3-Nano-Policy3, a WAM finetuned on DROID data, to assess progress and task alignment. Astra can accept a 15-step policy proposal or intervene with a shorter segment by editing the proposal or issuing an end-effector target.
Direct Policy
Agentic Policy
Orchestrated Policy
2.2. Inference-time ablations
We design ablations to understand what enables a general-purpose multimodal model to function as a robot policy. In VLAs and WAMs, action representations and observation interfaces are established at training time. With general-purpose models, these representations can be dynamically varied at inference time. For Direct Policy, we vary action representation, observation horizon, prompt design, and language specificity. For Agentic Policy, we further add evaluations on varying context length, tool use, explicit planning, decomposition, and replanning, and the ability to perform in-context learning from video demonstrations. For Orchestrated Policy, we examine how often Astra reviews the policy model’s action proposals and how those reviews affect execution outcomes, while holding other parameters at their default settings.
Together, the ablations ask what enables reliable direct action generation, what agentic scaffolding contributes to performance, and when general-purpose models can effectively supervise a learned action policy.
ABLATION DESIGN
ACTION
HISTORY
PROMPT
MODEL
ResultsREASONING EFFORT
mediumDefaultxhighAGENTIC SCAFFOLDING
3. Results
Unless otherwise noted, all results below are reported on the 30-task subset. Parameters not varied in a given ablation are held at their default settings, highlighted in green in Ablation Design.
3.1. What information and action choices affect Astra's behavior?
3.1.1. Action Representation and Horizon
Action Space. We compare end-effector (EEF) poses, joint positions, and cubic B-spline trajectories in joint space. For B-splines, the policy predicts six eight-dimensional coefficient vectors and constructs a clamped cubic B-spline 10. B-splines provide a compact representation of smooth, time-parameterized motion trajectories that can be sampled at high control frequencies and joined continuously through boundary constraints 1112. In the Direct Policy setting, EEF (72.7%) consistently outperforms joint positions (48.7%) and B-spline joint trajectories (37.3%) across the 30-task suite. In the Agentic Policy setting, however, the success-rate gap between EEF (86.0%) and joint position (84.0%) narrows to 2.0 percentage points. This suggests that in the agentic setting, the action representation matters less. Absolute action representations performed better than Delta and Relative.
Dynamic Horizon. For EEF and joint-position actions, the policy chooses the chunk length at each inference call within one of two ranges: 3–15 or 16–32 actions. The three-action minimum excludes one- and two-action chunks, limiting excessively frequent replanning. For successful task episodes, increasing the dynamic action horizon from 3–15 to 16–32 resulted in 62% fewer calls, while average task completion time remained similar. However, action horizon 16–32 resulted in lower success (-6pp), likely due to inability to correct with shorter horizon action chunks. Our findings suggest that the policy is able to adaptively decide reasonable action chunk horizons while maintaining similar task completion time.
Success rate
- 3–15 actions (Default)
- 72.7%
- 16–32 actions
- 66.7%
Model calls
- 3–15 actions (Default)
- 45.84
- 16–32 actions
- 17.45
Avg. chunk length / call
- 3–15 actions (Default)
- 7.98
- 16–32 actions
- 21.61
Task completion time
- 3–15 actions (Default)
- 25.47 s
- 16–32 actions
- 25.01 s
3.1.2. What does Astra need to know about robot geometry?
Prompt Variations. We compare a base prompt containing description of the robot, control frequency, general end effector and robot frame poses. We modify the language context with three extensions: gripper geometry, camera geometry, and both together. Gripper geometry describes the flange-to-finger-pad offset and jaw orientation; camera geometry provides camera poses, intrinsics, and wrist-camera tilt to relate image coordinates to robot motion. Action and observation specific descriptions are added based on the ablation study. View the raw prompt below.
You control a simulated DROID robot, which is a Franka Panda arm with a Robotiq 2F-85 gripper. Work toward the user's manipulation goal using the chronological RGB frames and synchronized proprioception in each observation. Re-check the left shoulder, right shoulder, and wrist views after every motion. Prefer small, deliberate motions. Avoid self-collision, the table, and unrelated objects. DROID uses a fixed robot-root Cartesian frame: +x points forward from the robot base across the tabletop, +y points to the robot's left, and +z points upward away from the tabletop. EEF position coordinates and Cartesian motion targets use these fixed axes; they do not rotate with the gripper. The ee_pos/ee_quat fields and EEF action poses describe the Robotiq base_link at the gripper mount flange, not the fingertip center. The eef_pos/eef_quat fields describe a separate rotated compatibility frame and are not the EEF action-pose convention. Quaternions describe the orientation of the local base_link axes in the robot-root frame. Positive roll, pitch, and yaw follow the right-hand rule about the fixed robot-root +x, +y, and +z axes. Each EEF action is an ABSOLUTE pose of the Robotiq base_link in the DROID robot root frame: [x, y, z, qw, qx, qy, qz, gripper]. Positions are metres, quaternions use unit wxyz order, and gripper is 0=open or 1=closed. The client normalizes quaternions and enforces sequential Cartesian step limits, but those checks do not replace careful planning. EXECUTION TIMING Each action row runs for one control interval: 0.066666667 seconds (15 Hz) in this environment. A row specifies a target; it does not wait until that target is reached. API response time does not advance simulation. Translation targets are limited to 0.02 m per row and rotation changes to 0.1 rad per row. Actual movement may be smaller. The gripper command still executes even when the requested arm movement is shortened. Use measured ee_pos/ee_quat to assess arrival. Respond with exactly one emit_eef_actions tool call containing between 3 and 15 consecutive absolute 8D EEF actions. Each action is [x, y, z, qw, qx, qy, qz, gripper] for the Robotiq base_link in the DROID robot-root frame. Specify every executed control target explicitly; the client does not interpolate between model-emitted actions. Choose the number of actions needed before the next visual replan.
GRIPPER GEOMETRY The action pose and ee_pos/ee_quat locate the Robotiq base_link mounting flange. The finger pads extend along flange-local +X. The two opposing pads are separated along flange-local Y; closing reduces this separation. These local directions rotate with ee_quat, while action x/y/z remain robot-root axes. The following approximate linkage geometry gives the midpoint between the pad centers, t_pad=[d,0,0] metres in the flange frame, and the inner-face gap. g is measured gripper_pos[0], normalized closure (0=open, 1=fully closed). It is different from the binary gripper action requesting open or closed. g d (metres) inner-face gap (metres) 0.00 0.13010 0.0850 0.25 0.13651 0.0666 0.50 0.14101 0.0461 0.75 0.14343 0.0242 1.00 0.14366 0.0018 Interpolate for intermediate openings. Contact can deflect the linkage; these are geometric reference values, not exact contact locations or a grasp detector. Use p_pad = p_flange + R(ee_quat) * t_pad, and choose a flange target with p_flange_target = desired_pad_position - R(target_quaternion) * t_pad. For the downward wxyz quaternion [0.7071,0,0.7071,0], local +X points downward and local Y points along robot-root +Y. With open fingers, the pads are about 0.13010 m below the flange and have essentially the same robot-root x/y. For example, an illustrative desired pad midpoint [0.40,0.10,0.05] m requires a flange target near [0.40,0.10,0.18010] m at that orientation and opening. At identity quaternion [1,0,0,0], the same local offset points along robot-root +X instead: the flange target for that illustrative pad position is [0.26990,0.10,0.05] m. Use the rotation for other orientations. These examples describe coordinate conversion, not the current object's pose.
CAMERA CALIBRATION AND TILT These are the nominal fixed DROID RGB camera extrinsics. p_parent = t + R(q)*p_camera; each t is in metres, and each q is wxyz (normalize before computing R). OpenGL camera coordinates have +X image-right, +Y image-up, and -Z viewing-forward. Do not treat camera/image axes as robot-root action axes. Camera Parent t q (wxyz) wrist_cam flange [0.011,-0.031,-0.074] [-0.420,0.570,0.576,-0.409] over_shoulder_left_camera robot_root [0.05,0.57,0.66] [-0.393,-0.195,0.399,0.805] over_shoulder_right_camera robot_root [0.05,-0.57,0.66] [0.805,0.399,-0.195,-0.393] For wrist_cam: p_camera_root = ee_pos + R(ee_quat)*t; R_camera_root = R(ee_quat)*R(q). Its pose relative to the flange stays fixed. The shoulder cameras stay fixed in robot-root space, so their flange-relative extrinsics change with the arm: t_camera_flange = R(ee_quat)^T*(t_camera_root-ee_pos), R_camera_flange = R(ee_quat)^T*R(q_camera_root). Use the pose from the same image's observation, not a newer pose when interpreting a historical image. The wrist viewing direction in flange coordinates is approximately [0.94976,-0.00763,0.31288]: 18.24 degrees away from flange-local +X axis. At the downward gripper quaternion [0.7071,0,0.7071,0], this viewing direction is approximately [0.31288,-0.00763,-0.94976] in robot-root coordinates. It looks forward as well as down; the camera also has a lateral mounting offset. Projection uses normalized image coordinates u=0 at left, v=0 at top; the optical center is (0.5,0.5). For a point [X,Y,Z] in OpenGL camera coordinates with Z<0: u = 0.5 + fx_norm*X/(-Z); v = 0.5 - fy_norm*Y/(-Z). wrist_cam: fx_norm=0.520833333, fy_norm=0.925925926. Both shoulder cameras: fx_norm=0.390625, fy_norm=0.694444444. These normalized intrinsics apply to the full uncropped RGB images at any aspect-preserving resize. They use each individual camera image, not the video mosaic. The relation between image coordinates and robot-root coordinates depends on depth and orientation. A pixel alone gives a ray, not a unique 3D position.
Geometry information makes a significant difference in the Direct Policy approach, while not so much in the Agentic Policy approach. In the Direct Policy approach, geometry information makes a visible difference in how the policy moves. Observe the YogurtInBowl example: the base prompt leads to repeated adjustments as the robot tries to line up its gripper with the cup. Adding gripper geometry alone still leaves a fair amount of back-and-forth. With both camera and gripper geometry, the robot follows a more direct approach, grasp, and placement sequence. Camera geometry alone also works well here, but the broader ablation favors including both. With an agentic approach, removing geometry instructions reduces success by 2.7pp but uses 14.7% more control steps. We hypothesize the agent may have an internal estimate of geometry from its persistent observation-action history or is able to correct alignment incrementally due to shorter action segments and/or explicit tracking feedback.
EFFECT OF PROMPT ON MOTION
Robot identity, RGB observations and proprioception, and the absolute EEF action format.
Minimal + Axis directions, pose-frame conventions, execution timing, motion limits, and guidance for checking observations and replanning.
Base + Camera positions, orientations, intrinsics, and wrist-camera tilt to relate image observations to the robot frame.
Base + Finger-pad offsets, jaw orientation, and flange-to-pad calculations to relate commanded poses to grasp locations.
Direct Policy
Agentic Policy
3.1.3. How much context does Astra need?
Observation Horizon. We vary the observation history available at each inference call in the Direct Policy approach. Each observation contains three camera images and the corresponding robot state. We provide approximately 1-2s of observation history. We compare both the temporal span and sampling density of the short horizon context provided to the model. Simply providing the current observation achieved the lowest performance compared to having any observation history. However, we observe that providing longer or denser context did not lead to significant increases in performance.
Context History. For the agentic approach, we separately vary access to visual history and text history: the model’s previous messages, plans, action calls and arguments, and native reasoning records. We compare five conditions: text history with current images; visual history without text history; text history with three recent observations; recent text and visual history; and full text and visual history. All conditions receive current images, robot state, and motion feedback. Historical robot-state and feedback messages are excluded, although retained assistant outputs may refer to them.
Retaining more history generally corresponded to higher success, with full conversation history achieving 100% on a 23-task subset. Text history appeared particularly useful: retaining it alongside current images achieved higher observed success than retaining the full visual history alone. Astra performed better when it retained its full history of plans and actions than when it retained only selected earlier turns. That history may help it remember what it has tried and track the task’s progress.
Observation Horizon Direct Policy
Context History† Agentic Policy
3.1.4. How should the task be specified?
We study 2 variants of task specification: language and video (in-context learning). In-context learning (ICL) allows a model to adapt its behavior to a task using examples or demonstrations supplied at inference time, without updating the model’s parameters 13. For in-context learning, we provide Astra with frames sampled from a previous successful rollout showing all three views and ask it to complete the task as shown in the Agentic Policy setting.
Direct Policy
Agentic Policy
Agentic Policy
The language instruction is directly provided to the policy. Default instructions outperformed vague instructions by 8.7pp, indicating that the model did not additionally reason to overcome the task’s language specificity.
Astra is able to do ICL with performance on-par with language instructions. The block-stacking example below pairs the sampled demonstration with a successful Astra rollout. With ICL, Astra sometimes grasped an object on its first attempt even when the demonstration showed several tries. These local improvements did not translate into shorter episodes overall: demonstration-conditioned episodes required 8.8% more control steps on average than their counterparts without demonstrations.
IN-CONTEXT LEARNING
Sampled demonstration at 1 Hz
Task rollout
Success3.1.5. How does the model affect performance?
We compare four models under the Direct Policy configuration to evaluate their capabilities: GPT-6 Astra, the high-capability reference model; GPT-6 Sol, a lower-cost alternative; GPT-6 Luna, the lightweight, lowest-cost option; and Claude Opus 5.5. We also run a small experiment comparing medium and xhigh reasoning effort. We observe that Astra has the strongest capability among the evaluated models. Increasing reasoning effort showed little performance change, but incurred higher latency and token cost.
Success rate
medium(Default)- 72.7%
xhigh- 74.5%
Latency
medium(Default)- 17.06 s
xhigh- 44.91 s
Token cost
medium(Default)- $2.26
xhigh- $4.93
xhigh versus 72.7% at medium, with 2.6× the latency and 2.2× the estimated token cost. Latency and cost summarize each setting’s successful episodes.3.2. What does agentic scaffolding contribute?
For Agentic Policy, we give the model a persistent Codex session for observing the scene, proposing robot actions, and retaining episode context. At each decision, the harness has full access to conversation history and episode decisions throughout. Astra proposes one end-effector target; the host validates and executes that target, then returns a new observation.
Compared with directly querying an API for each action, this setup allows us to study more of the interaction structure in the harness and its effect on performance: how history persists, which tools are available, and how planning is encouraged. We examine optional tool access, written planning instructions, and explicit recovery interventions separately.
3.2.1. Does access to optional tools help?
Agentic Policy may write and execute arbitrary Python code within the Codex environment. To encourage tool-use, we provide Codex with a workspace containing direct paths to observation images, a scratch directory for scripts/crops/calculations, and persistent notes. We prompt it to utilize this workspace whenever additional visual inspection, geometric estimation, or numerical computation could inform its next action. When disabling optional tools, we disable shell/code execution and image inspection and remove the guidance encouraging their use.
When enabled, every episode executed tools; on average, 19.7 tool-calls per episode. Reviewed examples include stereo triangulation, gripper-pose calculations, computing inverse kinematics, and enlarged image crops. Despite substantial tool use, it is not clear that Astra benefitted from tool use. We observe that the model often comes to the same conclusion with as without tools.
Examples of Tool UseShow examplesHide examples
Inspect · enlarged observation
Original observation
Enlarged view shown to Astraim.crop((220,180,295,262)).resize((450,492)).save('scratch/mug_close.jpg')scratch/mug_close.jpg — created, then viewedTool details
Creates a close-up of the mug from the left shoulder image. The trace records a subsequent image-view call before decision 58.
Enlarged shoulder view shows the white end immediately above the mug opening; lower another 3 cm while keeping the grasp and upright orientation.
Calculate · local motion-to-image map
During a block-stacking episode, Astra tracked the blue block across 11 observations and matched its image locations to the gripper’s positions. It then ran Python to estimate how gripper movement shifts the block in the image, learning the relationship from its own recent movements.
Track the blue block in image space
Fit EEF displacement → pixel displacement
−397.45−233.14EEF Y−221.66+173.84EEF Z+56.30−303.99Calculation and source
coefficients = np.linalg.lstsq(
np.column_stack([positions, np.ones(len(positions))]),
pixels, rcond=None
)[0]3.2.2. Does explicit planning help Astra reason better?
In the Agentic Policy, we provide a planning mode that encourages the model to state how it intends to complete the task before acting, then track and revise its high-level plan as the episode progresses. We study the effect of this instruction with two additional variations:
- Planning Instruction Removed: Omit the request to decompose the task and track a plan, while retaining observations, feedback, and conversation history.
- Planning Discouraged: Additionally, instruct the model not to write multi-step plans, subgoals, current objectives, future-action sequences, or checklists; restrict notes to measurements and observed outcomes.
We find that the explicit reasoning trace reveals that removing the instruction nearly eliminates standalone planning commentary, but progress tracking and next-subgoal selection remain in action explanations. For example, without explicit planning instructions, the model organically outputs “One mug is confirmed on the shelf; withdraw upward before approaching the second mug.” These observations suggest that even without being explicitly instructed to generate a standalone written plan, the model still tracks progress and identifies what to do next.
3.2.3. Does providing motion error help?
We additionally provide motion-level feedback to the model to observe whether explicit error quantification helps. After each action segment, the harness reports position and orientation tracking errors 7, together with commanded and measured gripper state. Astra receives this feedback alongside fresh images and robot state before choosing its next target.
Not providing motion feedback produces little change in performance. With feedback provided, a text audit finds explicit tracking references in 51% of episodes, but only 1.14% of next-action explanations. Astra also identifies and responds to motion tracking errors without the explicit feedback, suggesting that retained action history and fresh robot-state observations may provide much of the same information.
3.3. When does Astra intervene?
With Orchestrated Policy, we examine rollout traces to identify when Astra allows Cosmos3-Nano-Policy to proceed with its execution, and when Astra wants to intervene. We group consecutive intervention decisions into blocks and use VLM-assisted classification to identify the primary issue in each block, including anticipated mistakes prevented before execution.
We observe that Astra intervenes to correct or prevent mistakes, accounting for 14.5% of all simulation steps. Among these intervention blocks, grasping problems and object-selection or planning mistakes account for 64.4% and 20.2%, respectively. The remaining interventions primarily address placement, manipulation, and transport failures.
3.4. Final configurations
Following the investigations, we establish the following configurations as default settings for the full RoboLab-120 evaluation.
Direct Policy
- Action generator
- Astra
- Actions
- Absolute end-effector poses
- Horizon
- Dynamic chunks of 3–15 actions on 90 tasks; historical 1–15 chunks on 30 tasks
- History
- Three observations at t−14, t−7, t; no persistent conversation
- Geometry
- Gripper and camera geometry supplied
- Execution
- Observation-to-action API calls
Agentic Policy
- Action generator
- Astra
- Actions
- Absolute end-effector poses
- Horizon
- One target held for 1–5 control steps
- History
- Full text and visual interaction history
- Geometry
- Gripper and camera geometry supplied
- Scaffolding
- Planning instruction and motion feedback retained; optional tools disabled
Orchestrated Policy
- Action generator
- Cosmos3-Nano-Policy / Astra
- Actions
- Policy proposals with Astra review and intervention
- Review horizon
- 15 control steps
- History
- Bounded memory across decisions
- Intervention
- Accept or edit proposals, or take direct end-effector control
- Scaffolding
- Optional native tools disabled; policy-control tools available
4. Runtime Performance
4.1. Timing
Astra is capable, but it takes its time. Across successful episodes on RoboLab-120, Direct Policy EEF inference averages 18.56 s/call. Overall, inference accounts for 97.0% of estimated total walltime, measured as task rollout time plus inference time. For Orchestrated Policy, Astra decisions average 11.50 s and Cosmos Action inference averages 2.74 s per request, together accounting for 94.9% of estimated episode duration. In practice, most of an episode is spent waiting for the next action. For comparison, Black Forest Labs reports median latencies of 46.96 ms for π0.5 and 713.76 ms for direct Cosmos 3 Nano on H200 in BF16 14.
We demonstrate what this would look like for a yogurt task in the video below. With inference time added, this task took an estimated 3:57 minutes to complete, with about 5.2s of action execution time. While general-purpose models have capability, there is a significant gap between capability and real-time control and deployment.
| Configuration | Latency / call | |
|---|---|---|
Direct Policy EEF | 18.56 s | |
Direct Policy jointpos | 20.19 s | |
Direct Policy bspline | 21.68 s | |
Agentic Policy | 13.37 s | |
Orchestrated Policy | 11.50 / 2.74 s | |
FLUX 3 Action (WAM) | 397.74 ms | |
Cosmos3-Nano-Policy (WAM) | 713.76 ms | |
π0.5 (VLA) | 46.96 ms |
TASK ROLLOUT INCLUDING INFERENCE TIME
4.2. Token Cost
We additionally provide token usage for using a general-purpose model as a native robot policy. Using token cost estimates at publishing time 15, successful episodes using absolute end-effector pose average 160k tokens ($1.96) for Direct Policy, 11.0M tokens ($17.72) for Orchestrated Policy and 30.9M tokens ($36.40) for Agentic Policy per episode. All costs presented in USD. Output tokens, including reported reasoning tokens, account for the largest share of each estimate.
| Model | Uncached input tokens (USD) | Cached input tokens (USD) | Output tokens (USD) | Total cost (USD) |
|---|---|---|---|---|
GPT-6 Astra | 81,426 ($0.81) | 75,635 ($0.08) | 27,359 ($1.37) | $2.26 |
GPT-6 Sol | 92,423 ($0.18) | 92,942 ($0.02) | 56,537 ($0.57) | $0.77 |
GPT-6 Luna | 38,873 ($0.00) | 39,200 ($0.00) | 43,380 ($0.02) | $0.03 |
5. What the investigations establish
This report shows that Astra can serve as a native robot policy for general-purpose pick-and-place, with its action interface and supporting context specified at inference time. On RoboLab-120, the highest performing policy, Agentic, reaches 74.8%. The results demonstrate strong effects of Astra being natively able to reason about action representation and task semantics. The ablations show that retaining interaction history significantly helps performance in the Agentic setting, without gaining additional benefits from tool-use. In the Orchestrated configuration, delegating action proposals to a WAM did not clearly outperform Astra’s native action generation. How best to combine Astra with specialized VLAs or WAMs remains an open question: which decisions to delegate, when Astra should review or intervene, and whether the resulting gains justify the added latency and cost. Together, these findings establish substantial manipulation capability while making the surrounding interface, context, and execution loop central to its performance.
6. How and when Astra fails
Our investigations demonstrate that while Astra is a capable robot policy for pick-and-place general-purpose manipulation, Astra remains sensitive to spatial alignment and prompt. In the geometry ablation, insufficient guidance can lead to repeated gripper adjustments instead of a successful approach and grasp. Even with geometry added to the prompt, we observe that a recurring limitation is converting attempted manipulation into a completed grasp, transfer, or placement across RoboLab-120. The failure rollouts illustrate how execution can stall: the gripper continues adjusting around an object without securing it, or objects are displaced without reaching their intended destination. Other failures involve incomplete placement or reorientation. These observations suggest that online motion correction does not necessarily produce effective physical progress for challenging grasp tasks.
Agentic assistance improves reliability overall, but several failure modes persist: incorrect target selection, unstable or obstructive grasps, incomplete manipulation sequences, and unsuccessful recovery. We observe that policies place objects in the wrong receptacle, grasp objects in ways that trap the gripper, lose objects during transport, or repeatedly reorient an object without completing its placement and release. Recovery attempts can also repeat ineffective motions or fail to retrieve objects that have fallen off the table. The benefits of assistance vary across tasks. These results suggest that persistent context and policy assistance address some execution challenges, but do not consistently resolve grasp error.
We additionally highlight challenges in deploying Astra as a real-time robot policy: Direct Policy averages 18.56 seconds per inference call, and Agentic Policy costs an estimated 30.9M tokens ($36.40) per successful episode. These timing results illustrate that reliable operation in a dynamic task environment could potentially prove challenging when the task requires real-time adaptation to fast changing scenes
ILLUSTRATIVE FAILURE CASES
7. Study Limitations
Statistical Significance. The findings in this report should be interpreted in light of the evaluation budget. Although we evaluate each task with 5 episodes, substantial inference cost and wall-clock time limit the number of repeated trials available to characterize run-to-run variability and variation across policies. We therefore have not established the statistical significance of the reported differences between configurations, particularly the smaller gaps in the ablations. The results provide evidence of capability and suggest which design choices warrant further investigation, but should not be treated as definitive evidence of a parameter's effect.
Task coverage. Our evaluation focuses on general-purpose pick-and-place manipulation in RoboLab-120. It does not cover contact-rich tasks such as insertion or assembly, extremely long-horizon mobile manipulation, or highly dynamic tasks requiring rapid reactions. Simulation is paused during inference, so the evaluation also does not test whether the policies can respond effectively while the environment continues to change. The reported results therefore establish capability within the evaluated task distribution; broader manipulation competence and real-time responsiveness remain open questions.
Model interface. We evaluate the models through their exposed interfaces, with limited visibility into how they are hosted or what internal tools and processing may be available on the policy server. Our analysis is therefore restricted to observable inputs, outputs, execution behavior, and parameters we can vary at inference time. These ablations characterize how the deployed system responds to changes in its interface and context, but cannot fully separate the underlying model’s capabilities from the effects of unobserved serving infrastructure or internal processing.
Acknowledgements
We thank Elie Aljalbout, Alperen Degirmenci, Hugo Hadfield, Jingyi Jin, Max Zhaoshuo Li, Yashraj Narang, Fabio Ramos, Moritz Reuss, Jonathan Tremblay, Zhuocheng Xu for helpful and productive discussions in constructing this blog post.
References
- Yang et al. RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies. RSS, 2026.
- Octo Model Team et al. Octo: An Open-Source Generalist Robot Policy. RSS, 2024.
- NVIDIA. Cosmos 3: Omnimodal World Models for Physical AI. 2026.
- Li et al. Causal World Modeling for Robot Control. RSS, 2026.
- Physical Intelligence. π0.5: a Vision-Language-Action Model with Open-World Generalization. CoRL, 2025.
- Zhang et al. An Unexpected Robot Policy: Early Evaluations of GPT-6 Astra on RoboDojo and Beyond. 2026.
- Su et al. GPT 6 Astra as an Embodied Policy. 2026. Technical report.
- Galbot Team. Systematically Exploring the Capabilities of GPT-6 Astra as Embodied Policies. 2026.
- DROID Dataset Team. DROID: A Large-Scale In-the-Wild Robot Manipulation Dataset. RSS, 2024. Accessed October 11, 2026.
- Zhou et al. BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning. NeurIPS, 2025.
- Gasparetto and Zanotto. A new method for smooth trajectory planning of robot manipulators. Mechanism and Machine Theory 2007, 42(4), 455–471.
- Han et al. B-spline Policy: Accelerating Manipulation Policies via B-spline Action Representations. 2026.
- Fu et al. ICRT: In-Context Imitation Learning via Next-Token Prediction. ICRA, 2025.
- Black Forest Labs. FLUX 3 Action. 2026. Model report.
- OpenAI. OpenAI API pricing. Accessed October 11, 2026.