CoRL 2026 Workshop on Learning from Corrections and Interventions

What Can One Sentence Fix?

A Failure Taxonomy and a Minimal Language Correction for Frozen VLA Policies

Owen Kwon, Pablo Ortega-Kral, Arthur Bucker, Jean Oh

Robotics Institute, Carnegie Mellon University

Paper (Coming Soon) Code (Coming Soon)
Base policy✗ Failure
+ one sentence✓ Success

One sentence per task, written once and replayed on every rollout. The frozen policy does the rest.

Rollouts

Before and after one sentence.

Overview

One sentence can fix a miscalibrated magnitude.

Failure taxonomy & correctability

Four failure modes, and one a sentence can fix.

We labelled every failing LIBERO-90 instruction for OpenVLA-OFT and π0.5, 133 instruction–model pairs, with one of four failure modes.

Share of failing instructions by mode 133 instruction–model pairs, LIBERO-90
Execution misalignment: a sentence can reach it Needs a target or a plan instead
Table view
ModeOpenVLA-OFTπ0.5
Execution misalignment23%22%
Task misunderstanding47%50%
Perception failure25%22%
Multi-step planning5%7%

Execution misalignment

Right target, but how far or how hard is miscalibrated.

Ask a person for a magnitudeOne sentence reaches it

Task misunderstanding

“The middle bowl” is read in the image frame, not the table frame.

Ask a person for a target

Perception failure

A different object is picked: asked for the milk, it takes the can.

Ask a person for a target

Multi-step planning

The first sub-goal succeeds; the policy stops before the second.

Ask a person for a plan

Failure taxonomy. One rollout per mode and model; ✓ marks the correct answer, orange the correctable mode.

Human correction

One sentence becomes a short bias on a frozen policy.

A person watches one failed rollout and writes one sentence. It becomes a short bias on the frozen policy’s actions and is reused on every later rollout, with no retraining and no new data.

  1. 1

    Watch

    One failed rollout, one sentence (one or two clauses for most tasks).

  2. 2

    Ground

    GPT-5 mini turns each clause into a dimension, sign, magnitude, and time window.

  3. 3

    Shape

    The magnitude word sets a weight, ramped up and down within the window.

  4. 4

    Replay

    The bias is added to the frozen policy on every later rollout.

Move down more in 1 to 2 seconds (z, −1, more, [1.0, 2.0])

Where the magnitude ranges come from

Each word maps to a fixed fraction of the controller’s largest step, set empirically from a few simulation trials. The ranges depend on the robot and could be learned.

Where the correction sits in time Two π0.5 rollouts from the paper. Hover or drag across the timeline.

δd(t) = ∑k : dk = d wk φk(t) acorrt,d = at,d + δd(t)

Action-space adaptation

People speak in Cartesian terms (“move left”); on the xArm7, the Jacobian pseudoinverse maps each correction to joint velocities (Figure 1, panel 3).

What the signal assumes

  • A1 Target and plan are right; just the magnitude needs adjusting. Taxonomy
  • A2 A few biases turn the near-miss into a success. Results
  • A3 One viewing shows which dimension drifted, and when. Results
One correction, then the policy on its own. Failed above, corrected below; orange is the human’s window.

Does the signal suffice?

Where the sentence reaches, it usually suffices.

Simulation: LIBERO, π0.5

Corrections are written for execution-misalignment tasks and re-run on the base policy’s failed episodes. The VLM baseline (GPT-5 mini) gets the same failure video, with no person in the loop.

Task success Base π0.5 → with one sentence per task
Base π0.5 (VLM baseline adds 0) CorrectVLA
Failures recovered Out of distribution, 64 failures fall in a correctable task
Recovery is highest when a task fails the same way each time Failed episodes recovered, per out-of-distribution task

Corrections were written for 4 of the 13 correctable instructions, so the overall 10.1% is a lower bound.

Table 1: simulation
SettingMetricBase π0.5VLM baselineCorrectVLA
In-distributionSuccess1861/2000 (93.1%)–1926/2000 (96.3%)
Failures recovered0065/139 (46.8%)
Out-of-distributionSuccess122/450 (27.1%)–155/450 (34.4%)
Failures recovered0033/328 (10.1%)

Real robot: xArm7, π0.5-DROID

π0.5-DROID reached 95% on pick-and-place in its fine-tuning workspace. We then moved the robot base and ran 3 runs of 10 trials per condition, with one sentence per run, written in 40–80 s.

Fine-tuning workspace (a) and the shifted workspace (b).
Trials that succeeded after the base was moved 3 runs × 10 trials per condition; one sentence per run
Success, base policy Success, with one sentence Failure

† The object happened to fall inside the fine-tuning distribution.

Table 2: real robot
ConditionBase π0.5-DROID (runs 1–3)CorrectVLA (runs 1–3)
Environment shift only1/109/100/1010/1010/1010/10
Object moved0/1010/100/1010/1010/1010/10
Object and location changed2/100/100/108/109/107/10
Human engagement per run0 s0 s0 s40–80 s40–80 s40–80 s

Where the taxonomy says to stop. Folding a shirt and stacking boxes stay at 0/30 with or without correction: they need a plan rather than a magnitude.

Real robot. After the shift, π0.5 misses both grasps (red); one sentence restores success (green).

Discussion

Where this goes next.

Limitations

  • One annotator labelled the taxonomy.
  • One correction per task assumes one failure pattern and timing per task.
  • Corrections were translational, language-only, and written by the authors; a user study is future work.
  • Magnitude ranges were set empirically; learning them is future work.
  • In simulation, the bias was applied only to failed episodes.