Fly-by-Code Gives a Drone a Separate Diagnostic Job

A robot opens its gripper. The tool disappears from the camera image. Has the delivery succeeded? In one experiment described in Fly-by-Code, the agent accepted evidence like this as confirmation even though the hammer had not reached its basket. The software had recorded an approach and a release, not a completed delivery. That distinction gives the new aerial-manipulation paper its strongest argument: sometimes the missing capability is a deliberate trip to look at the result.

Fly-by-Code, an October 8 preprint by Jaewoo Lee and coauthors, gives a coding agent a separate diagnostic job after a simulated task attempt. The researchers report substantially better completion rates and fewer false declarations of success than their comparison systems. They also report higher estimated model spending per trial and a small, imperfect physical demonstration. The paper and the authors' project page are the sources for this account. No reputable independent validation was found in the reporting reviewed here, and the project page still describes the code as forthcoming. These are research-team results, not an independently reproduced product claim.

The agent first writes a task program using documented robot functions. It can acquire camera observations, locate objects, estimate geometry, move the manipulator and operate the gripper. After that program runs, the proposed Active Feedback stage can write another program with a different purpose: resolve what the execution evidence leaves uncertain. A new camera position can reveal the inside of a basket. A physical probe in simulation can test an unresolved hypothesis about an object. The program is gathering information for the next decision, rather than simply repeating the original movement.

That diagnostic freedom has an important boundary. It operates in simulation, starting from the state left by the task attempt. The task is scored before the diagnostic stage, so moving an object into place during investigation cannot earn completion credit. Before a revised task program executes, both the environment and its semantic map are reset. What carries forward is the information gathered, not a partially repaired scene. This makes the experiment a test of whether investigation improves the next program, rather than a way to hide extra task execution inside an evaluation step.

The agent receives selected images, associated notes and positions, measurements and execution output. It then either declares completion or supplies a revised task program. The evaluator's ground-truth state, scores and success labels are kept out of that feedback. This separation matters: an agent told by an external grader that it failed has a different problem from one that must decide whether its own observations justify confidence. Fly-by-Code asks the latter question. It does not assume that the last line printed by a task program is a trustworthy description of the world.

The main comparison covers six tasks in a laboratory digital twin and a warehouse simulation, with ten trials per task and feedback condition. Each trial permits an initial execution and up to ten revisions. In the authors' results, terminal-image-and-log feedback succeeds in 19 of 60 trials, or 31.7 percent. Adding execution traces raises that to 29 of 60, or 48.3 percent. Active Feedback reaches 45 of 60, or 75 percent. Those are outcomes on this task suite. They are not a forecast for a drone arriving at an unfamiliar customer site.

The completion judgments deserve their own accounting. The terminal-feedback condition ends with 25 false success declarations; the trace condition has 19; Active Feedback has five. Active Feedback also has ten trials that reach no final success declaration within the budget. Its reported 90 percent completion-judgment precision means 45 correct declarations among 50 declarations. It does not mean 90 percent of tasks succeeded. A team evaluating this approach should keep both denominators visible. Confidently reporting an unfinished job and refusing to claim completion are different operational outcomes, even though neither delivers the requested result.

The comparison is useful without being universal. The researchers adapt a CaP-X-based coding loop and add traces following ASPIRE's per-primitive recording design. They do not reproduce every feature of those original systems. The conditions share the robot interfaces and principal coding-agent configuration, but the diagnostic condition is allowed additional information-gathering work. The result therefore supports the value of this particular feedback arrangement, not a claim that active diagnosis beats every alternative at equal total compute. Performance also remains uneven: Active Feedback succeeds in four of ten cabinet trials and five of ten sorting trials.

The budget table makes the tradeoff concrete. Across all tasks, the estimated token-based model cost per trial is $9.60 for terminal feedback, $9.40 for traces and $17.10 for Active Feedback. Counting spending on failed trials as well, the corresponding estimated cost per success is $30.40, $19.50 and $22.80. Active Feedback achieves the most successes, but the trace condition has the lowest estimated model cost per success. These are token-cost estimates, not actual subscription charges or a full accounting of robot operation. They do not price hardware, flight time, maintenance or the consequences of an incorrect completion report.

For a builder, that is a more interesting result than an unconditional efficiency claim. The choice depends on which failure matters and what it costs. A workflow in which an unfinished placement can be cheaply retried may value a different tradeoff from one in which a false completion report sends the next process down the wrong path. That is an operating judgment, not something this paper measures. The research supplies counts and model-cost estimates that let the question be asked honestly. Fewer task-program executions do not, by themselves, establish less total work when a separate diagnostic stage has been added.

The system's spatial memory is another part of the result that should not disappear behind the language-model headline. A background semantic map accumulates object estimates from color-and-depth observations and camera poses. It can suggest where to inspect when the target is outside the current view. In these experiments, the detection vocabulary is fixed by the task configuration. The map's labels and positions remain fallible; image-only hypotheses are directional cues, not authoritative coordinates. A remembered basket is a reason to look somewhere, not proof that a tool is currently inside it.

When the researchers remove map support throughout the loop, two-object pick-and-place success drops from nine of ten trials to four, and sorting falls from five of ten to two. That experiment does not isolate the map's contribution to diagnosis, because it also removes help during task execution. It does show why the full system cannot be reduced to a clever verification prompt. Finding a useful viewpoint depends on accumulated spatial information as well as the ability to ask a good question. The paper also records cases in which incorrect labels and stale locations complicate the assessment.

A detailed placement case illustrates what the extra observations actually establish. The task program reports that both tools have been placed successfully. The diagnostic program examines the destinations and nearby surfaces, and its images and depth-based estimates put the tools near floor level, outside the required destinations. This is evidence against completion. It does not establish exactly when the objects fell or why. That restraint is valuable: a useful diagnostic system can reject an unsupported success claim before it has a complete causal explanation. Discovering the miss and producing a reliable correction remain separate achievements.

Generated Python is also not a direct substitute for flight control. The motion interface sits above a backend that searches for and verifies whole-body trajectories against collision geometry, respects movement limits and monitors tracking. It can refuse a requested motion or report a timeout. Task code chooses goals and composes available operations; the lower-level system decides whether and how a movement can be executed. This division places a large amount of engineering beneath the apparent simplicity of an instruction such as moving to a better viewpoint. The paper does not establish safety for unrestricted real-world probing.

The physical demonstration uses a custom aerial platform with eight rotors, a four-degree-of-freedom arm, a parallel gripper and a wrist-mounted color-and-depth camera. It also uses external motion capture for localization and an offboard perception workstation communicating over Wi-Fi. The claim is consequently not self-contained onboard intelligence operating without laboratory infrastructure. External cameras used to record demonstration footage are not supplied as observations to the task program, but that is distinct from the external localization system the robot does use. Those two facts should not be collapsed into a claim that all sensing is onboard.

The researchers transfer simulation-synthesized task programs to the physical platform without changing the task code. The shared interfaces connect them to hardware-specific sensing, calibration and control, and the programs use live observations rather than replaying a fixed trajectory. The reported results are four successful two-object trials out of five and three successful cabinet trials out of five. Changed arrangements and starting positions make this more informative than a single successful recording. Ten trials across two known laboratory tasks still cannot establish broad deployment reliability, and transferring task code is not the same as transferring unrestricted diagnostic behavior into the physical world.

The failed physical trials identify concrete integration problems. In one, an erroneous basket position causes the hammer to strike the rim and fall outside. In another, the gripper remains caught on a cabinet handle and subsequent movements are refused. A third involves a bottle grasped too low, blocking the wrist camera and preventing a fresh measurement of the destination basket. These are not interchangeable reasoning errors. They involve perception, contact and control interacting with what the agent can observe next. More elaborate written reasoning cannot restore a view that the current grasp physically obstructs.

Fly-by-Code's useful contribution is the explicit allocation of action to obtaining evidence, with task credit separated from the investigation. Its own tables also show the price and the remaining failures. The next convincing demonstration would preserve that accounting on a broader set of physical tasks, disclose the full resource budget and let other teams reproduce the comparison. For now, the basket is the right place to keep attention: not whether the robot executed a release command, but whether it gathered enough evidence to know where the tool ended up.

LaunchPad positionSeparate evidence-gathering from task execution, and account for its cost before calling the system more efficient.