// FDE-Gym: Agents as Forward-Deployed Engineers //

They found that the best agent configuration fully resolved 14 of 95 real customer-delivery tasks.

FDE-Gym tests agents as forward-deployed engineers. Each task comes from a real delivery project. Agents must question scripted stakeholders, work in production-like systems, and find constraints the customer never stated.

A delivery is scored by how much it improves the customer's own success metric, with penalties for extra messages, trial runs, and waiting time.

Across 13 configurations from seven providers, the top mean score is 0.56. The main failure is missing unstated constraints. Even on 23 tasks where the best configuration collected every key fact, it reached 0.8 or higher on only 10.

Highly recommended benchmark to look at.

https://arxiv.org/abs/2610.22350

Chat with Paper: https://academy.dair.ai/papers/fde-gym-the-last-mile-of-ai-delivery-2610.22350