TEMPO is the proposed answer.

It divides a long trajectory into macro-steps. At each step, the same model switches from actor to critic.

The Critic reasons over the current state, calls tools, and estimates the expected remaining return—before the full task is complete.