15.
Why PPO beats GRPO on long-horizon tasks
Group-level synchronization makes long-horizon training harder, which changes the variance and staleness profile of the method
@QPHutu on Backlist
2 appearances on the backlist front page in the last 30 days.
72.
PPO is always my favorite RL algorithm, from game to LLM era (t.co)
PPO is always my favorite RL algorithm, from game to LLM era DAPO identified a critical issue with PPO’s ratio clipping. However, I don’t think the clip_higher solution addresses the root cause. Our DPPO work ( http:// arxiv.org/pdf/2602.0