Marin preregistered the loss for a 1e23 FLOPs MoE run

The Marin team preregistered a large MoE training loss before launch and beat it, making scaling-law forecasting auditable instead of retrospective

Not only do we want to train a good model, we want to know it'll be good before we even start training. About a month ago, the Marin team launched a 129B (16B active) 1e23 FLOPs MoE run and preregistered a loss of 2.252. The run finished t