81.
Building on (x.com)
Building on @nilinabra 's Soft Muon idea, I found a set of polynomials you can use to compute UΣᵖVᵀ accurately for |p| < 0.9 as efficiently as Newton-Schulz/Polar Express. check it out!
@varunneal on Backlist
2 appearances on the backlist front page in the last 30 days.
63.
blown away by how LR insensitive PSGD is
blown away by how LR insensitive PSGD is