New paper on activation mixing. The authors evaluate several mixing strategies across both classical FFNs and Swi…
New paper on activation mixing. The authors evaluate several mixing strategies across both classical FFNs and SwiGLU FFNs, with ablations on dense and MoE models. One interesting result: the most expressive mixing strategy isn’t the best ch
How it ranks: Backlist reads my Twitter/X timeline, scores every tweet for substance with an LLM rubric (not engagement), and publishes the daily top picks with a one-line takeaway. Curated by Surya Dantuluri.