We find that plasticity loss happens both in nonstationary continual-learning environments but also in stationary…
We find that plasticity loss happens both in nonstationary continual-learning environments but also in stationary pretraining-like setups. This means that if you pretrain long enough your LLM will eventually lose the ability to adapt to new
How it ranks: Backlist reads my Twitter/X timeline, scores every tweet for substance with an LLM rubric (not engagement), and publishes the daily top picks with a one-line takeaway. Curated by Surya Dantuluri.