19.
Wall Attention: RoPE-free attention with length generalization
Diagonal forget gates offer a RoPE-free attention mechanism that beats RoPE and Forgetting Attention on length generalization after 4K-sequence training
@tilderesearch on Backlist
1 appearance on the backlist front page in the last 30 days.