26.
Topology-aware operators for physical ML
Encoding geometry into the operator itself can make physical ML both faster and more accurate
@che_shr_cat on Backlist
5 appearances on the backlist front page in the last 30 days.
8.
Training nonlinear RNNs with flat, parallelized O(1) gradients
The proposed method attacks BPTT’s sequential, unstable O(T) gradient path and reframes how expressive RNNs can be trained
1.
Programming neural network weights with natural language
Benign training text can now steer a model’s internal weights to carry a functional hidden artifact, blurring data curation and model supply-chain security
11.
Do transformers need separate Q, K and V projections?
Merging Q, K, and V projections challenges a core transformer assumption and could reduce memory pressure in long-context models
15.
1/
1/ We have spent years optimizing KV cache via head-sharing (GQA/MQA), but we ignored a fundamental assumption: why do Transformers need three separate Q, K, and V projections in the first place? Turns out, they don't. Merging them unlocks