Why larger models may learn better: less neuron competition

The paper argues that larger models reduce interference between frequent and rare examples by giving tasks more representational capacity to coexist

We take for granted that larger models are better than smaller ones, but why is this so? Our new paper, led by Jing Huang and @EkdeepL , traces this to a data-induced competition for resources (neurons), using formal analysis, idealized ta