14.
When SGLang OOMs, what exactly runs out of memory?
SGLang OOMs are not one failure mode; serving memory can disappear into weights, KV cache, activations, fragmentation, queues, or scheduler choices
@alecttrona on Backlist
1 appearance on the backlist front page in the last 30 days.