38.
Btw, I've been doing this with a vllm fork that adds a steering runtime that only shows about 2.7% throughput los…
Btw, I've been doing this with a vllm fork that adds a steering runtime that only shows about 2.7% throughput loss at 32 batch size on Gemma 3 27B. It also adds an activation capture consumer plugin system with the example being a file stor