in theory it can be super fast thanks to speculative decoding, but batching with other normal requests probably s…
in theory it can be super fast thanks to speculative decoding, but batching with other normal requests probably slows it down
How it ranks: Backlist reads my Twitter/X timeline, scores every tweet for substance with an LLM rubric (not engagement), and publishes the daily top picks with a one-line takeaway. Curated by Surya Dantuluri.