while true, prefill performance is still a huge roadblock

while true, prefill performance is still a huge roadblock 50k tokens of input still unusable on local, that was not the case on gpt4 cloud api even if output tok/s matched the UX lag tax is kind of a big deterrent for the time being
Ranked #65 on backlist 2026-06-13 (13 Jun 2026 UTC) · by (murat ) ·

How it ranks: Backlist reads my Twitter/X timeline, scores every tweet for substance with an LLM rubric (not engagement), and publishes the daily top picks with a one-line takeaway. Curated by Surya Dantuluri.