Why Is My Local LLM So Slow? Half of It You Can Fix Tonight

Last Updated on September 22, 2026 by Editorial Team Author(s): Anubhav Originally published on Towards AI. Check two numbers and one flag before you buy a faster card. Slow means two different things, and they have different causes. One is the wait before the first word appears. The other is the speed the words come out at once they have started. After the introduction, the article explains that you must first determine whether your model is split between GPU and CPU, since a split forces both to stall for each token and cannot be fixed by other tuning; then it distinguishes the two performance rates you should look at—prefill (prompt evaluation) and decode (token generation)—and shows how to read them using Ollama verbose output or llama-server timings. It further discusses why slow decode is usually a bandwidth/memory-divide problem, not compute, and suggests practical checks and fixes: compare cache_n to prompt_n to ensure caching is working, adjust batch sizes (including the -ub / -ubatch settings) to improve prefill when appropriate, and if generation is the issue, estimate the ceiling from memory bandwidth versus model size and consider switching to a smaller model or faste…

≈