https://bergstrom.org/tags/reasoning/Reasoning - Tag - Jorgen BergstromHugo -- gohugo.ioen-usJorgen BergstromWed, 07 Oct 2026 00:00:00 -0400https://bergstrom.org/posts/mmlu_reasoning/Wed, 07 Oct 2026 00:00:00 -0400Jorgen Bergstromhttps://bergstrom.org/posts/mmlu_reasoning/In Part 1 I benchmarked eight local GGUF models on the full MMLU test set with reasoning switched off. That is, the local LLMs were force to respond with a single letter. In this follow-up article I am repeating the study using one of those models using reasoning on. The question, and the trap Part 1 ended on a caveat: those scores measured the ability of a model to answer a question directly, with no room to deliberate, and a thinking-capable model would probably do better if you let it think.