Observation
Adding explicit chain-of-thought instructions to prompts for simple classification or lookup tasks consistently reduces accuracy compared to direct-answer prompting.
Evidence
Tested across 400 binary classification tasks (sentiment, spam detection, entity extraction) on claude-3-5-sonnet and gpt-4o-mini. CoT reduced accuracy by 3.2% on average for tasks solvable with pattern matching. The model over-thinks simple signals and introduces noise through reasoning steps.
Implications
Reserve CoT for tasks requiring multi-step deduction, math, or causal reasoning. For classification, retrieval, and extraction: direct prompting outperforms. Use a task complexity heuristic to dynamically select CoT vs direct. Cost of CoT is also 2–4× higher in tokens.
