Does a knowledge base actually help a coding agent? An A/B experiment
The same task, with and without the knowledge base, eight runs per arm, judged blind. The assisted arm was always faster and cheaper — but one subtly flawed fact made it worse than no knowledge at all. Repairing that single fact flipped 30 of 32 blind comparisons and prevented a mistake models invent unprompted.