Baloney Detection Kit by Carl Sagan

“Like all tools, the baloney detection kit can be misused, applied out of context, or even employed as a rote alternative to thinking. But applied judiciously, it can make all the difference in the world — not least in evaluating our own arguments before we present them to others.”
— Carl Sagan (The Demon-Haunted World, 1995)

Tools 🧰

For skeptical thinking

• Independent confirmation — facts need outside verification
• Substantive debate — invite all knowledgeable perspectives
• No argument from authority — experts err; there are no authorities, only experts
• Multiple hypotheses — generate all possible explanations, then test to eliminate
• No attachment — your hypothesis is a waystation, not a destination
• Quantify — vague + qualitative = open to anything; measure to discriminate
• Chain of argument — every link must hold, including the premise
• Occam’s Razor — when two hypotheses fit equally, choose the simpler
• Falsifiability — if it can’t be disproved in principle, it’s not worth much
• Controlled experiments — design out bias; run double-blind when needed
• Separate variables — one change at a time

Fallacies ⚠️

Which are common

• Ad hominem — attacking the person, not the argument
• Appeal to authority — unverifiable claims from the powerful
• Adverse consequences — “it must be true or else…”
• Appeal to ignorance — absence of proof is not proof of absence
• Special pleading — invented exceptions to rescue a failing argument
• Begging the question — assuming the answer in the premise
• Observational selection — counting the hits, forgetting the misses
• Small numbers — drawing big conclusions from tiny samples
• Statistics misread — e.g. confusing median with average
• Inconsistency — applying standards selectively
• Non sequitur — conclusion doesn’t follow from premise
• Post hoc — “it happened after, therefore caused by”
• Meaningless question — false paradox with no real referent
• False dichotomy — ignoring the vast middle ground
• Short-term vs. long-term — urgency used to dismiss larger stakes
• Slippery slope — extreme outcome presented as inevitable
• Correlation ≠ causation — co-occurrence is not mechanism
• Straw man — caricature a position to make it easier to attack
• Suppressed evidence — half-truths, selective disclosure
• Weasel words — euphemism as camouflage for meaning