Epistemological Peeves

Getting involved in online arguments is a great way to go nuts dealing with naive folk philosophy sayings. It’s bad when those sayings are used by people who you disagree with and worse when used by people you agree with. So here’s a rant about some of them. There must be many more that readers would like to add.

“Absence of evidence is not evidence of absence.”

Really? Is that why when you look both ways and don’t see any vehicles coming down the street you’d still better not cross? Because “Absence of evidence is not evidence of absence”? Of course, if you look at your feet and then straight up in the sky, maybe you shouldn’t risk it because absence of evidence in that case is not very good evidence of absence.

This whole dumb issue is trivial from a Bayesian viewpoint.
P(absence)/P(presence) =
(P₀(absence)/P₀(presence))*(P(no evidence|absence)/P(no evidence|presence)).

P₀(absence)/P₀(presence) is just the prior odds. Do cars often come down that street?
If you’ve looked where the cars often come from and see nothing,
P(no evidence|presence) is very small, so the odds P(absence)/P(presence) become very large.
If you’ve looked somewhere else,
P(no evidence|absence)/P(no evidence|presence)) =~1. so you haven’t changed the odds.
This is all so obvious that it’s embarrassing to write it down. But the dopey saying will still be used repeatedly by all sides.

“You cannot prove a negative”

Good point, and negatively phrased. Can you prove it?

In some sense nothing can be “proved”, but that’s not the sense meant here. People who say it are not worried about Gödel Incompleteness.

In fact, in the realm where “proof” comes closest to making sense, one proves “negatives”, i.e. nonexistence claims, all the time. I can prove the nonexistence of a largest prime.

I guess “You cannot prove a negative” is closest to being useful in rough areas without sharply defined rules. It’s hard to prove the non-existence of six-headed geese, because it’s hard to make an exhaustive list of all geese and it’s hard to make air-tight anatomical arguments. Still, using a general philosophical claim that’s obviously false in an argument is a good indicator that that someone is not thinking clearly

“Burden of proof”

People argue about which hypothesis should have “the burden of proof”. Should a new chlorinated hydrocarbon be presumed safe unless some high standard of evidence shows it isn’t? Should it be presumed unsafe until some high standard of evidence shows it’s safe? When there are big differences in costs and benefits between different types of errors then it makes sense to ask for stronger evidence for one hypothesis than for another, but in general the “burden of proof” can be verbally manipulated to favor any side of an argument. It gets in the way of sorting out evidence and evaluating costs and benefits in an even-handed fashion.

“Null hypothesis”

The flip side of the “burden of proof”, the “null hypothesis” is widely but mistakenly taken to be the cornerstone of statistics. Some claim gets to be called the null and is accepted unless some observation is extremely inconsistent with it. The problems with the null-hypothesis-significance-test approach are well-known among statisticians but seem unfamiliar to most MDs struggling through reading research papers and, I think, to most social scientists.

As with the burden of proof, verbal agility in getting an idea null-hypothesis status can replace actual evidence favoring a hypothesis. Is the null hypothesis that that chlorinated hydrocarbon is dangerous? Is it that it’s safe? What about heart attacks from a Cox-2 inhibitor? Life and death decisions should not depend on who manages to call dibs on null status.

Here’s a less fraught example.
Which of these should have been the null hypothesis in a high-energy particle physics experiment?
a) There’s no previously unobserved particle in the energy range of the experimental search.
b) The whole structure of particle physics needs to be turned upside-down.
Anyone trained in NHST will immediately recognize that (a) is a good null and (b) is the opposite of a good null.

In the search for the Higgs boson, however, (a) and (b) were the same hypothesis.

NHST was designed as a way to avoid being distracted by the forest of accidental evidence suggesting possible relations, in order to focus on more promising leads. It’s still useful for that but the costs of its misuse may exceed the benefits of its proper use.

Repeated use of priors

To paraphrase an argument used against a Bayesian analysis: “Each of your probabilities assumes that your hypothesis is true.” Each likelihood in a Bayesian analysis is a conditional probability. P(observation|hypothesis) by definition means “the probability that that observation would occur if the hypothesis were true." If the hypothesis itself seems improbable that has no effect on those conditional probabilities. The high or low starting probability of the hypothesis shows up in the prior probability, which gets used as a factor exactly once in calculating the posterior probability. Including the prior probability in each conditional probability leads to nonsense, e.g. posterior probabilities that change whenever even a completely non-informative observation is included.

“Correlation is not causation”

True, if taken to mean that a correlation between two variables does not tell you that either caused the other, much less which way any possible causation flowed. It doesn’t even imply that the causal effect has the same sign as the correlation. That does not mean, however, that one cannot extract estimates of causal effects from a set of correlations and some plausible assumptions about causal pathways. There are good books about this.

The outcome “is associated with” some suspected cause.

This is an important careful wording for observational studies. Often, however, it’s embedded in language that is meant to imply causation. The problem isn’t the suspicion of causation, it’s the fuzzy unclear language.
What’s worse, I’ve now seen many randomized controlled trials whose conclusions are couched in “is associated with” terms. WTF was the point of doing an RCT if you then have to pretend that you weren’t trying to assess causal effects of a treatment? Using “is associated with” in cases where what’s actually meant is “causes, within our confidence interval” not only dilutes the RCT message but by erasing the distinction with observational studies makes their results seem just as causal as RCTs.

More suggestions welcome