Statistics is good, Statistics is bad, the way you do it matters.

Some blogs have been discussing a recent article claiming that statistics is a bad way to do science. See here and here. I have to say that I completely agree with Lubos about the importance of statistical methods. I think I will elaborate on why I think statistics is important in doing science.

Suppose we have two hypotheses, only one of which can be correct. Suppose that these hypotheses both allow a certain observed phenomena to occur, but one suggests that such a phenomena should be exceedingly rare while the other suggest it should be fairly common. Obviously we can’t say that the latter hypothesis is proven because the former hypothesis is less likely to reproduce the observed results. But I think any rational person would say that the latter hypothesis is more likely to be correct. Statistics is importance because it allows us to quantify the probabilities involved so that we can make a judgment about such issues.

However, statistics can be misused. As Lubos says it is clear that sometimes people don’t use the right null hypotheses, they don’t use realistic assumptions about the probability distribution, etc. The reason this blog exists is because in my mind climate science is full of misapplication of statistics of precisely that nature, in particular the failure to do tests on realistic null hypotheses. But statistics is not at fault here. When something goes wrong with statistics, it is because the user did something wrong, not because the system itself is fundamentally flawed. Indeed, since it is basically math, it cannot, itself, be wrong. But as anyone who has taken a math course knows, that mathematics is more infallible than the Pope* doesn’t prevent you from getting less than a perfect grade.

*I understand that not everyone is likely to appreciate this joke, but I like to use hyperbolic examples and this was the best example I could think of.