In a recent post, Larry Wassermann asks whether Bayesian inference is a religion. He goes on suggesting that, in itself, Bayesian inference is not a religion. However, he says, there is a minority of Bayesian people who tend to behave as if they were belonging to a sect. Apart from the fact that they tend to be very cliquish and aggressive, they obviously consider their statistical paradigm as absolute truth and are unwilling to entertain the idea that Bayesian inference might have flaws.
I have good reasons to agree with Larry Wassermann on what he says: simply because I used to be one of those "thin-skinned, die-hard" Bayesians not so long ago. Since then, I have lost my faith, in part because of all the dirty things I have seen (and done!) over several years of applied Bayesian data analysis, but also because, on a more philosophical front, I have become more pragmatic and more relativistic, even for logical matters. To be clear: I still find Bayesian data analysis useful in practice, and I will not stop using it before long. But it's just that I do not anymore believe in Bayesian inference as a system.
In any case, because I used to be one of those believers in the Bayesian Truth, I feel entitled to add a few words here. Unlike Larry Wassermann who seems to believe that the partisan attiude of a minority of Bayesians has nothing to do with the content of the theory, I personally think that there is no smoke without fire. If some Bayesians tend to behave as if Bayesian inference were a religion (whereas non-Bayesian statisticians rarely do that with their own paradigm), perhaps this is because the philosophical underpinnings of Bayesian inference somehow gives them a predisposition to behave that way.
And indeed, Bayesian inference often claims to be a coherent and complete theory of plausible reasoning. As such, if taken literally, it works like a closed system of thought.
Just look at how it is supposed to work. First, you have a prior, which you do not freely invent, but which you find out by introspection. Second, your mind, inasmuch as it is rational, is compelled to follow the laws of probability as its only guide for rational thinking in the presence of uncertainty. Any new empirical observation automatically triggers an update of your probabilities according to Bayes' rule. Third, given your posterior probabilities and your utilities (which you also found out by introspection), you choose the option that will maximize your posterior expected utility.
Altogether, a Bayesian does not seem to have much opportunity to think outside of the Bayesian box. Instead, at all steps, he is supposed to stay within the logical boundaries defined by his paradigm. As van Fraasen (1989) puts it, by doing so, he will "live a happy and useful life by conscientiously updating the opinions gained at his mother's knees, in response to his own experience thereafter" .
In this sense, Bayesian inference is, perhaps not exactly a religion, but at least something like an ideology: something that is meant to take control of your rational mind, without leaving any room for you to think outside of the paradigm.
The frequentist school has a very different, and much less invasive, philosophical perspective on the question of the relation between the formalism and the state of mind of the statistician. This is very clear, for instance, in Neyman's writings, when he develops his idea of behavioral induction, as opposed to inductive reasoning. In his own words, "to accept a hypothesis H means only to decide to take action A rather than action B. This does not mean that we necessarily believe that the hypothesis H is true". In other words, the aim is to develop statistical decision procedures that have well-defined operational characteristics, not to tell you what you are supposed to believe.
I think I prefer this more agnostic philosophical stance. We need some space between our logical formalisms and our personal thoughts, some space for us to breathe. The current orthodox interpretation of Bayesian inference does not really allow for that.
Of course, all this is true only if you take the standard view literally. In practice, there are much more open and more pragmatic stances with respect to the Bayesian statistical formalism (see in particular the view often expressed by Andrew Gelman, e.g. Gelman and Shalizi, 2012). In real life, most applied Bayesian statisticians do take some freedom and regularly allow themselves for some fresh air and some free-thinking outside of the system. That's also what I have done over the years, and it is probably the only way for us to arrive at sensible results anyway. But then, our philosophical stance should reflect the possibility of doing so. Otherwise, we maintain ourselves in an uncomfortable state of cognitive dissonance, between what we do in practice and what we say we do.
More fundamentally, by maintaining some distance between the formalism and our thoughts and, more generally, by questioning the commonly accepted view(s) of Bayesian inference, I am sure that we will gain some more interesting insights about its practical meaning.
--
Gelman and Shalizi, 2012, Philosophy and the practice of Bayesian statistics, British Journal of Mathematical and Statistical Psychology, 66:8-38.
Neyman J., 1950. First course in Probability and Statistics, New York, Holt.
van Fraasen B., 1989, Laws and Symmetry, p. 178.
I have good reasons to agree with Larry Wassermann on what he says: simply because I used to be one of those "thin-skinned, die-hard" Bayesians not so long ago. Since then, I have lost my faith, in part because of all the dirty things I have seen (and done!) over several years of applied Bayesian data analysis, but also because, on a more philosophical front, I have become more pragmatic and more relativistic, even for logical matters. To be clear: I still find Bayesian data analysis useful in practice, and I will not stop using it before long. But it's just that I do not anymore believe in Bayesian inference as a system.
In any case, because I used to be one of those believers in the Bayesian Truth, I feel entitled to add a few words here. Unlike Larry Wassermann who seems to believe that the partisan attiude of a minority of Bayesians has nothing to do with the content of the theory, I personally think that there is no smoke without fire. If some Bayesians tend to behave as if Bayesian inference were a religion (whereas non-Bayesian statisticians rarely do that with their own paradigm), perhaps this is because the philosophical underpinnings of Bayesian inference somehow gives them a predisposition to behave that way.
And indeed, Bayesian inference often claims to be a coherent and complete theory of plausible reasoning. As such, if taken literally, it works like a closed system of thought.
Just look at how it is supposed to work. First, you have a prior, which you do not freely invent, but which you find out by introspection. Second, your mind, inasmuch as it is rational, is compelled to follow the laws of probability as its only guide for rational thinking in the presence of uncertainty. Any new empirical observation automatically triggers an update of your probabilities according to Bayes' rule. Third, given your posterior probabilities and your utilities (which you also found out by introspection), you choose the option that will maximize your posterior expected utility.
Altogether, a Bayesian does not seem to have much opportunity to think outside of the Bayesian box. Instead, at all steps, he is supposed to stay within the logical boundaries defined by his paradigm. As van Fraasen (1989) puts it, by doing so, he will "live a happy and useful life by conscientiously updating the opinions gained at his mother's knees, in response to his own experience thereafter" .
In this sense, Bayesian inference is, perhaps not exactly a religion, but at least something like an ideology: something that is meant to take control of your rational mind, without leaving any room for you to think outside of the paradigm.
The frequentist school has a very different, and much less invasive, philosophical perspective on the question of the relation between the formalism and the state of mind of the statistician. This is very clear, for instance, in Neyman's writings, when he develops his idea of behavioral induction, as opposed to inductive reasoning. In his own words, "to accept a hypothesis H means only to decide to take action A rather than action B. This does not mean that we necessarily believe that the hypothesis H is true". In other words, the aim is to develop statistical decision procedures that have well-defined operational characteristics, not to tell you what you are supposed to believe.
I think I prefer this more agnostic philosophical stance. We need some space between our logical formalisms and our personal thoughts, some space for us to breathe. The current orthodox interpretation of Bayesian inference does not really allow for that.
Of course, all this is true only if you take the standard view literally. In practice, there are much more open and more pragmatic stances with respect to the Bayesian statistical formalism (see in particular the view often expressed by Andrew Gelman, e.g. Gelman and Shalizi, 2012). In real life, most applied Bayesian statisticians do take some freedom and regularly allow themselves for some fresh air and some free-thinking outside of the system. That's also what I have done over the years, and it is probably the only way for us to arrive at sensible results anyway. But then, our philosophical stance should reflect the possibility of doing so. Otherwise, we maintain ourselves in an uncomfortable state of cognitive dissonance, between what we do in practice and what we say we do.
More fundamentally, by maintaining some distance between the formalism and our thoughts and, more generally, by questioning the commonly accepted view(s) of Bayesian inference, I am sure that we will gain some more interesting insights about its practical meaning.
--
Gelman and Shalizi, 2012, Philosophy and the practice of Bayesian statistics, British Journal of Mathematical and Statistical Psychology, 66:8-38.
Neyman J., 1950. First course in Probability and Statistics, New York, Holt.
van Fraasen B., 1989, Laws and Symmetry, p. 178.
No comments:
Post a Comment