I don't know much about statistical uses of Bayesianism but can say something opinionated about the underlying philosophy.
From a philosophical point of view, Bayesianism is fairly weak and lacks argumentative support. The underlying idea of probabilism - that degrees of belief have to be represented by probability measures - is in my opinion wrong for many reasons. Basically the only well-developed arguments for this view are Dutch book arguments, which make a number of questionable assumptions. Besides, priors are also often not known. As far as I can see, subjective utilities can only be considered rational as long as they match objective probabilities, i.e., if the agent responds in epistemically truth-conducive ways (using successful learning methods) to evidence and does not have strongly misleading and skewed priors.
I also reject the use of simple probability representations in decision theory, first because they do not adequately represent uncertainty, second because they make too strong rationality assumptions in the multiattribute case, and third because there are good reasons why evaluations of outcomes and states of affairs ought to be based on lexicographic value comparisons, not just on a simple expected utility principle. Generally speaking, Bayesians in this area tend to choose too simple epistemic representations and too simple value representations. The worst kind of Bayesians in philosophy are those who present Bayesian updating as if it was the only right way to respond to evidence. This is wrong on many levels, most notably by misunderstanding how theory discovery can and should work.
In contrast, frequentism is way more cautious and does not make weird normative-psychological claims about how our beliefs ought to be structured. It represents an overall more skeptical approach, especially when hypothesis testing is combined with causal models. A propensity analysis of probability may also sometimes make sense, but this depends on analytical models and these are not always available.
There are good uses of Bayesian statistics that do not hinge on subjective probabilities and any of the above philosophical views about them, and for which the priors are well motivated. But the philosophical underpinnings are weak, and whenever I read an application of Bayesian statistics I first wonder whether the authors haven't just used this method to do some trickery that might be problematic at a closer look.
I'd be happy if everyone would just use classical hypothesis testing in a pre-registered study with a p value below 1%.
> The underlying idea of probabilism - that degrees of belief have to be represented by probability measures - is in my opinion wrong for many reasons. Basically the only well-developed arguments for this view are Dutch book arguments, which make a number of questionable assumptions.
Why don't you consider Cox's theorem - and related arguments - well-developed?
That's an excellent question. The answer is that I don't really count such kind of theorems as positive arguments. They are more like indicators that carve out the space of possible representations of rational belief and basically amount to reverse-engineering when they are used as justifications. Savage does something similar in his seminal book, he stipulates some postulates for subjective plausibility that happen to amount to full probability (in a multicriteria decision-making setting). He motivates these postulates, including fairly technical ones, by finding intuitively compelling examples. But you can also find intuitively compelling counter-examples.
To mention some alternative epistemic representations that could or have also been axiomatized: Dempster-Shafer theory, possibility theory by Dubois/Prade, Halpern's generalizations (plausibility theory), Haas-Spohn ranking theory, qualitative representations by authors like Bouyssou, Pirlot, Vincke, convex sets of probability measures, Jøsang's "subjective logic", etc. Some of them are based on probability measures, others are not. (You can find various formal connections between them, of course.)
The problem is that presenting a set of axioms/postulates and claiming they are "rational" and others aren't is really just a stipulation. Moreover, in my opinion a good representation of epistemic states should at least account for uncertainty (as opposed to risk), because uncertainty is omnipresent. That can be done with probability measures, too, of course, but then the representation becomes more complicated. There is plenty of leeway for alternative accounts and a more nuanced discussion.
Thanks. I found interesting that you like the Dutch book arguments more than the axiomatic ones.
> Moreover, in my opinion a good representation of epistemic states should at least account for uncertainty (as opposed to risk), because uncertainty is omnipresent.
Maybe I'm misunderstading that remark because the whole point of Bayesian epistemology is to address uncertainty - including (but definitely not limited to) risk. See for example Lindley's book: Understanding Uncertainty.
Now, we could argue that this theory doesn't help when the uncertainty is so deep that it cannot be modelled or measured in any meaningful way.
But it's useful in many settings which are not about risk. One couple of examples from the first chapter of the aforementioned book: "the defendant is guilty", "the proportion of HIV [or Covid!] cases in the population currently exceeds 10%".
Dutch book arguments are at least intended to provide a sufficient condition and are tied to interpretations of ideal human behavior, although they also make fairly strong assumptions about human rationality. The axiomatizations do not have accompanying uniqueness theorems. The situation is parallel in logic. Every good logic is axiomatized and has a proof theory, thus you cannot take the existence of a consistent axiom system as an argument for the claim that this is the one and only right logic (e.g. to resolve a dispute between an intuitionist and a classical logician).
The point about uncertainty was really just concerning the philosophical thesis that graded rational belief is based on a probability measure. A simple probability measure is not good enough as a general epistemic representation because it cannot represent lack of belief - you always have P(-A)=1-P(A). But of course there are many ways of using probabilities to represent lack of knowledge, plausibility theory and Dempster-Shafer theory are both based on that, and so are interval representations or Josang's account.
I'll check out Lindley's book, it sounds interesting.
> "degrees of belief have to be represented by probability measures", "the philosophical thesis that graded rational belief is based on a probability measure"
Of course it all depends on how we want to define things, we agree on that. There is some "justification" for Bayesian inference if we accept some constraints. And even if there are alternatives - or extensions - to Bayesian epistemology I don't think they have produced a better inference method (or any, really). [I know your comment was about the philosophical foundations, not about the statistical methods. But the alternative statistical methods do not have better philosophical foundations.]
Sorry, I can't agree with you on that one at all. It doesn't "...all depend on how we want to define things." Whether the representation of an epistemic state -- any state, really -- is suitable and adequate for a task is not just a matter of definition, it depends on the reality of what you want to describe. You cannot represent the throw of a six-sided die with a set {1, 2, 3, 4, 5}, for example. If you model in ideal rational agent's belief with a probability measure, then you cannot adequately represent lack of belief. Whether that's okay or not depends on the task.
> I know your comment was about the philosophical foundations, not about the statistical methods.
Absolutely, at the risk of sounding picky I have to say that you've answered to a comment I've never made.
> But the alternative statistical methods do not have better philosophical foundations.
Frequentism and the propensity view have better philosophical justifications, though. You may disagree, but that was the whole point of my first comment. We know that there are genuine stochastic processes with corresponding objective probabilities, for example. Frequentism also prevents incorrect applications of probability such as using statistics to predict the outcome of singular events based on mere conjecture about the priors. You can only do that with an analytic model.
> Frequentism and the propensity view have better philosophical justifications, though.
Not really if the knowledge we care about is related to a concrete situation (rather than the frequency of something under hypothethical replications defined in some ad-hoc way). As you said, whether that's okay or not depends on the task.
If we care about whether there was life on Mars or whether Aduhlem is an effective treatment for Alzheimer's I don't think that frequentist inference has good philosophical support. Frequentist epistemology is not directly applicable.
Of course if you consider the frequentist methods themselves as genuine stochastic processes with corresponding objective probabilities (which also requires a valid model, by the way) you have good philosophical support to say things about those methods and their long-term frequency properties.
But this knowledge about the statistical methods used doesn't translate into knowledge about the existence of life on Mars or the efficacy of Aduhlem unless you are ready to make additional assumptions - 'philosophically unjustified' as they may be.
You're involuntarily confirming my negative criticism of Bayesianism by suggesting Bayesian methods could tell us whether there is life on Mars. Sometimes you really need to gather more information and/or develop an analytic model. It seems that a lot of Bayesianism consists of wishful thinking and trying to take shortcuts (e.g. trying to avoid randomized controlled trials for new drugs).
> suggesting Bayesian methods could tell us whether there is life on Mars.
What I suggest is that Bayesian methods provide a framework to reason about the plausability of some statement about the world in a systematic way (unlike Frequentist methods, whatever the limitations in Bayesian methods).
> Sometimes you really need to gather more information and/or develop an analytic model.
Bayesian methods are definitely not a way to escape the need for an analytic model (including all the prior knowledge) and data gathering. What they provide is a mechanism to integrate the data using the model and calculate the impact of incremental information on our knowledge / uncertainty.
I’m not saying that it’s easy to have a good model and useful data for complex questions. But with Frequentist methods in addition to the model and the data you’d be missing the mechanism to use them in a meaningful way.
I wonder why do you say that Bayesians try to avoid randomized controlled trials for new drugs, by the way. Bayesian methods are increasingly used in randomized clinical trials.
You asked, "Why don't you consider Cox's theorem - and related arguments - well-developed?" I consider Cox's argument not well-developed because D–S theory shows the postulates miss useful and important alternatives. So it fails as an argument for a particular interpretation of probability.
I quoted 13415 saying that the only well-developed arguments were […] and asked him why didn’t he consider […] well-developed - compared to the former. I apologize if the scope of the question was not clear.
You can get to a frequentist technique from a given instance of a bayesian 'trajectory', so I don't really understand what leg frequentistism has left to stand on? How is frequentistism more 'cautious'?
The anti-bayesian frequentist argument, especially re. priors has always reminded me of that story about Minksy and his student 'randomly wiring' his machine. http://magic-cookie.co.uk/jargon/mit_jargon.htm
" In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-Tac-Toe" Sussman replied.
"Why is the net wired randomly?", asked Minsky.
"I do not want it to have any preconceptions of how to play", Sussman said.
Minsky then shut his eyes.
"Why do you close your eyes?", Sussman asked his teacher.
My impression is that philosophers and statisticians are often working with different focal examples. I think that in many fields important scientific knowledge essentially takes the form of a point estimate (e.g. the R0 of Covid is XXXX). It is also easy to come up with useful priors (e.g. the R0 is likely below 20) that arise more from characteristics of the model rather than theory.
Note that it is possible to reformulate the Covid example into a Null hypothesis test at the cost of being less informative (e.g. Is the R0 significantly above 1?) but then the knowledge becomes less useful for making certain important decisions.
Anyways, my general impression is that Bayesian statistics are probably more useful for making good decisions that require precise numerical knowledge of certain types of information but maybe less useful for many of the sorts of conceptual issues philosophers are often interested in.
Regarding "and third because there are good reasons why evaluations of outcomes and states of affairs ought to be based on lexicographic value comparisons, not just on a simple expected utility principle": do you have any suggested references that describe this in more detail?
Same question for "This is wrong on many levels, most notably by misunderstanding how theory discovery can and should work."
Also, do you have any suggestions for statistics books that you do like? Especially those with an applied bent (i.e. actually working with data, not philosophical discussions).
From a philosophical point of view, Bayesianism is fairly weak and lacks argumentative support. The underlying idea of probabilism - that degrees of belief have to be represented by probability measures - is in my opinion wrong for many reasons. Basically the only well-developed arguments for this view are Dutch book arguments, which make a number of questionable assumptions. Besides, priors are also often not known. As far as I can see, subjective utilities can only be considered rational as long as they match objective probabilities, i.e., if the agent responds in epistemically truth-conducive ways (using successful learning methods) to evidence and does not have strongly misleading and skewed priors.
I also reject the use of simple probability representations in decision theory, first because they do not adequately represent uncertainty, second because they make too strong rationality assumptions in the multiattribute case, and third because there are good reasons why evaluations of outcomes and states of affairs ought to be based on lexicographic value comparisons, not just on a simple expected utility principle. Generally speaking, Bayesians in this area tend to choose too simple epistemic representations and too simple value representations. The worst kind of Bayesians in philosophy are those who present Bayesian updating as if it was the only right way to respond to evidence. This is wrong on many levels, most notably by misunderstanding how theory discovery can and should work.
In contrast, frequentism is way more cautious and does not make weird normative-psychological claims about how our beliefs ought to be structured. It represents an overall more skeptical approach, especially when hypothesis testing is combined with causal models. A propensity analysis of probability may also sometimes make sense, but this depends on analytical models and these are not always available.
There are good uses of Bayesian statistics that do not hinge on subjective probabilities and any of the above philosophical views about them, and for which the priors are well motivated. But the philosophical underpinnings are weak, and whenever I read an application of Bayesian statistics I first wonder whether the authors haven't just used this method to do some trickery that might be problematic at a closer look.
I'd be happy if everyone would just use classical hypothesis testing in a pre-registered study with a p value below 1%.