The Meaning of Probability

The formalization of the concept of probability has a long history. Probability Theory is now a well-founded and very mature part of Mathematics, mainly due to its axiomatization by Kolmogorov who grounded the concept of probability in measure theory.
It may thus seem that defining and combining probabilities can be done in a unique way, without any questions.
However, there is still a lot of disagreement on the crucial issue: the interpretation of probability. The problem here is that interpretation means connection to the real world. In that respect, the issue is not just a technical one but also a philosophical one, which explains why there can be many different points of view.

First of all, it is possible to distinguish between the objective and the subjective points of view:

• Objective probability: the objective point of view consists in postulating that probabilities do not depend on the person observing events or performing experiments. This means that there exists some absolute notion of probability for every possible event and this probability originates from Nature itself. Once this is assumed, the question becomes : how to "measure" these pre-existing probabilities, or how to confirm that the probability of a given event has a given value?
• Subjective probability: in the subjective point of view, probabilities are not something that can be measured, but something one assumes. The idea is that events either occur or do not occur and the probability is not a property of Nature but rather a convenient way of representing someone's uncertainty prior to the event actually occurring.

There are two classical (and opposed) ways of interpreting probabilities: the frequency and Bayesian interpretations.

• Probability as frequency: in this approach, the probability of an event is defined as the ratio of how  many times the event occurs to the number of times a similar experiment is performed. For example, if you repeatedly flip a coin, the probability of this coin landing on "heads" will be defined as the percentage of trials where it does land on "heads". Of course, this will highly depend on the number of such trials and may vary from one sequence to another. However, this issue is solved by the theorem called "the law of large numbers" which essentially states that the frequency of an event in successive independent trials will converge to a fixed value (its probability). In other words, if you flip your coin again and again, the frequency will (slowly but surely) converge to a definite value. There are some issues about the definition of independent trials and about the fact that one can really perform successive experiments in exactly the same way, but we will not worry about this now.
• Bayesian probability: it is obvious that not all notions of probability (as they are used in every day life) can be properly captured by the frequency definition given above. For example, when one speaks about the probability of an event that may occur only once (hence it is not possible to perform repeated experiments) such as the probability of a politician being elected at a given election, it is clear that frequency does not make practical sense and cannot be tested. Another issue with frequency is that it makes sense in the limit only: say we start flipping a coin and it keeps landing heads up; how many times does it need to land heads up before we decide that this is not happening with probability 1/2?  Five?  Ten?  A thousand?  A million? There is no reasonable answer to this question. Hence (subjective) Bayesians do not attempt to measure probabilities, rather they consider that a probability is a "degree of belief" that someone may have in the fact that a given event will occur. The whole point is that how you obtain your "prior" probability or initial degree of belief (before observing anything) does not matter. What matters is how these values are combined and updated when events are observed.

There is of course a lot to be said about the above two interpretations and there are many refinements or deviations from these. I hope to be able to explore this in more details in later posts.

Comments

[skilop]

I disagree that for Bayesians "how you obtain prior probability" does not matter. The only thing that a Bayesian would say is that the probability theory - being a purely mathematical theory - does not provide any advice in that matter; it ensures internal consistency, not external validity of inference.

However, prior probabilities matters a lot in any real application, and the Bayesian methods employed for prior probability assignments have a "frequentist" touch. Not in the sense of repeatedly performing some random experiment, but in the sense of considering combinatorics, symmetries, and populations. See for example the priniciple of maximum entropy, which is related to the idea of multiplicities, which in turn is simply based on counting and sets.

[buy cialis]

next month I will present my graduation project. i was looking for about this long. your blog looks really interesting for me what I want to congratulate all the information and the work I do on his blog. great work

[Frieda]

This is why i visit you site daily. Great read thanks. Incidently http://www.rapidhawk.com has a similar topic.

[Ben]

Good Morning dude, i like your news site ! Have a nice day

[Supra Shoes]

The ideal is to strike a medium between ideology and inspiration.

[hanimant patio furniture]

Seeing things in a clearer side is already possibility.

[mi0sev]

Great minds thing alike, i agree with you mate, probability is a wide unkown fir me but you enlightened my mind

[Global Stores]

Thank you very much I really appreciate your approach in business. This is how business should be done.

[Laptop Blog Canada]

Firstly thank you for the quick setup of my web site and email accounts. Thanks! Secondly, the mail setup went quite smoothly. Also thanks.

[Omega Review]

Thanks for all the great service and customer attention, I appreciated the support and haven't found better. I intend to use your hosting for future projects.