Probability Theory: Concepts of Probability, Distributions, Moments, Central Limit Theorem – UGC NET Notes

Introduction to Probability Theory

Probability theory is the branch of mathematics that deals with the analysis of random phenomena and quantifies the likelihood of occurrence of uncertain events. In economics, probability theory forms the mathematical foundation for econometrics, statistical inference, decision theory under uncertainty, and risk analysis. The concept originated from the study of games of chance by mathematicians like Pascal and Fermat in the seventeenth century, and was later formalized axiomatically by Andrey Kolmogorov in 1933, who provided the modern axiomatic foundation of probability theory.

Basic Concepts of Probability

Random Experiment, Sample Space, and Events

A random experiment is any process of observation or experimentation that has more than one possible outcome, and it is not possible to predict with certainty which outcome will occur, although all possible outcomes are known in advance. Examples include tossing a coin, rolling a die, or drawing a card from a deck.

The sample space, denoted by (S) or \(\Omega\), is the set of all possible outcomes of a random experiment. For instance, when tossing a coin twice, the sample space is (S = {HH, HT, TH, TT}).

An event is any subset of the sample space. A simple event consists of a single outcome, while a compound event consists of more than one outcome. Events can be combined using set operations: union \(A \cup B\), intersection \(A \cap B\), and complement \(A^c\) or \(\bar{A}\).

Two events (A) and (B) are said to be mutually exclusive (or disjoint) if they cannot occur simultaneously, that is, \(A \cap B = \phi\). Events are said to be exhaustive if their union covers the entire sample space, that is, \(A_1 \cup A_2 \cup \dots \cup A_n = S\).

Classical, Relative Frequency, and Axiomatic Definitions of Probability

The classical (a priori) definition of probability, given by Laplace, states that if a random experiment has (n) mutually exclusive, exhaustive, and equally likely outcomes, and (m) of them are favorable to an event (A), then the probability of (A) is:

$$P(A) = \frac{m}{n} = \frac{\text{Number of favorable outcomes}}{\text{Total number of outcomes}}$$

This definition suffers from the limitation that it cannot be applied when outcomes are not equally likely, and it involves circular reasoning since “equally likely” itself presupposes the notion of probability.

The relative frequency (empirical) definition, associated with von Mises, defines probability as the limiting value of the relative frequency of occurrence of an event as the number of trials tends to infinity:

$$P(A) = \lim_{n \to \infty} \frac{f_A}{n}$$

where \(f_A\) is the number of times event (A) occurs in (n) trials. This is more practical but requires repeatability of experiments under identical conditions.

The axiomatic definition, given by Kolmogorov, defines probability as a function (P) that assigns to each event (A) in the sample space a real number \(P(A)\) satisfying three axioms:

1. Non-negativity: \(P(A) \geq 0\) for every event (A).
2. Normalization: \(P(S) = 1\), where (S) is the sample space.
3. Countable additivity: For mutually exclusive events \(A_1, A_2, \dots), (P(A_1 \cup A_2 \cup \dots) = P(A_1) + P(A_2) + \dots\)

This axiomatic approach avoids the circularity and practical limitations of the earlier definitions and forms the rigorous basis of modern probability theory.