Sets and Events

The core classical theorems in probability and statistics are the following: The law of large numbers (LLN): Suppose {X n ,n ≥ 1} are independent, identically distributed (iid) random variables with common mean E(X n ) = μ. The LLN says the sample average is approximately equal to the mean, so that 1 n ∑ i = 1 n X i → μ $$ \frac{1}{n}\sum\limits_{i = 1}^n {{X_i}} \to \mu $$ . An immediate concern is what does the convergence arrow “→” mean? This result has far-reaching consequences since, if X i { 1 , i f e v e n t A o c c u r s , 0 , o t h e r w i s e $$ {X_i}\left\{ {\begin{array}{*{20}{c}} {1,}&{ifeventAoccurs,} \\ {0,}&{otherwise} \end{array}} \right. $$ then the average ∑ i = 1 n X i / n $$ \sum\nolimits_{i = 1}^n {{X_i}} /n $$ is the relative frequency of occurrence of A in n repetitions of the experiment and μ = P(A). The LLN justifies the frequency interpretation of probabilities and much statistical estimation theory where it underlies the notion of consistency of an estimator. Central limit theorem (CLT): The central limit theorem assures us that sample averages when centered and scaled to have mean 0 and variance 1 have a distribution that is approximately normal. If {X n , n ≥ 1} are iid with common mean E(X n ) = μ and variance Var(X n ) = σ 2, then P [ ∑ i = 1 n X i − n μ σ n ⩽ x ] → N ( x ) : = ∫ − ∞ x e − u 2 / 2 2 π d u $$ P\left[ {\frac{{\sum\nolimits_{i = 1}^n {{X_i} - n\mu } }}{{\sigma \sqrt n }}x} \right] \to N(x): = \int_{ - \infty }^x {\frac{{{e^{ - {u^2}/2}}}}{{\sqrt {2\pi } }}} du $$ . This result is arguably the most important and most frequently applied result of probability and statistics. How is this result and its variants proved? Martingale convergence theorems and optional stopping: A martingale is a stochastic process {X n , n ≥ 0} used to model a fair sequence of gambles (or, as we say today, investments). The conditional expectation of your wealth X n +1 after the next gamble or investment given the past equals the current wealth X n . The martingale results on convergence and optimal stopping underlie the modern theory of stochastic processes and are essential tools in application areas such as mathematical finance. What are the basic results and why do they have such far reaching applicability?