Weak Law of Large Numbers

Theorem 2.2.3 ( L2 weak Law ) Let $X_1, X_2, \dots$ be uncorrelated random variables with $EX_i =\mu$ and $var(X_i)<\infty$. If $S_n=X_1 + \dots X_n$ then as $n\rightarrow\infty$, $S_n/n \rightarrow \mu$ in $L^2$ in probability.

Theorem 2.2.6 Let $\mu_n = ES_n$ , $\sigma^2_n = var S_n$. If $ \sigma^2/b^2_n \rightarrow 0$ then

$$ \frac{S_n-\mu_n}{b_n} \rightarrow 0 \quad\text{in probability}$$ 

Theorem 2.2.11 ( Weak Law for Triangular arryas) For each $n$ let $X_{n,k}$ be independent. Let $b_n>0$ with $b_n\rightarrow \infty$ and let $\bar{X}_{n,k}=X_{n,k}1(|X_{n,k}|\leq b_n)$.

Suppose $n\rightarrow\infty$

(i) $\sum\limits^n_{k=1} P( |X_{n,k}|>b_n) \rightarrow 0 $ , and

(ii) $b_n^{-2} \sum\limits^n_{k=1} E \bar{X}^2_{n,k} \rightarrow 0 $

If we let $S_n = X_{n,1}+ \dots + X_{n.n}$ and put $a_n = \sum\limits^n_1 E \bar{X}_{n,k}$. Then

$$ (S_n - a_n)/b_n \rightarrow 0 \quad \text{in probability}$$ 

Proof)

Let $ \bar{S}_n = \bar{X}_{n,1} + \dots \bar{X}_{n,n}$

$$ P(|  \frac{S_n-a_n}{b_n}  | > \epsilon ) \leq P(S_n \neq \bar{S}_n) + P(\frac{\bar{S}_n - a_n}{b_n}) $$

Useful trick here is 

$P( | \frac{S_n-a_n}{b_n}  | > \epsilon ) \leq P ( | \frac{\bar{S}_n - a_n}{b_n} |> \epsilon ) + P ( S_n \neq \bar{S}_n)$ 

The first part can be bounded by Markov Inequality.

The second part implies $P(\bar{S}_n \neq S_n)  \leq P ( \exists{i} \quad\text{s.t}\quad \bar{X}_{n,i}=X_{n,i})$

Using union bound we can get $P(S_n \neq \bar{S}_n) \rightarrow 0$

Theorem 2.2.12 (Weak Law of Large Numbers) $X_1,X_2 \dots$ be i.i.d with

$$ x P( |X_i} > x ) \rightarrow 0 \quad \text{a.s} \quad x \rightarrow 0 $$