Definition (Probability Space).

The probability space $(\Omega,\mathcal{F},\mathbb{P})$ is defined as follows:

• \(\Omega\): sample space, the set of all possible outcomes.
• \(\mathcal{F}\): \(\sigma\)-field (\(\sigma\)-algebra) on \(\Omega\) or collection of events.
• \(\mathbb{P}\): Probability measure on \((\Omega,\mathcal{F})\).

Definition ($\sigma$-algebra).

For a sample space \(\Omega\), \(\mathcal{F}\) is said to be a \(\sigma\)-algebra if it satisfies:

• \(\Omega \in \mathcal{F}\);
• closed under complement, that is, if \(E\in \mathcal{F}\), then \(E^c \in \mathcal{F}\);
• closed under countable unions of events, that is, \(E_i \in \mathcal{F}\), \(i=1,2,\cdots\), then \(\cup_iE_i \in \mathcal{F}\).

Remark: \(\mathcal{F}\) is collection of sets (For example, \(\Omega = \{a,b,c,d\}\), \(\mathcal{F}=\{\emptyset,\Omega,\{a,b\},\{c,d\}\}\)).

Definition (Probability).

Any event \(E \in \Omega\), \(\mathbb{P}(E)\) is called the probability of the event \(E\) if it satisffies:

• \(0 \leq \mathbb{P}(E) \leq 1\);
• If \(E_i \in \mathcal{F}\), \(E_i \cap E_j = \emptyset\) for \(i\neq j\) (mutually disjoint events), then \(\mathbb{P}(\cup E_i)=\sum_i\mathbb{P}(E_i)\); (概率的可数可加性)
• \(\mathbb{P}(\Omega) = 1\). (\(\mathbb{P}(\emptyset) = 0\))

Remark: \(\mathbb{P}(\cup_iE_i)\leq \sum_i\mathbb{P}(E_i)\) (subadditivitiy property or Boole’s inequality).

Definition (Infimum and Supremum, 上下确界).

Suppose \(\{A_n\}_{n=1}^{\infty}\) is a sequence of sets.

Define：

\[\text{(All except finitely often, a.e.f.o.)} \quad \lim_\limits{n\to \infty}\inf A_n = \cup_{n\geq 1}\cap_{j \geq n}A_j \]

and

\[\text{(Infinitely Often, i.o.)} \quad \lim\limits_{n \to \infty}\sup A_n = \cap_{n \geq 1}\cup_{j \geq n} A_j \]

---

Remark:

• (a.e.f.o)

\[\begin{aligned} &x \in \lim_\limits{n\to \infty}\inf A_n = \cup_{n\geq 1}\cap_{j \geq n}A_j \newline &\Longleftrightarrow \exists n_i,x \in \cap_{j \geq n_i}A_j, \text{for some } i \newline &\Longleftrightarrow \exists n_i,x \in A_{n_i},A_{n_{i+1}}\cdots,A_{\infty} \end{aligned} \]

• (i.o.)

  \[\begin{aligned}&x \in \lim_\limits{n\to \infty}\sup A_n = \cap_{n\geq 1}\cup_{j \geq n}A_j \newline &\Longleftrightarrow \forall n,x \in \cup_{j \geq n}A_j \newline &\Longleftrightarrow \forall n, \exists m\geq n, x \in A_m \end{aligned} \]

---

Simple Proof: (Why the infimum and superemum is defined as above?)

Let \(L_n=\cap_{j\geq n}A_j\), \(U_n = \cup_{j\geq n}A_j\), then \(L_n \leq A_n \leq U_n\).

Taking \(\lim\limits_{n\to\infty}\) we have: \(\lim\limits_{n\to\infty}L_n \leq \lim\limits_{n\to\infty}A_n \leq \lim\limits_{n\to\infty} U_n\).

Since \(\{L_n\}\) and \(\{U_n\}\) are monotonically increasing and decreasing sequences seperately, by the continuity property of \(\mathbb{P}\), we have:

\[\underbrace{\lim\limits_{n\to\infty}\cup_{n\geq1}\cap_{j\geq n}A_j}_{\lim_\limits{n\to \infty}\inf A_n} \leq \lim\limits_{n\to\infty}A_n \leq \underbrace{\lim\limits_{n\to\infty}\cap_{n\geq1}\cup_{j\geq n}A_j}_{\lim_\limits{n\to \infty}\sup A_n} \]

参考：Limits of Sets

Definition (Random variable, Distribution function).

A random variable \(X\) is a mapping from \(\Omega\) to \(\mathbb{R}\), and for any Borel Set \(A\subset \mathbb{R}\) (这里等价认为，\((-\infty,a],\forall a \in \mathbb{R}\)),

\[\mathbb{P}(X(\omega)\in A) = \mathbb{P}(X^{-1}(A))=\mathbb{P}\{\omega:X(\omega)\in A\}. \]

The distribution function \(F\) of a random variable \(X\) is defined for any real number \(x \in \mathbb{R}\),

\[F(x) = \mathbb{P}(X \leq x)=\mathbb{P}(X \in (-\infty,x]), \]

and the survival (or tail) function is \(\overline{F}(x)=1-F(x)=\mathbb{P}(X > x)\).

---

Properties of distribution function:

• Right-continuous; Non-decreasing (\(\lim_\limits{x\to+\infty}F(x)=1,\lim_\limits{x\to-\infty}F(x)=0\));
• If \(X\) is discrete random variable, \(F(x)=\sum_{z\leq x}\mathbb{P}(x)\). If \(X\) takes uncountably many values, summation makes NO sense.

---

(Continuous Random Variable). \(X\) is said to be a continuous random variable if there exists a function \(f(x)\) such that for any Borel set \(B \subset \mathbb{R}\),

\[\mathbb{P}(X \subset B)=\int_Bf(x)dx. \]

\(f(x)\) is called the probability density function (pdf). For continuous r.v. \(X\),

\[F(x) = \int_{-\infty}^xf(y)dy \]

and

\[f(x) = F^{\prime} (x). \]

---

(Joint distribution). For random variables \(X\), \(Y\) , the joint distribution is defined by

\[F(x,y)=\mathbb{P}(X\leq x,Y \leq y). \]

Correspondingly, the distributions (marginal distributions) of \(X,Y\) are:

\[\begin{align} F_X(x)=\mathbb{P}(X\leq x)=\lim\limits_{y \to \infty}\mathbb{P}(X\leq x, Y \leq y)=\lim\limits_{y \to \infty}F(x,y) \newline F_Y(y)=\mathbb{P}(Y\leq y)=\lim\limits_{x \to \infty}\mathbb{P}(X\leq x, Y \leq y)=\lim\limits_{x \to \infty}F(x,y) \end{align} \]

Definition (Expectation).

For a random variable \(X\), if \(\int_{-\infty}^\infty xdF(x)\) exists, then we say it is the expectation or mean of \(X\), denoted by \(\mathbb{E}[X]\).

\[\mathbb{E}[X]=\int_{-\infty}^{\infty} x d F(x)=\left\{ \begin{array}{ll}\Sigma_{x} x \mathbb{P}(X=x) & \text { if } X \text { is discrete. } \newline \int_{-\infty}^{\infty} x f(x) d x & \text { if } X \text { is continuous. }\end{array}\right. \]

1
2

一、集合\{\}的转义问题: 使用结合goldmark-mathjax的hugo解决（该PR没有被hugo合并）。https://github.com/xjzsq/hugo
二、公式换行：必须使用\begin{aligned} \newline \end{aligned}或\begin{align} \newline \end{align}

1
2

一、Definition 使用 \{\{% admonition tip "Definition XXX" %\}\} \{\{% /admonition %\}\}
二、Proposition, Remark 使用 \{\{% admonition info "Definition XXX" %\}\} \{\{% /admonition %\}\}