h;=
202405271904
Status: #reference
Tags: Probability Theory

MAT3172 ~ Foundations of Probability

Lecture 1

Professor Mahmoud Zarepour (great teacher by the way if you have the option to take him) starts with an introduction of set theory since it seems nothing serious can be done in math without some Set Theory.

He introduces the need for the rigorous Kolmogorov definition of probability that came as a way to unify a field that until now was quite disparate due to people working under subtly different assumptions, to solve agreed upon problems. As a result, contradictions would prop up and many problems would have different but properly reasoned answers.

In this video, he goes over the following points concluding with the concepts of $inf$ and $sup$ of both sequences and sets.

Lesson Points:

Outcome
Event
Set Theory
Indicator Functions
Limits of Sequences
Limits of Sets

Lecture 2

We start the lecture with the above screenshot which gives us a reminder of the essential facts. And sets the workbench for the remainder of the class.

We introduce the fact that for an increasing sequence of sets say $B_n$, it's limit will be $\bigcup ^{\mathrm{\infty }}{B}_n$
On the other side, if we have a decreasing sequence of sets say $D_n$, it's limit will be $\bigcup ^{\mathrm{\infty }}{D}_n$.

Here we motivate the use of Borel-Sigma Algebras for our definitions of probability. More specifically we introduce the Cantor Set and show how there are many weird sets that can be constructed on the $\mathbb{R}$ line that are not suitable for probability. While the Cantor Set itself is borel there are sets that can be constructed from it which are not Borel Sets, and therefore sets we cannot define Lebesgue Measures on (not yet introduced in this class.)

He also reassures us that while we've been doing Real Analysis and Calculus for these two lectures, and we will keep seeing to have a rigorous understanding of the theorems, we will be evaluated on our understanding of Probability Theory as it pertains to Probability.

Lesson Points

Limits of Sets
Borel-Sigma Algebras
Borel Sets

Lecture 3

Here we start defining the notion of probability measures, and we start by conceiving them as set functions. This is a crucial reminder of the fact that we assign probabilities to events and not individual outcomes.

We then spend the rest of the lecture defining probability measures (without really mentioning measure theory) both from the inherited version from Measure Theory and simple measures and from the more "intuitive" interpretation of expected value.

Really insightful class, and after having seen how some things like the formula for $P(\bigcup _{i=1}^{\mathrm{\infty }}{A}_i)$ are proven using the standard Kolmogorov approach, the existence of a more intuitive approach that makes such proofs almost trivially simple is very welcome.

Lesson Points:

Probability Measure (According to Kolmogorov)
Probability Measure (Based on Expected Value)

Lecture 4

After defining the probability measures, we ignore them for a bit to return to limits and their usage. Starting with Fatou's Lemma. We spend the first few minutes of the class going over it and explaining how it follows.

We then define independence of events.

It seems we define it only so that we can prove the Borel-Cantelli Lemma right after. Honestly, considering this is a probability class, there are worse moments that Dr Zahrepour could have chose to present it, lmao.

Overall, this lecture focused on these points.

Lesson Points
Fatou's Lemma
Event#Independence of Events
Borel-Cantelli Lemma

Lecture 5

After proving the Fatou's Lemma which give us the ability to take the limit in and out of measures given the limit of a family of event $E_n$ exists and then proving the Borel-Cantelli Lemma which gave us a way to say decisively if a given family of event $E_n$ did not occur infinitely many times (and under independence they did), we now start talking about what we typically think of when mentioning probability. Probability Mass Function and Probability Density Functions, and how the heck do we measure a probability for arbitrary events?

This lecture does not break what we've learned before in older courses, but it re-contextualizes a lot of it and give us a more rigorous underpinning. Less rigorous than Measure Theory, but rigorous enough for use.

Lesson Points

Borel-Cantelli Lemma
Random Variable
Cumulative Density Function
Riemann Integrals
Riemann-Stieltjes Integrals ~ Generalization of Riemann integrals
Lebesgue Integrals
Expected Value
Probability Density Functions
Markov's Inequality

Lecture 6

So we start where we left off last lecture with Markov's Inequality.
From Lecture 6 and from there and then we start our journey into stating and proving inequalities.

Besides inequalities galore, we introduce the concept of Moment Generating Function and we prove the moment generating functions for well known distributions like the Gamma Distribution, and the Chi-Squared Distribution.

Lesson Points

Markov's Inequality
Chebyshev's Inequality
Cauchy-Schwarz's Inequality
Moment Generating Function

Lecture 7

We state the really important fact that there's a 1-1 correspondence between the pdf $f_X$ of a random variable $X$ and it's random variable. You can multiply the MGFs of random variables only when they are independent, similarly to densities.

The MGF is also unique. We knew that, but this is restated here. After that we go on a rampage, covering distributions after distributions. Before returning to inequalities at the end. If it wasn't clear, from MAT21... moment generating functions are the GOAT and you should memorize them.

We finish by explaining the rational behind Holder's Inequality and comparing it to Cauchy-Schwarz's Inequality.

Lesson Points:

Moment Generating Function
Gamma Distribution
Chi-Squared Distribution
Bernoulli Distribution
Binomial Distribution
Normal Distribution
Holder's Inequality

Lecture 8

We start by reviewing Holder's Inequality

Lesson Points

Holder's Inequality
Jensen's Inequality
Liapunnov's Inequality

Lecture 9

We spend this lecture on the glorious Cumulative Density Function, the mother of probability and the thing that props all the time. All roads lead to Rome, no, all rome leads to the CDF.
First, it is defined as follows $F(x)=P(X\le x)$ for some random variable $X$, then we can see the following properties of $cdf$:

1. Non-decreasing
2. Right continuous
3. $F(\mathrm{\infty })=1$ and $F(-\mathrm{\infty })=0$
   If any of the above fails we do not have a cdf, and we prove those properties. We especially explain why we need right-continuous, and show how this property emerges from our definition of cdfs, and Fatou's Lemma.

After that we take our time proving results about cdfs to make our understanding of them richer.

Lesson Points

Cumulative Density Function
[[The Number of Discontinuities in a CDF can be At Most Countably Infinite
All CDFs are either Continuous, Discrete or Composed of Discrete and Continuous Density Functions

Lecture 10

This is a continuity on our lecture on how any CDF is either continuous, discrete or partitionable into continuous and discrete parts.

Lesson Points

Cumulative Density Function
All CDFs are either Continuous, Discrete or Composed of Discrete and Continuous Density Functions
Inverse Transformations

Relevant Links

MAT3172 ~ Flashcards

Prerequisites (I took):

MAT2122 ~ Multivariable Calculus
MAT2341 ~ Introduction to Applied Linear Algebra
MAT2371 ~ Introduction to Probability