202405031405
Status: #idea
Tags: Regression

Simple Linear Regression

The above represent the standard equation of a Simple Linear Regression. Since we typically don't know the exact Y, ${\beta }_0$ and ${\beta }_1$ we will typically write them with a ^ on them.

For the purposes of SLE the parameters are the following:
Y : random variable (dependent variable, response)
X : fixed variable, the thing we control (independent variable, regressor, predictor)
${\beta }_0$ and ${\beta }_1$: Fixed parameters (true value is unknown)
$ϵ$: Random error term (Random Variable)
The data is arranged in pairs $(X_1,Y_1),(X_2,Y_2),\dots ,(X_n,Y_n)$

While the term Linear Regression might lead you to assume that all regressions will have the form $y=mx+b$ we would also call linear a regression of the form $y=n{x}^2+mx+b$, while we have quadratic term, the parameters ($n,m,b$) are all raised to the $1$ AND the equation is written as a summation of the parameters with some coefficient. Also please be keenly aware that a line that fits neatly does not mean anything!

We could relate stock market prices in N.Y to the price of bananas in an offshore island, it does not mean anything and it would be silly to make any financial decisions based on that.

Assumptions of Simple Linear Regression
The Method Of Least Squares
Goodness of Fit

Relevant Links

Simple Linear Regression Intuitions by the Goat JBStats
Derivating the OLS coefficients Video 1
Derivating the OLS coefficiences Video 2
How To Show Unbiasedness of Estimators for m and b