202405202141
Status: #idea
Tags: Regression Analysis

Test On Individual Regression Coefficient (Assuming We know our Model Is Significant)

We test the hypothesis that
$H_0:{\beta }_j=0$
$H_1:{\beta }_j\ne 0$
We know that our $\hat{\beta }$ is distributed according to a normal distribution with mean $\beta$ since it is a BLUE Estimator and that it has variance ${\sigma }^2(x^{T}x{)}^{-1}$.

By this logic we know that all the elements in the $\hat{\beta }$ vector have to be normally distributed as well. So like in Simple Linear Regression we can simply use z-tests and t-tests to find our answer.

Except we lack the variance for a specific ${\beta }_j$ or do we? We need to observe that the variance of $\hat{\beta }$ is ${\sigma }^2$ $\times$ $(x^{T}x{)}^{-1}$ which is really just the covariance matrix of all ${\beta }_i$. To find the variance for a specific ${\beta }_j$ we simply index this matrix at the diagonal such that the variance of ${\beta }_j$ is $\text{covariance matrix @ index (j,j)}$ which is $(x^{T}x{)}_{jj}^{-1}$.

From there we can easily compute our statistic, keep in mind that in the image below we're checking for significance of the slope so we are assuming ${\beta }_j$ is $0$. If the test was say ${\beta }_j=1$ then we'd subtract 1 from ${\hat{\beta }}_j$.

As usual we do not have ${\sigma }^2$ in pretty much all the cases, so we use $MSE$ instead which is computed by as follows $\frac{SSE}{n-k}$ where $k$ is the number of parameters. Look at The Method Of Least Squares in Multiple Linear Regression for the specific formulas.

Obviously, the $t$ distribution it follows will have $n-k$ number of parameters.

Accustom yourself with the following image

Relevant Links