General Equation of Linear Regressor
$$ E[Y|X] = f(X)= \beta_0 +\beta_1X_1+\epsilon $$
Error in Linear Regressor
$$ e_i = y_i-\^{y}_i $$
$$ RSS = e^2_1+e^2_2+....+e^2_n=E[(Y-\^{Y})^2 |X] $$
Estimation of parameters:
$$ \hat{\beta}1 = \frac{\sum{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2} = \frac{S_{XY}}{S_{X^2}}
$$
$$ \hat{\beta}_0 = \bar{y} - \hat{\beta}_1 \bar{x} $$
$$ \bar{y} = \frac{1}{n}\displaystyle\sum_1^n y_i $$
$$ \bar{x} = \frac{1}{n}\displaystyle\sum_1^n x_i $$
Standard error of estimators:
$$ SE(\hat{\beta}1)^2 = \frac{\sigma^2}{\sum{i=1}^{n} (x_i - \bar{x})^2} $$
$$ SE(\hat{\beta}0)^2 = \frac{1}{n} \sum{i=1}^{n} \sigma^2 \left( \frac{1}{n} + \frac{\bar{x}^2}{\sum_{i=1}^{n} (x_i - \bar{x})^2} \right) $$
$$ \sigma^2 = Var(\epsilon) $$
Confidence Intervals:
$$ [\;\hat\beta_1\pm2*SE(\hat\beta_1)\;] $$
More Generalized Confidence Intervals
$$ [\;\hat\beta_1\pm t_{n-2,\alpha/2}*SE(\hat\beta_1)\;] $$
Hypothesis Testing:
Assume
$$ H_0: \beta_1 = 0 \newline H_A:B_1\ne0 $$
We get that
$$ t_{obs} = \frac{\hat\beta_1 - 0}{SE)\hat\beta_1} ;\; given \;\beta_1=0 $$
Reject if:
$$ |t_{obs}| > t_{n-2,\alpha/2} $$
Types of Error:
$Type \;I:$ Convicting someone of a crime when they are actually innocent. (Failing to Reject Null Hypothesis)
$Type \;II:$ Acquitting a guilty individual when they are guilty of a crime. (Rejecting the Null Hypothesis even when it was True.)
Acquitting a guilty individual when they are guilty of a crime.
F-Test:
It is for overall Significance of the model.
$$ \mathbf{F} = \frac{(TSS-RSS)/p}{RSS/n-p-1} $$
Rejects $\mathbf{H_0} \;if\;\; \mathbf{F}>\mathbf{F}_{p,n-p-1,\alpha}$