Hypothesis Testing

It the the action of declaring a hypothesis (a fundamental logical component) and proving its occurrence of proving its non-existence.

Types of Hypothesis

Null Hypothesis

H₀ : μ1 = μ2x

The means of the two groups (μ1 & μ2) belong to the same population. (p > 0.05)

Alternative Hypothesis

H_A : μ1 ≠ μ2
The means of the two groups (μ1 & μ2) belong to different populations. In case of multiple groups, the mean of all groups shouldn’t be the same. At least one group should be different. (p < 0.05)

Significance level : α

The significance level, also denoted as alpha or α, is the probability of rejecting the null hypothesis when it is true. Typically 0.05 (5%)

Confidence level : (1 - α)

Errors

Type I : Rejecting the null hypothesis even if it’s true False Positive

Type II : Accepting the null hypothesis even when it’s false False Negative

Probability (type I error) = α | Probability (type II error) = β

Analogy

Person on trial :
Null = He is innocent    Alternate = He is guilty
Type I : reject the null when true : he is innocent but rejected : innocent person in jail ⛔ avoid
Type II : accept the null when false : he is guilty but declared innocent : guilty person free

Person with cancer :
Null = Doesn't have cancer    Alternate = Has cancer
Type I : reject the null when true : doesn’t have but detected : person lives
Type II : accept the null when false : has but not detected : person dies ⛔ avoid

Misclassification Errors (Confusion Matrix)