Example: LR
Logistic Regression — Mathematical Example
Problem Statement
Predict whether a student will pass or fail based on study hours.
0 = Fail
1 = Pass
Dataset
| Study Hours (x) | Result (y) |
|---|---|
| 1 | 0 |
| 2 | 0 |
| 3 | 0 |
| 4 | 1 |
| 5 | 1 |
Step 1: Logistic Regression Formula
Logistic Regression uses the sigmoid function:
P(y = 1) = 1 / (1 + e^(-z))
Where:
z = b0 + b1x
Step 2: Assume Model Values
Suppose the model learned:
b0 = -6
b1 = 2
So:
z = -6 + 2x
Step 3: Predict for a New Student
Suppose:
Study Hours = 4
Substitute x = 4:
z = -6 + 2(4)
z = -6 + 8
z = 2
Step 4: Apply Sigmoid Function
P(y = 1) = 1 / (1 + e^(-2))
Approximate value:
e^(-2) = 0.135
So:
P(y = 1) = 1 / (1 + 0.135)
P(y = 1) = 1 / 1.135
P(y = 1) = 0.881
Step 5: Apply Decision Rule
Usually, threshold is:
0.5
Decision rule:
If probability >= 0.5 → Predict 1
If probability < 0.5 → Predict 0
Here:
0.881 >= 0.5
So prediction is:
1
Final Prediction
The student is predicted to PASS.
Another Example
Predict for:
Study Hours = 2
Calculate z:
z = -6 + 2(2)
z = -6 + 4
z = -2
Apply sigmoid:
P(y = 1) = 1 / (1 + e^(-(-2)))
P(y = 1) = 1 / (1 + e^2)
Approximate:
e^2 = 7.389
P(y = 1) = 1 / (1 + 7.389)
P(y = 1) = 1 / 8.389
P(y = 1) = 0.119
Since:
0.119 < 0.5
Prediction is:
0
Final Prediction
The student is predicted to FAIL.
Summary
In Logistic Regression, the model first calculates a linear value z. Then the sigmoid function converts this value into a probability between 0 and 1. Finally, a threshold value, usually 0.5, is used to classify the output as class 0 or class 1.
