Logistic Regression - Classifying with Probabilities

Logistic Regression differs from linear regression in that it does not predict numerical values. Instead, it is a machine learning algorithm that classifies data into specific categories.

For example, Logistic Regression can be used to answer the question "Is this email a spam?" by classifying it as either 0 (not spam) or 1 (spam).

Logistic Regression not only outputs a 0 or 1 but rather calculates the probability that the email is spam, and if it exceeds a certain threshold, it is classified into that category.

Logistic Regression, like linear regression, learns weights (W) and a bias (B) for the input data, but transforms the result into a probability value between 0 and 1.

The function used for this transformation is the Sigmoid function.

Logistic Regression takes the output of a linear regression and applies the following Sigmoid function to convert it into a probability value between 0 and 1.

It's okay if you don't fully understand the formula below. Understanding how logistic regression is structured like linear regression is the key takeaway.

$$\sigma(z) = \frac{1}{1 + e^{-z}}$$

Here, $z$ is calculated in the same manner as in linear regression.

$$z = W X + B$$

Consequently, the final prediction of Logistic Regression can be expressed as follows.

$$P(Y=1|X) = \frac{1}{1 + e^{-(WX + B)}}$$

This value is converted into a probability between 0 and 1, and classification is determined based on a specific threshold (commonly 0.5).

Copy
Input X = Email content
Prediction P(Y=1|X) = 0.85 → 85% probability of being spam

Logistic Regression is used in various real-world classification problems such as disease prediction and credit card fraud detection.

In the next lesson, we will explore Decision Trees.