A Brief Overview of Machine Learning Models

Supervised Learning

In supervised learning, we learn a mapping from inputs to outputs based on labeled examples. The input-output pairs in the training data act as "supervision" for the model, guiding it to predict outcomes for new data.

Mathematical Foundation of Supervised Learning

The goal is to find a function f such that:

where x is the input and y is the output. The model is trained using a dataset
D = {(x1, y1),(x2,y2),....,(xn,yn)}, where each pair consists of an input feature and its corresponding label.

Subcategories of Supervised Learning

1. Regression

Regression models predict continuous values. For example, predicting a person's height based on their age.

where:
y is the predicted output (e.g., height),
x is the input (e.g., age),
m is the slope of the line,
b is the y-intercept.

2. Classification

In classification, the goal is to predict discrete labels. For example, classifying emails as spam or not spam.

Decision Trees & Random Forests

Decision Trees: These break down a dataset into smaller, homogenous groups based on a series of questions or rules.

Random Forests: These combine multiple decision trees to reduce overfitting. Each tree in the forest makes a prediction, and the final output is the majority prediction from all trees.