What is Supervised Learning?

The Core Idea

In supervised learning, the training data consists of input-output pairs — you provide both the question and the correct answer, and the model learns to map one to the other.

(X, y) pairs in training:
  ("warfarin 5mg daily", drug_class=anticoagulant)
  ("metformin 500mg BID", drug_class=antidiabetic)
  ("lisinopril 10mg", drug_class=antihypertensive)

Model learns: text → drug_class

The word "supervised" means a teacher (the labels) is guiding the learning process.

The Two Main Tasks

Classification — Predicting a Category

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import make_classification

# Binary classification: will patient need dose adjustment?
X, y = make_classification(n_samples=500, n_features=10, random_state=42)
# y: 0 = no adjustment needed, 1 = adjustment needed

model = LogisticRegression()
model.fit(X, y)

prediction = model.predict([[65, 78, 2.4, 1.1, ...]])   # [1] = needs adjustment
probability = model.predict_proba([[65, 78, ...]])       # [[0.23, 0.77]] = 77% chance

Examples: spam detection, drug adverse event classification, medical image diagnosis, sentiment analysis, churn prediction.

Regression — Predicting a Number

from sklearn.linear_model import LinearRegression
import numpy as np

# Regression: predict a patient's INR given current dose and vitals
X_train = np.array([
    [5.0, 65, 78, 1.1],   # dose_mg, age, weight, creatinine
    [3.0, 72, 85, 1.4],
    [7.5, 58, 62, 0.9],
])
y_train = np.array([2.8, 1.9, 3.5])   # INR values

model = LinearRegression()
model.fit(X_train, y_train)

predicted_inr = model.predict([[5.0, 68, 75, 1.2]])
print(f"Predicted INR: {predicted_inr[0]:.2f}")   # 2.54

Examples: predicting drug dose, estimating length of hospital stay, forecasting sales, estimating click-through rates.

Common Supervised Learning Algorithms

| Algorithm | Best For | Key Strength | |---|---|---| | Linear/Logistic Regression | Baseline; interpretable | Fast, coefficient insight | | Decision Tree | Non-linear, interpretable | Handles mixed types | | Random Forest | Tabular data, robust | Handles noise well | | Gradient Boosting (XGBoost, LightGBM) | Tabular competitions | Often best on structured data | | SVM | High-dimensional | Works well with small datasets | | k-NN | Simple baselines | No training, just distance | | Neural Networks | Images, text, audio | Learns complex representations |

The Supervised Learning Workflow

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import numpy as np

# 1. Collect labeled data
X = np.random.randn(1000, 15)   # 1000 patients, 15 features
y = np.random.randint(0, 3, 1000)  # 3 drug classes

# 2. Split into train/test
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

# 3. Choose and train a model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

# 4. Evaluate
y_pred = model.predict(X_test)
print(classification_report(y_test, y_pred,
      target_names=["anticoagulant", "antidiabetic", "antihypertensive"]))

# 5. Deploy and monitor

Assumptions in Supervised Learning

1. I.I.D. data — training examples are independent and identically distributed (drawn from the same distribution)
2. Training distribution ≈ inference distribution — if you train on 2020 data and deploy in 2026, data drift is a risk
3. Label quality — noisy labels (mislabeled examples) degrade model quality
4. Sufficient coverage — the training set must represent the full range of inputs the model will see at inference

Supervised Learning and LLMs

LLMs are trained with a form of supervised learning:

1. Pre-training (self-supervised): next-token prediction — the "labels" are the next tokens in the training corpus, auto-generated from text
2. Supervised Fine-Tuning (SFT): human-written (prompt, response) pairs — explicit labels
3. RLHF reward model: human preference labels (which response is better?) — explicit supervised signal

So when you use GPT-4, you're using a model that was fundamentally shaped by supervised learning at multiple stages.

Interview Answer Template

Q: What is supervised learning?

Supervised learning is training a model on labeled data — input-output pairs where the correct answer is provided for each example. The model learns to predict outputs from inputs by minimizing a loss function. The two main tasks are classification (predict a discrete category) and regression (predict a continuous value). Common algorithms include logistic regression, random forests, gradient boosting, and neural networks. LLMs are trained with a form of supervised learning — next-token prediction during pre-training and human-labeled prompt-response pairs during fine-tuning.