ML coding · Ranking and evaluation metrics
Macro-averaged F1 across classes
Given true labels y_true and predictions y_pred (integers), let the classes be
every label that appears in either list. For each class c, F1 = 2TP / (2TP + FP + FN), counting TP, FP and FN for c
against all other classes. Return the unweighted mean of the per-class F1 scores.
Examples
Input y_true = [0, 0, 1, 1, 2], y_pred = [0, 1, 1, 1, 0]
Output 0.4333
Class 2 is never predicted, so its F1 is 0 and it pulls the mean down.
Input y_true = [5, 5, 5], y_pred = [5, 5, 5]
Output 1
One class, all correct: 1.0.
Constraints
- 1 ≤ n ≤ 10⁶
- Labels are integers
- Checked to a relative tolerance of 10⁻¹²
Hints
Hint 1
np.union1d(y_true, y_pred) gives the classes.
Hint 2
For each class, TP is the count where both equal c; FP where only the prediction does; FN where only the label does.
Hint 3
Every class in the union appears at least once, so 2TP + FP + FN is never zero.
Where candidates go wrong
- Taking the classes from y_true only: a class that is predicted but never true is silently ignored.
- Computing precision and recall first and dividing by zero when a class is never predicted.
- Returning micro-F1, which for single-label problems is just accuracy.
The idea
Get the classes as the union of both arrays. For each class, compute TP, FP and FN with boolean masks, then F1 as 2TP / (2TP + FP + FN), a form that never divides by zero here. Average the scores.
Time O(n · C) · Space O(n)
Reference solution
import numpy as np
def macro_f1(y_true, y_pred):
t = np.asarray(y_true)
p = np.asarray(y_pred)
scores = []
for c in np.union1d(t, p):
tp = np.sum((t == c) & (p == c))
fp = np.sum((t != c) & (p == c))
fn = np.sum((t == c) & (p != c))
scores.append(2 * tp / (2 * tp + fp + fn))
return float(np.mean(scores))Follow-ups an interviewer asks
- Vectorise over classes with a confusion matrix built by np.bincount on y_true · C + y_pred.
- Weighted F1: average by each class's support. When does it hide a failing class?
- F-beta, for when recall matters more than precision.
Python runs in your browser with NumPy. The first run downloads the runtime, which takes a few seconds.
Why interviewers ask it
The default metric for imbalanced classification, and a quick test of whether a candidate knows the difference between macro, micro and weighted averaging.
The theory behind it is in the lesson NDCG, calibration error and macro-F1 from scratch.
Keep going
NDCG at k
Normalised discounted cumulative gain for a ranked list: exponential gains, a log discount, and the ideal ranking computed from all items..
Open the problem → Medium15 minExpected calibration error
Expected calibration error for binary predictions with equal-width bins, empty bins skipped, and every bin edge defined exactly..
Open the problem → Easy10 minMacro-averaged F1 across classes
Per-class F1 from a confusion of labels and predictions, averaged over every class that appears in either, without scikit-learn..
Open the problem →