Understanding the Difference between Loss Functions and Metrics in Machine Learning/Deep Learning
Yes! You read the heading right. There’s indeed a difference between loss functions and Metrics in the field of Machine Learning. However, these two terms are often used interchangeably. Read through as I share and explain the difference between the two important concepts with examples.
Introduction
As a beginner in the ML space, it is common to become perplexed by the use of certain terms. For a long time, I frequently used the terms “Loss function” and “Evaluation metrics” interchangeably, despite the fact that they refer to distinct concepts.
The primary goal of this article is to shed some light on the concepts and how they apply differently in building a Machine Learning model.
What is an Evaluation Metric?
An Evaluation Metric, also known as a "Criterion" is a method of evaluating/comparing the performance of a learning function. It's like a quantifier with which you can judge if a learning function for a given learning problem(or hypotheses class) is good or bad, based on standards. Example, Accuracy, F1 Score, Word Error Rate(WER), Bleu Score et cetera
N.b: “A Higher value does not always indicate a good performance, and lower value does not always indicate a bad performance”
What is a Loss function?
A loss function, is more like an “error” function that calculates how far apart the output/predicted value of a learning function deviates/differs from the ground truth/actual value. The main focus is usually “optimization(can either be a minimization or maximization problem)”.
“How far/close are we from the target destination?Are we moving in the right direction?”
For example, the Logistic/Sigmoid loss function, the hinge loss, Cross-Entropy loss, Mean Squared Error, CTC Loss, et cetera.
Spoiler Alert:
There are some loss functions that also play the role of an evaluation metrics i.e can be used to judge performance. You’ll often find these in regression learning problems. E.g MSE, MAE et cetera. Typically these are loss functions, however they also serve as a measure of performance.
Key Differences
- A loss function is implemented during training to optimize a learning function. It is not a judge of overall performance.
- A Criterion/Evaluation Metric is used after training to measure overall performance.
Interesting Fact: A confusion Matrix is neither a loss function nor an evaluation metrics. It’s simply a diagnostic tool that maps the type of errors a given learning function is making or inclined towards (How many False Positives Versus False Negatives)
Thank you for reading!!!
You can find my previous Articles Here: https://mardiyyah.medium.com
Connect on LinkedIN Here: https://www.linkedin.com/in/aminah-mardiyyah-rufa-i
