Bias-Variance Tradeoff

Definition

In machine learning, the bias-variance tradeoff is the conflict in trying to simultaneously minimize two sources of error: bias (the error from erroneous assumptions in the learning algorithm) and variance (the error from sensitivity to small fluctuations in the training set).

Why It Matters

This is a fundamental concept in machine learning. A simple model has high bias and low variance (it underfits). A very complex model has low bias but high variance (it overfits). The goal is to find a model that has the best balance between the two for a given problem.

Contextual Example

A simple linear model might have high bias because it assumes the data is linear. A very deep decision tree might have high variance because it can perfectly fit the training data, including its noise.

Common Misunderstandings

• High bias leads to underfitting.
• High variance leads to overfitting.

Related Terms