Artificial Intelligence & Machine Learning
k-Nearest Neighbors (k-NN)
Definition
k-Nearest Neighbors (k-NN) is a simple, supervised machine learning algorithm that can be used for both classification and regression. It makes predictions by finding the "k" most similar examples in the training data and using their outcomes to predict the outcome for the new data point.
Why It Matters
k-NN is one of the simplest machine learning algorithms to understand and implement. It is a "lazy learner" because it doesn't build a model during training; it simply stores the entire training dataset.
Contextual Example
To classify a new email, a k-NN model might look at the 5 most similar emails in its training set (the 5 nearest neighbors). If 4 of those 5 were spam, it would classify the new email as spam.
Common Misunderstandings
- The "k" is a hyperparameter that you must choose (e.g., k=3, k=5).
- Because it has to compare a new data point to every single point in the training set, k-NN can be very slow for inference on large datasets.