Artificial Intelligence & Machine Learning
Hyperparameter
Definition
A hyperparameter is a parameter whose value is used to control the learning process of a machine learning model. Its value is set before the learning process begins.
Why It Matters
Hyperparameters have a significant impact on model performance. Choosing the right hyperparameters (a process called "hyperparameter tuning") is crucial for building a good model.
Contextual Example
In a k-Nearest Neighbors algorithm, the number of neighbors "k" is a hyperparameter. In a neural network, the "learning rate" and the number of layers are hyperparameters.
Common Misunderstandings
- Hyperparameters are different from model parameters. Model parameters (like the weights in a neural network) are learned from the data during training. Hyperparameters are set by the data scientist before training.
- Hyperparameter tuning is often done using techniques like Grid Search or Random Search.