Artificial Intelligence & Machine Learning
Batch Size
Definition
In machine learning, the batch size is a hyperparameter that defines the number of samples to work through before updating the model's internal parameters. The dataset is broken down into one or more batches.
Why It Matters
The batch size is a key hyperparameter that affects the speed and stability of the training process. A larger batch size can lead to faster training but requires more memory, while a smaller batch size can be slower but may help the model generalize better.
Contextual Example
A dataset has 1000 samples. If the batch size is 100, the model will be updated 10 times per epoch (one full pass through the dataset). If the batch size is 10, it will be updated 100 times.
Common Misunderstandings
- Stochastic Gradient Descent (SGD) technically uses a batch size of 1.
- Mini-batch gradient descent (using a batch size between 1 and the full dataset size) is the most common approach.