AI Terminology
/Advanced
Learning Rate
Definition
A critical hyperparameter that determines the step size at each iteration while moving toward a minimum of a loss function. It controls how much the model changes its mind in response to an error.
Explain Like I'm New
You are blindfolded trying to walk to the bottom of a valley. The Learning Rate is your stride length. If you take massive leaps (High Learning Rate), you will walk right past the bottom and start going up the other side. If you take tiny baby steps (Low Learning Rate), it will take you a million years to reach the bottom.
Real World Example
If an AI is told it misclassified a dog as a cat, a High learning rate makes it drastically rewrite its entire brain immediately. A Low learning rate makes it cautiously tweak one tiny synapse.
Common Use Cases
- •Model convergence
- •Hyperparameter tuning
Interview Questions
basic
- If your AI training process is taking a ridiculously long time to improve, is your Learning Rate likely too high or too low?
intermediate
- What is a 'Learning Rate Scheduler'?