AI Terminology
/Advanced
Activation Functions
Definition
Mathematical equations attached to each neuron in a network that determine whether the neuron should 'fire' (activate) or remain dormant, introducing non-linearity to the model.
Explain Like I'm New
A neuron does a bunch of math and gets a number, like 452. The Activation Function acts as a gatekeeper. It looks at the number and says: 'Anything greater than 0 becomes a 1. Anything less than 0 becomes a 0.' It decides if the signal is strong enough to pass to the next layer.
Real World Example
The ReLU (Rectified Linear Unit) activation function is used in almost all modern AI. Its math is incredibly simple: `if (x < 0) return 0; else return x;`. This tiny rule allows the network to learn incredibly complex shapes and patterns.
Common Use Cases
- •Adding non-linearity
- •Network design
Interview Questions
basic
- What happens to a Neural Network if you do not use any Activation Functions at all?
intermediate
- What does the 'Softmax' activation function do at the very end of an output layer?