Inference

Definition

The process of using a fully trained machine learning model to make predictions, classifications, or generate output on new, unseen data.

Explain Like I'm New

If 'Training' is going to medical school and studying textbooks, 'Inference' is graduating and actually diagnosing a patient in the hospital. The learning is over; now the AI is just doing its job.

Real World Example

When you unlock your iPhone with FaceID. Apple already 'trained' the model at the factory. When you look at the camera, the phone runs 'Inference' in milliseconds to predict if the face matches you.

Common Use Cases

•Production AI
•API endpoints

Interview Questions

basic

Does the AI model continue to learn and update its knowledge during Inference?

intermediate

Why are tech companies so focused on 'Inference Optimization'?

Flash Cards

Question

Learn during inference?

Click to reveal answer

Answer

No. A standard model's 'brain' is frozen after training. During inference, it only applies what it already knows. (To teach it new things, you must re-train or fine-tune it).

Question

Why optimize inference?

Click to reveal answer

Answer

Because a model might be trained once, but it will perform inference millions of times a day for users. If inference takes 10 seconds, users will quit the app. Optimizing inference makes the AI run faster and uses less server electricity, saving millions of dollars.