AI Terminology Course
AI Terminology
/
Intermediate

Lemmatization

Definition

A sophisticated NLP process that uses vocabulary and morphological analysis of words to accurately reduce them to their proper dictionary base form (the lemma).

Explain Like I'm New

A digital English teacher. Unlike Stemming (which blindly chops off letters), Lemmatization actually understands grammar. It knows that the root word of 'Better' is 'Good', and the root word of 'Mice' is 'Mouse'.

Real World Example

Modern chatbots use lemmatization to understand user intent. If the user says 'I am buying a car' or 'I bought a car', the AI lemmatizes both to 'buy car' to trigger the sales algorithm.

Common Use Cases

  • •Chatbot intent recognition
  • •Text analysis

Interview Questions

basic

  • Which is generally more accurate: Stemming or Lemmatization?

intermediate

  • Why does Lemmatization require significantly more computing power than Stemming?

Flash Cards

Question

Which is more accurate?

Click to reveal answer
Answer

Lemmatization. It preserves the actual meaning of the words.

Question

Why more computing power?

Click to reveal answer
Answer

Stemming just uses simple Regex rules to delete letters. Lemmatization requires the computer to load a massive dictionary into memory and perform complex linguistic analysis to figure out the context of the word before converting it.