AI Terminology Course
AI Terminology
/
Beginner

Stemming

Definition

A crude heuristic process in NLP that chops off the ends of words to reduce them to their base or root form.

Explain Like I'm New

A digital meat cleaver. It looks at the words 'Running', 'Runner', and 'Runs', and blindly chops off the ends, reducing all of them to 'Run'. This helps the computer realize they all mean the same thing.

Real World Example

Old search engines used stemming heavily. If you searched for 'fishing', the search engine chopped it to 'fish' and showed you results for 'fished', 'fisher', and 'fishes'.

Common Use Cases

  • •Search indexing
  • •Spam filtering

Interview Questions

basic

  • What is the primary purpose of Stemming in text processing?

intermediate

  • What is the major downside of Stemming compared to Lemmatization?

Flash Cards

Question

Primary purpose?

Click to reveal answer
Answer

To reduce the size of the vocabulary the computer has to memorize by treating all variations of a word as the exact same root word.

Question

Downside of stemming?

Click to reveal answer
Answer

It is dumb. It literally just chops off letters using hardcoded rules (like 'delete -ing'). If you stem the word 'Caring', it becomes 'Car'. This completely destroys the meaning of the sentence.