Word Embeddings • Text Classification • Named Entity Recognition • Deployment Basics
Natural Language Processing (NLP) focuses on teaching computers to understand human language. Sequence models play a major role in NLP because sentences, words, and characters all follow a sequential order. This chapter explains key concepts used in modern NLP: word embeddings, text classification, Named Entity Recognition (NER)—all written in clear language for grade 10–11 students.
What Are Word Embeddings?
Words are not just characters; they carry meaning. However, computers cannot understand words directly—they understand numbers. This is where word embeddings come in.
Word embeddings convert words into dense numerical vectors that represent meaning.
Simple Explanation
Think of a dictionary where:
- Similar words have similar meanings
- In embeddings, similar words have similar vectors
Examples
- "King" and "Queen" will have vectors close to each other
- "Dog" and "Cat" will also be close
- But "Dog" and "Car" will be far apart
Why Embeddings Are Important
- Capture meaning
- Capture relationships between words
- Used in most NLP models
A. Word2Vec
Word2Vec learns embeddings by predicting either a word based on its neighbors or neighbors based on a word.
Real Example
Sentence: "The cat sat on the mat."
Word2Vec learns relationships like:
- "cat" appears near "sat"
- "sat" appears near "mat"
This helps the model learn meaning and context.
B. GloVe (Global Vectors)
GloVe learns word meanings using global statistics of the entire text. It captures how frequently words appear together across the whole dataset.
Simple Explanation
If "king" appears often with "royal", "crown", and "throne", GloVe understands "king" has a royal meaning.
Code Example: Using Word Embeddings (Keras)
from tensorflow.keras.layers import Embedding
from tensorflow.keras.models import Sequential
model = Sequential([
Embedding(input_dim=5000, output_dim=64, input_length=20)
])
model.summary()Explanation: This layer converts word IDs into 64-dimensional vectors.
What Is Text Classification?
Text classification means assigning a label to a piece of text.
Examples
- Email → spam or not spam
- Movie review → positive or negative
- News article → sports, politics, technology
- Student query → homework, exam, general info
Sequence models like RNNs, LSTMs, GRUs, and Transformers perform this task extremely well.
How Text Classification Works
- Convert words into embeddings
- Use a sequence model to understand meaning
- Pass final output to a classifier
- Predict the correct label
Simple Example
Sentence: "The movie was amazing!"
Model learns:
- "amazing" → positive meaning
- Output → "Positive Review"
Code Example: Text Classification Using LSTM
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense
model = Sequential([
Embedding(input_dim=5000, output_dim=64, input_length=100),
LSTM(64),
Dense(1, activation='sigmoid')
])
model.compile(
optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy']
)
print(model.summary())Explanation: This model can classify text as positive/negative, spam/not spam, etc.
What Is Named Entity Recognition?
Named Entity Recognition identifies important words in a sentence and labels them.
Examples of Entities
- Person names: Ali, Ayesha, John
- Locations: Pakistan, Lahore, London
- Organizations: WHO, Google, UNICEF
- Dates: 2020, Friday, January
- Numbers: 25%, 10 million
NER helps computers understand sentences more deeply.
Real Example
Sentence: "Ayesha traveled to Karachi on Monday."
NER would identify:
- Ayesha → Person
- Karachi → Location
- Monday → Date
Why NER Is Useful
- Search engines
- Chatbots
- Medical analysis
- Banking (detecting account numbers)
- News summarization
How NER Works
- Convert words to word embeddings
- Pass them into a sequence model (LSTM/GRU/Transformer)
- Predict a label for each word in the sentence
Labels Example
- B-PER → Beginning of a person name
- I-PER → Inside a person name
- O → Not an entity
Code Example: Simple NER Model Using Bidirectional LSTM
import tensorflow as tf
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, Bidirectional, LSTM, Dense, TimeDistributed
model = Sequential([
Embedding(input_dim=5000, output_dim=64, input_length=50),
Bidirectional(LSTM(64, return_sequences=True)),
TimeDistributed(Dense(10, activation='softmax'))
])
model.compile(
optimizer='adam',
loss='categorical_crossentropy',
metrics=['accuracy']
)
print(model.summary())Explanation: This model predicts an entity label for each word.
Why Sequence Models Matter in NLP
Sequence models help computers understand:
- Word meanings
- Sentence order
- Context
- Relationships between words
Without Sequence Models
- Chatbots wouldn't understand questions
- Translators wouldn't work
- Search engines couldn't find what you need
- AI assistants couldn't talk naturally
Summary Table: NLP Concepts
| Topic | Meaning | Example |
|---|---|---|
| Word Embeddings | Numerical meaning of words | "King" close to "Queen" |
| Word2Vec | Learns context-based meanings | "Cat" near "Mat" |
| GloVe | Learns global meanings | "King" with "royal" words |
| Text Classification | Label entire text | Spam detection |
| NER | Label each word | Identify names & places |