Keyword Search vs Semantic Search

Key Takeaway: Semantic search finds documents by meaning, not just keywords. By building a search engine from scratch — embedding documents, computing cosine similarity, and ranking results — you gain a deep understanding of how modern search systems work under the hood.

When you search for "how to secure a network," a keyword search might return documents that happen to contain those exact words — but miss an article titled "Protecting Computer Systems from Cyber Attacks" that answers your question perfectly.

Semantic search solves this by understanding meaning. It converts text into numerical vectors (embeddings) and compares them by semantic similarity rather than word matching.

In this tutorial, you will build a complete semantic search engine in Python — from raw documents to ranked results — with no heavy frameworks. If you have read our Private Vector Store tutorial, this article builds on those concepts with a focus on the complete search workflow.

Keyword Search vs Semantic Search

Keyword search (like traditional database queries or basic text matching) looks for exact word matches. Semantic search looks for conceptual similarity.

Consider these three documents and the query "protecting networks from attacks":

• Doc A: "Network security basics" → keyword search finds it ✓
• Doc B: "Securing computer networks" → keyword search misses it ✗
• Doc C: "Protecting systems from cyber attacks" → keyword search misses it ✗

A semantic search engine would rank all three as relevant because they discuss the same concept — even though they use different words. This is the fundamental advantage of embeddings-based search.

The Semantic Search Pipeline

The pipeline has two phases — an offline indexing phase and a real-time query phase:

Indexing Phase (Offline)

1. Load documents — read files, extract text, attach metadata.
2. Generate embeddings — convert each document's text into a numerical vector.
3. Store vectors — keep the document vectors in memory (or a vector database for larger collections).

Query Phase (Online)

1. Embed the query — convert the search query into a vector using the same model.
2. Compare — compute cosine similarity between the query vector and every stored document vector.
3. Rank — sort results by similarity score, return top-k.

Building It in Python

Our implementation uses a mock embedding function for demonstration. This lets us focus on the search mechanics without requiring a GPU or large model download. At the end, we show how to swap in a real embedding model.

Step 1: Document Preparation

from dataclasses import dataclass, field

@dataclass
class Document:
    id: str
    text: str
    metadata: dict = field(default_factory=dict)
    embedding: list = field(default_factory=list)

# Sample document collection
documents = {
    "python": Document(
        id="python",
        text="Python is a high-level programming language known for its "
             "simplicity and readability. It is widely used in web development, "
             "data science, automation, and artificial intelligence.",
        metadata={"category": "programming"},
    ),
    "machine_learning": Document(
        id="machine_learning",
        text="Machine learning enables systems to learn from experience "
             "without being explicitly programmed. It focuses on algorithms "
             "that access data, learn from it, and make predictions.",
        metadata={"category": "ai"},
    ),
    "cybersecurity": Document(
        id="cybersecurity",
        text="Cybersecurity involves protecting computer systems, networks, "
             "and data from digital attacks. It includes encryption, access "
             "control, intrusion detection, and vulnerability management.",
        metadata={"category": "security"},
    ),
    # ... more documents
}

Step 2: Embedding Generation

For this tutorial, we use a deterministic mock embedding that creates vectors based on word-frequency patterns. A real embedding model would replace this single function:

import math

def mock_embed(text: str) -> list:
    """Pseudo-embedding for demonstration purposes."""
    words = text.lower().split()
    seeds = {
        0: {"python", "programming", "language", "code", "web"},
        1: {"machine", "learning", "algorithm", "model", "neural"},
        2: {"security", "attack", "encryption", "malware", "protect"},
        3: {"database", "data", "storage", "sql", "query"},
        4: {"statistics", "probability", "regression", "analysis"},
    }
    vec = [0.0] * 5
    for i, topic_words in seeds.items():
        vec[i] = sum(1 for w in words if w in topic_words) / max(len(words), 1)
    # Normalize to unit vector
    mag = math.sqrt(sum(x * x for x in vec))
    return [x / mag for x in vec] if mag > 0 else vec

Step 3: Cosine Similarity and Ranking

def cosine_similarity(a: list, b: list) -> float:
    dot = sum(x * y for x, y in zip(a, b))
    mag_a = math.sqrt(sum(x * x for x in a))
    mag_b = math.sqrt(sum(x * x for x in b))
    return dot / (mag_a * mag_b) if mag_a and mag_b else 0.0

def search(query: str, documents: list, top_k: int = 3) -> list:
    query_vec = mock_embed(query)
    results = []
    for doc in documents:
        score = cosine_similarity(query_vec, doc.embedding)
        results.append((doc, score))
    results.sort(key=lambda x: x[1], reverse=True)
    return results[:top_k]

Step 4: Run the Search

# Build the index
for doc in documents.values():
    doc.embedding = mock_embed(doc.text)

# Search
query = "protecting networks from attacks"
results = search(query, documents, top_k=3)

for rank, (doc, score) in enumerate(results, 1):
    print(f"  #{rank} [{score:.4f}] {doc.id}")
    print(f"       {doc.text[:70]}...")

Expected output for this query:

  #1 [0.9759] cybersecurity
       Cybersecurity involves protecting computer systems, networks, and...
  #2 [0.0756] machine_learning
       Machine learning enables systems to learn from experience...
  #3 [0.0000] python
       Python is a high-level programming language known for its...

The cybersecurity document ranks highest because its vocabulary closely matches the query's meaning — even though the exact phrase "protecting networks from attacks" does not appear in the document text.

Evaluation: Did the Search Get It Right?

A search engine is only useful if it returns the right results. Three standard metrics help measure quality:

def precision_at_k(results, relevant_ids, k):
    top_k = {doc.id for doc, _ in results[:k]}
    return len(top_k & relevant_ids) / k

def recall_at_k(results, relevant_ids, k):
    top_k = {doc.id for doc, _ in results[:k]}
    return len(top_k & relevant_ids) / len(relevant_ids)

def mrr(results, relevant_ids):
    for rank, (doc, _) in enumerate(results, 1):
        if doc.id in relevant_ids:
            return 1.0 / rank
    return 0.0

P@3    = 0.33  (1 of 3 is relevant)
Recall@3 = 1.00  (found the relevant document)
MRR    = 1.00  (first result is relevant)

Why Semantic Search Can Still Return the Wrong Result

Semantic search is powerful, but it is not infallible. Common failure modes include:

• Poor embeddings: If the embedding model was not trained on your domain, it may not capture domain-specific meaning correctly.
• Ambiguous queries: "Python" could refer to the programming language or the snake. Without context, the search engine may return the wrong meaning.
• Bad chunking: If documents are too long and embedded as a whole, important details may be averaged out. If too short, context is lost.
• Insufficient metadata: Without category, date, or source information, you cannot filter results to the right context.
• Domain mismatch: A general-purpose embedding model may underperform on legal, medical, or scientific text compared to a domain-specific model.
• Query-document length mismatch: Very short queries may not capture enough semantic signal to distinguish between similar documents.

Swapping in a Real Embedding Model

To replace the mock embedding with a real model, install Sentence Transformers and change one function:

# pip install sentence-transformers

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")  # 384 dimensions

def real_embed(text: str) -> list:
    return model.encode(text).tolist()

# Now use real_embed() instead of mock_embed()
# Everything else stays exactly the same.

The model all-MiniLM-L6-v2 is small enough to run on CPU (~80 MB), produces 384-dimensional vectors, and provides strong semantic representations. For local AI setups, see our guide on Running AI Locally on CPU.

Adding Metadata Filtering

Real search engines combine semantic similarity with metadata filters. For example, you might want to search only within a specific category:

def search_with_filter(query, documents, top_k=3, category=None):
    query_vec = mock_embed(query)
    results = []
    for doc in documents:
        if category and doc.metadata.get("category") != category:
            continue  # Skip documents not in the target category
        score = cosine_similarity(query_vec, doc.embedding)
        results.append((doc, score))
    results.sort(key=lambda x: x[1], reverse=True)
    return results[:top_k]

# Search only in the "ai" category
results = search_with_filter(
    "learning algorithms", documents, top_k=3, category="ai"
)

Putting It All Together

The complete pipeline — from document loading through ranking and evaluation — is a reusable foundation. For a full working implementation with sample documents and evaluation, see the complete semantic_search.py example available alongside this article.

For understanding how embeddings work in depth, see our Embeddings Explained tutorial. For building a persistent vector store, see Building a Private Vector Store in Pure Python.

Key Takeaways

• Semantic search finds documents by meaning, not just keyword matching.
• The pipeline is: Embed → Store → Query → Compare → Rank.
• Cosine similarity is the standard metric for comparing embedding vectors.
• Evaluate search quality with Precision@K, Recall@K, and MRR.
• Mock embeddings demonstrate the mechanics; real models (like Sentence-BERT) provide actual semantic understanding.
• Metadata filtering adds precision to semantic search results.
• Semantic search can fail due to poor embeddings, ambiguous queries, or domain mismatch — always evaluate and iterate.

Related BestWordz Resources

• Embeddings Explained: How Text Becomes Meaningful Vectors
• How to Build a Private Vector Store in Pure Python
• Running AI Locally on CPU
• Model Context Protocol (MCP) Guide

Further Reading

• Sentence-BERT: Reimers & Gurevych, "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks" (2019)
• all-MiniLM-L6-v2: Hugging Face Model Card
• Sentence Transformers Library: sbert.net
• Vector Search Concepts: BestWordz Vector Store Tutorial