Overview

Naive RAG relies on semantic similarity alone, which fails on queries with logical constraints — e.g., “Find technical specs for product ‘Phoenix’ from the ‘Core Engineering’ team in the last quarter.” The central architectural challenge for production RAG is fusing conceptual semantic search with the logical precision of structured metadata filtering, answering not just “what” but also “who,” “when,” “where,” and “which.”

Key Concepts

• Two-phase system architecture — an offline indexing pipeline (ingest, clean, chunk, extract metadata, vectorize) followed by an online retrieval & generation chain (query processing, hybrid retrieval, context compilation, generation).
• Metadata as a first-class citizen — three categories: system metadata (auto-generated, e.g., filename/timestamp), user-defined metadata (explicit structured fields), and automatic metadata extraction (AI-extracted entities/keywords/topics).
• Pre-filtering vs. post-filtering — the fundamental tradeoff. Pre-filtering (filter then search) is more accurate but slower and can harm recall in graph-based indexes like HNSW by disconnecting the graph. Post-filtering (search then filter) is faster but can miss relevant documents outside the initial top-K.
• Self-querying retrieval — the most advanced filtering approach: an LLM translates a natural-language query into a structured semantic query plus metadata filters, given a schema of available fields.

Preprocessing Pipeline (Five Steps)

1. Data examination & extraction (PDFs, DOCX, HTML, etc.)
2. Data cleaning (remove noise, preserve structure as metadata)
3. Data chunking (semantically coherent pieces)
4. Metadata addition (source, date, author labels)
5. Indexing (vector embedding + metadata storage)

Chunking strategies: recursive (prioritized separator list), document-based/semantic (structure or meaning-driven boundaries), hierarchical (small child chunks for search, larger parent chunks returned for context).

Vector Database Filtering Comparison

Filtering Technique Comparison

RAG vs. Natural Language-to-SQL

NL-to-SQL's fundamental weakness is executing LLM-generated code, which creates a real injection attack surface that RAG (which never generates executable code) avoids entirely.

Implementation Patterns

• LangChain: metadata added at ingestion (split.metadata["year"] = 2024), explicit filters via search_kwargs={'filter': {...}}, or a SelfQueryRetriever that auto-generates filters from natural language.
• LlamaIndex: Document(metadata={...}) at ingestion, MetadataFilters/MetadataFilter objects passed to the query engine for pre-filtering.
• Secure multi-tenant RAG — tag every chunk with access-control metadata (group_id: 'finance'), authenticate the user/group at query time, and have the backend automatically and non-bypassably inject the corresponding metadata filter into every retrieval request — enforcing data isolation at the retrieval layer itself, not in application logic that could be bypassed.

Evaluation & Failure Mitigation

• Failure modes metadata filtering helps fix: missed top-ranked documents (pre-filtering surfaces them), and “not extracted” errors where the answer is present but buried in noisy context (filtering produces a cleaner, more focused context).
• Key metrics: Context Precision (relevance of retrieved docs), Context Recall (completeness), Answer Relevancy, Answer Faithfulness (grounded vs. hallucinated).
• The future: modular/agentic RAG (LLM orchestrators performing multi-hop retrieval and reasoning) and end-to-end retriever optimization (fine-tuning retrieval not for generic relevance, but for what produces the best final generation).

Key Takeaways

• Metadata filtering isn't a bolt-on optimization — it's the mechanism that lets a RAG system answer the logical ("who/when/where/which") half of a query that pure semantic similarity search structurally cannot address.
• The pre-filter/post-filter tradeoff is index-architecture-dependent, not universal — the same choice that's "more accurate" can actively harm recall on graph-based indexes like HNSW, which is why advanced systems use a query planner rather than a fixed strategy.
• The secure multi-tenant RAG pattern reframes metadata filtering as a security control, not just a relevance improvement — the critical detail is that filter injection happens in the backend, non-bypassably, rather than being something the retrieval call could omit.

Related Reading

• RAG Systems with Metadata-Driven Filtering
• Full Research Paper