What Is RAG?

Retrieval-Augmented Generation has two primary stages: retrieval and generation.

  • Retrieval - find relevant passages from blog posts, documentation, FAQs, or other approved content.
  • Generation - provide those passages to an LLM so it can produce a natural-language answer grounded in the retrieved information.

RAG Architecture: Indexing and Querying

A practical RAG implementation has two workflows: an offline indexing pipeline and an online query pipeline.

Architecture

INDEXING
Blog CMS / Markdown / HTML
        ↓
Extract text + metadata
        ↓
Split into focused chunks
        ↓
Generate embeddings
        ↓
Store vectors + metadata
        ↓
Vector database

QUERYING
User question
        ↓
Generate query embedding
        ↓
Similarity search
        ↓
Select top relevant chunks
        ↓
Build grounded prompt
        ↓
LLM generates answer + sources

 

The LLM does not need to receive the entire blog for every question. Only the most relevant chunks are retrieved and supplied as context.

Technology Choices

A minimum implementation requires access to website content, an embedding model or API, a vector store, an LLM, and a backend service that coordinates ingestion, retrieval, and generation.

  • Python or Node.js for backend services.
  • A managed or self-hosted vector database.
  • An embedding model appropriate for the application's language and content.
  • An LLM for grounded answer generation.
  • A content source such as a CMS API, sitemap, RSS feed, database, Markdown files, or another controlled source.

PostgreSQL with pgvector is a practical option when PostgreSQL is already part of the application's infrastructure because it keeps vector storage close to existing application data.

Step 1: Prepare and Ingest Blog Content

Retrieval quality depends heavily on source quality. Normalize each article into a consistent structure and preserve metadata that will be useful for retrieval and citations.

  • Unique document ID or slug
  • Title
  • Canonical URL
  • Clean article content
  • Publication or update date
  • Tags or categories
  • Author, when relevant

Python

from dataclasses import dataclass
from datetime import datetime

@dataclass
class BlogDocument:
    id: str
    title: str
    url: str
    content: str
    published_at: datetime
    tags: list[str]
    author: str | None = None

 

If content is HTML, remove navigation, scripts, styles, footers, and other non-content elements before indexing. For Markdown, preserve headings because they can improve chunk boundaries.

Step 2: Chunk Documents for Better Retrieval

Embedding an entire long article as one vector makes retrieval less precise. Split articles into focused passages so a query can retrieve the section that actually answers the question.

  • Fixed-size chunks for simple, uniform content.
  • Heading-aware or recursive chunks for technical documentation.
  • Semantic chunks for long-form content with varied sections.
  • Sentence-based chunks for short articles.

A practical starting point for technical blog content is approximately 500–800 tokens per chunk with 10–20% overlap. Store metadata such as document ID, title, URL, chunk index, and section heading.

Python

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=100,
    separators=["\n## ", "\n### ", "\n\n", "\n", " "],
)

chunks = splitter.split_text(document.content)

 

Overlap helps preserve meaning at chunk boundaries. Without it, a sentence or explanation that spans two chunks can lose important context.

Step 3: Generate Embeddings and Store Them

Embeddings convert text into numerical vectors that represent semantic meaning. Similar passages tend to be close together in vector space, enabling semantic retrieval.

Python

from openai import OpenAI

client = OpenAI()

def embed_texts(texts: list[str]) -> list[list[float]]:
    response = client.embeddings.create(
        input=texts,
        model="text-embedding-3-small",
    )
    return [item.embedding for item in response.data]

 

The vector dimension depends on the selected embedding model, so the database schema must match the model output.

SQL

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE blog_chunks (
    id SERIAL PRIMARY KEY,
    doc_id TEXT NOT NULL,
    chunk_index INT NOT NULL,
    title TEXT NOT NULL,
    url TEXT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(1536) NOT NULL,
    created_at TIMESTAMPTZ DEFAULT NOW()
);

 

The example dimension above must be changed if a different embedding model is selected.

Step 4: Build the Retrieval Layer

At query time, convert the user's question into an embedding and compare it with stored chunk vectors. The highest-scoring chunks become the context for generation.

Python

def retrieve_relevant_chunks(query: str, top_k: int = 5):
    query_embedding = embed_texts([query])[0]

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=top_k,
        include=["documents", "metadatas", "distances"],
    )

    return [
        {"text": text, "metadata": metadata, "score": 1 - distance}
        for text, metadata, distance in zip(
            results["documents"][0],
            results["metadatas"][0],
            results["distances"][0],
        )
    ]

 

  • Hybrid search can combine vector similarity with keyword search.
  • Metadata filters can restrict results by category, tag, date, or content type.
  • Re-ranking can improve the ordering of retrieved candidates.
  • Query expansion can help when the original question is vague.
  • A similarity threshold can reject weak matches.

If no chunk reaches the required confidence level, return a controlled fallback rather than asking the LLM to guess.

Step 5: Generate Grounded Answers

The generation step should explicitly instruct the LLM to answer from retrieved website content when accuracy matters.

Python

SYSTEM_PROMPT = """
You are a helpful assistant for a technical website.
Answer questions using only the provided context.
If the context is insufficient, say so clearly.
Always cite the source URL provided with the context.
Do not invent technical details or policies.
"""

 

A lower temperature can be useful for factual question answering. Source URLs should be returned with the answer so visitors can verify the original article.

Step 6: Expose RAG Through an API

Keep model credentials and retrieval logic on the server. A simple API can accept a question and return both the generated answer and source articles.

Python

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class QueryRequest(BaseModel):
    question: str
    top_k: int = 5

@app.post("/api/ask")
async def ask(request: QueryRequest):
    return generate_answer(request.question, request.top_k)

 

A frontend widget can call this endpoint and display the answer together with links to the source articles.

Security Considerations

  • Rate-limit the question endpoint.
  • Validate and sanitize user input.
  • Keep API keys on the server; never expose them in browser code.
  • Protect unpublished or restricted content with authorization and metadata filters.
  • Use HTTPS in production.
  • Treat prompt injection as a security concern and keep system instructions separate from user content.
  • Log queries carefully and follow applicable privacy requirements.

Production Best Practices

Evaluate Before Launch

Create a representative test set of real user questions with expected source articles. Measure retrieval quality, answer faithfulness, citation accuracy, and end-to-end latency before launch.

Keep Content Fresh

Re-index when an article is published, updated, deleted, or its URL changes. Incremental indexing is preferable to rebuilding the entire vector store for every small change.

Observe the Pipeline

  • Retrieved chunk IDs and similarity scores
  • Model token usage
  • Fallback rate
  • User feedback
  • Latency across retrieval and generation

Cache Carefully

Caching repeated query embeddings or responses can reduce cost and latency. Invalidate cached results when the underlying content changes.

Common Pitfalls and How to Avoid Them

Pitfall

Recommended Approach

Hallucinated answers

Use grounded prompts, similarity thresholds, and explicit fallback behavior.

Stale content

Trigger incremental re-indexing when content changes.

Poor chunk boundaries

Split by headings where possible and use controlled overlap.

Missing source links

Store canonical URLs in chunk metadata.

Prompt injection

Validate input and separate system instructions from user content.

High API cost

Pre-compute embeddings, limit context, and cache repeated queries.

Cost and Performance Considerations

The main cost drivers are embedding generation, LLM generation, and vector storage. Generate embeddings when content is indexed rather than for every query. At query time, retrieve a small set of high-quality chunks to control context size and generation cost.

  • Pre-compute embeddings during indexing.
  • Tune top-k using evaluation data.
  • Keep vector storage geographically close to the application when practical.
  • Consider streaming model responses for perceived responsiveness.

Implementation Checklist

  • Normalize blog content and metadata.
  • Choose a chunking strategy appropriate for the content.
  • Generate embeddings and store them with metadata.
  • Implement similarity retrieval.
  • Add thresholds and fallback behavior.
  • Build a grounded generation prompt.
  • Return source URLs with answers.
  • Protect the backend API and model credentials.
  • Create an evaluation dataset before launch.
  • Automate re-indexing when content changes.
  • Monitor retrieval quality, latency, cost, and user feedback.

Conclusion

RAG is a practical way to turn a static website blog into an interactive knowledge base. The core pipeline is straightforward: ingest content, split it into retrieval-friendly chunks, generate embeddings, store those vectors, retrieve relevant context, and generate a grounded answer with citations.

The most important part is the retrieval pipeline. Clean source content, sensible chunking, useful metadata, reliable similarity search, confidence thresholds, and evaluation determine whether the resulting assistant is useful.

Start with a small collection of articles, measure retrieval and answer quality, and expand incrementally. Because content changes can be handled through re-indexing, the knowledge base can grow without retraining the language model.

FAQ

What is the difference between RAG and fine-tuning?

Fine-tuning changes model weights using training data. RAG retrieves current content at query time and supplies it as context. For frequently changing website content, RAG is generally easier to maintain because updates can be handled through re-indexing.

Do I need a GPU to build RAG?

Not necessarily. Managed embedding and generation APIs can handle the model workload. A GPU becomes more relevant when running models locally at meaningful scale.

How do I prevent the assistant from making up answers?

Use retrieval thresholds, a strict grounded-answer prompt, controlled generation settings, explicit no-result fallbacks, and source citations. The assistant should not fill missing website information with unsupported assumptions

Recent Posts