Advanced RAG, Part 0: Foundation and folder structure

This is the first part of a series building an agentic RAG system: document ingestion with several chunking strategies, hybrid retrieval, a knowledge graph, and a LangGraph agent that can search more than once before it answers.

This part builds none of that. It sets up the skeleton those parts get built into — the folder structure, configuration, logging, and a FastAPI app that serves one endpoint.

That sounds thin, and deliberately so. The point of this part is the shape: by the end you will have a directory for every piece of the system, each one saying what it will hold, and nothing in it yet. Every later part fills exactly one of them.

What we're building, in one paragraph

A user uploads a document. We parse it, split it into chunks, embed those chunks, and store them in Postgres with pgvector. When the user asks a question, we search those chunks — by vector similarity, by keyword, and later by walking a graph of extracted entities — rerank the results, and hand them to an LLM to answer from. The "agentic" part is that the system decides how to search, and can decide its first search wasn't good enough and go again.

What a foundation is, and what it isn't

The temptation with a foundation is to build the things you know you'll need — a database client, a base repository class, an auth layer, error handling, a test harness. That is how foundations become architecture nobody asked for.

So a rule for this part:

Nothing goes in the foundation unless something in the foundation uses it.

That means no database client yet — nothing queries a database. No error taxonomy — nothing raises. No tests — there is no behaviour to test. Those all arrive in the part that needs them, and they'll make more sense there, because you'll be able to see what they're for.

What genuinely belongs here:

• The folder structure, because it's the map for everything that follows
• Configuration, because every part reads settings and they should be validated once, at startup
• Logging, for the same reason
• A running app, so there's something to curl at the end

Prerequisites

• Python 3.12+
• uv for dependency management — it's fast and it handles the virtualenv for you

curl -LsSf https://astral.sh/uv/install.sh | sh

That's all this part needs. The series as a whole needs more, and it's better to know now than to find out halfway through Part 1:

• An OpenAI API key. Embeddings use text-embedding-3-small, from Part 1 onward. Following along costs money — not much, but not nothing.
• Around 1 GB of downloads for document parsing. Part 1 parses PDFs with Docling, which pulls in PyTorch and downloads its models the first time it runs.
• Docker, running, for the local Supabase stack — also from Part 1 onward.

Step 1 — Create the project

Start from wherever you keep projects — mine live in ~/Documents/projects/RAG-apps. Create the project directory and drop into the backend:

cd ~/Documents/projects/RAG-apps
 
mkdir -p advanced-rag/backend
cd advanced-rag/backend

mkdir -p creates parent directories as needed, so that one command makes both advanced-rag/ and advanced-rag/backend/ — plain mkdir only creates the last component and would fail here. It also stays quiet if the directory already exists, which makes every mkdir in this article safe to re-run.

Everything from here on runs from backend/. The project name isn't load-bearing; call the outer folder whatever you like. Nothing in the code refers to it.

Pin the Python version so everyone gets the same interpreter:

echo "3.12" > .python-version

Step 2 — Dependencies

Create pyproject.toml in backend/, next to .python-version. You're still in backend/ from Step 1, so:

touch pyproject.toml

It's the only configuration file in the project: dependencies, build backend, linter, and type checker all live here, so there's one place to look. Open it in your editor and paste in:

[project]
name = "advanced-rag-backend"
version = "0.1.0"
description = "Advanced RAG backend"
requires-python = ">=3.12"
 
dependencies = [
    # Web framework
    "fastapi==0.128.0",
    "uvicorn[standard]==0.40.0",
    # Configuration and validation
    "pydantic==2.12.5",
    "pydantic-settings==2.12.0",
    # Logging
    "structlog==25.5.0",
]
 
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
 
[tool.hatch.build.targets.wheel]
packages = ["app"]
 
[tool.ruff]
line-length = 100
target-version = "py312"
 
[tool.ruff.lint]
select = ["E", "F", "I", "N", "W", "B", "UP"]
ignore = ["E501"]
 
[tool.mypy]
python_version = "3.12"
strict = true
warn_return_any = true
warn_unused_configs = true
plugins = ["pydantic.mypy"]
 
[dependency-groups]
dev = [
    "mypy==1.19.1",
    "ruff==0.14.14",
]

Then:

uv sync

That creates .venv/ and installs everything. You never activate it by hand — uv run does that for you.

Five runtime dependencies

Not five categories — five packages. There is no LangChain here, no OpenAI client, no vector library, no database driver, because nothing in this part talks to a model, a vector store, or a database.

This is a rule worth keeping for the whole series: a dependency arrives with the code that imports it. When someone asks "why does this project depend on X?", the answer should be a file you can point at. Front-loading dependencies makes that question unanswerable within a month.

It's a rule that's easy to state and easy to quietly break, so here it is being applied against my own instincts. This part configures Supabase settings — a URL and a service key, validated at startup in Step 4 — but does not install the supabase package. The settings are read by the config object; the client isn't imported by anything until Part 1 opens a connection. A setting with no reader is a typo waiting to happen; a package with no importer is dead weight in your lockfile. They're different problems, and only one of them is solved by adding a dependency today.

The toolchain, briefly

Three of those config blocks are for tools you haven't met yet, so before they show up in Step 7 as commands you're told to run:

• ruff is the linter and the formatter, replacing what used to be flake8 plus isort plus black. select picks which rule families are on: pycodestyle (E, W), pyflakes (F), isort (I), pep8-naming (N), bugbear (B), and pyupgrade (UP). ignore = ["E501"] switches off pycodestyle's own line-length rule because line-length = 100 already governs that, and having both means being told about the same line twice.
• mypy is the type checker, and strict = true is doing real work: it rejects untyped function definitions, implicit Any returns, and unchecked decorators. It's much easier to start strict than to retrofit it. You'll see the one place it needs help in Step 5.

Both are dev-only — they're in [dependency-groups], not dependencies, so they never ship.

Step 3 — The folder structure

This is the part that matters. Create it:

mkdir -p app/{api/v1,agents/nodes,core,db,ingestion/chunkers,llm,retrieval/rerankers,schemas,utils}

The braces are shell expansion: that one line becomes nine paths — app/api/v1, app/agents/nodes, app/core, and so on. -p then fills in the intermediate app/api/ and app/agents/ that nothing asked for explicitly, so thirteen directories end up under app/.

Which gives you:

backend/
├── pyproject.toml
├── .python-version
└── app/
    ├── core/                    configuration, logging
    ├── api/
    │   └── v1/                  versioned HTTP routes
    ├── db/                      database client and query helpers
    ├── ingestion/
    │   └── chunkers/            chunking strategies
    ├── retrieval/
    │   └── rerankers/           reranking implementations
    ├── agents/
    │   └── nodes/               individual graph nodes
    ├── llm/                     the LLM call wrapper
    ├── schemas/                 Pydantic request/response models
    └── utils/                   shared helpers

Each folder is a package, and each one's __init__.py says what it will hold — so the structure documents itself and an empty folder isn't a mystery:

The column says first filled in, not the part that's about it. Those differ in two rows, and the difference is the honest one: you need a splitter before you can compare splitters, and you need retrieval before you can measure whether a chunking strategy retrieves well.

Create the packages with a line each:

printf '"""Advanced RAG — backend application package."""\n' > app/__init__.py
printf '"""Process-wide concerns: configuration and logging."""\n' > app/core/__init__.py
printf '"""HTTP layer: routers, request dependencies, and error handling."""\n' > app/api/__init__.py
printf '"""Version 1 routes. Each router is mounted by the feature that owns it."""\n' > app/api/v1/__init__.py
printf '"""Database access: the Supabase client and user-scoped query helpers."""\n' > app/db/__init__.py
printf '"""Document ingestion: parsing, chunking, embedding, and storage."""\n' > app/ingestion/__init__.py
printf '"""Chunking strategies — splitters and augmenters."""\n' > app/ingestion/chunkers/__init__.py
printf '"""Retrieval: vector search, full-text search, hybrid fusion, and reranking."""\n' > app/retrieval/__init__.py
printf '"""Reranker implementations, behind a common interface."""\n' > app/retrieval/rerankers/__init__.py
printf '"""The LangGraph agent: graph construction, state, and control flow."""\n' > app/agents/__init__.py
printf '"""Individual graph nodes — routing, retrieval, generation, grading."""\n' > app/agents/nodes/__init__.py
printf '"""The single wrapper every LLM call goes through: timeout, retry, tokens, cost."""\n' > app/llm/__init__.py
printf '"""Pydantic models for request and response bodies."""\n' > app/schemas/__init__.py
printf '"""Shared helpers with no better home."""\n' > app/utils/__init__.py

Why this shape

Grouped by responsibility, not by type. There is no models/, services/, controllers/ split. Chunking code lives in ingestion/chunkers/ because chunking is part of ingestion. When you're changing how chunking works, everything you need is in one place instead of spread across three folders that each hold a third of the feature.

Each folder maps to a part of the series. That's not a coincidence — the structure and the build order were designed together. retrieval/rerankers/ being its own folder is a bet that there will be more than one reranker worth comparing.

llm/ is separate from agents/. Tempting to nest it, but every part that calls a model needs it, not just the agent — ingestion calls a model to enrich chunks, the graph build calls one to extract entities. Making it a peer keeps the agent from becoming a dependency of things that aren't agentic.

Step 4 — Configuration

Settings come from environment variables and get validated once, at import. A misconfigured deployment should fail to start, not fail on the first request that touches the bad value.

From here on you're writing code rather than running commands, so open the project in your editor if it isn't already. Every file in Steps 4 to 6 is new: create it at the path shown, relative to backend/, and paste in the contents.

Point the editor at the project's interpreter first. uv run finds .venv/ on its own, but your editor doesn't, and until it does every import of pydantic gets a red squiggle even though the package is installed. In VS Code, run Python: Select Interpreter from the command palette. If you opened backend/ itself, .venv is usually listed — pick it. If you opened advanced-rag/, it won't be, because .venv is a level down: choose Enter interpreter path… and give it the full path to backend/.venv/bin/python. The status bar should then show Python 3.12, not whatever your system Python is.

Create app/core/config.py:

"""Application configuration."""
 
from functools import lru_cache
from typing import Annotated, Literal
 
from pydantic import Field, model_validator
from pydantic_settings import BaseSettings, NoDecode, SettingsConfigDict
 
 
class Settings(BaseSettings):
    """Environment-driven application settings."""
 
    model_config = SettingsConfigDict(
        env_file=".env",
        env_file_encoding="utf-8",
        case_sensitive=False,
        extra="ignore",
    )
 
    # -- Application ------------------------------------------------------
    app_name: str = "Advanced RAG"
    app_version: str = "0.1.0"
    environment: Literal["development", "staging", "production"] = "development"
    debug: bool = Field(default=False, description="Enable debug mode")
 
    # -- Server -----------------------------------------------------------
    host: str = Field(default="0.0.0.0", description="Bind address")
    port: int = Field(default=8000, description="Bind port")
    reload: bool = Field(default=True, description="Auto-reload")
 
    # -- CORS -------------------------------------------------------------
    cors_origins: Annotated[list[str], NoDecode] = Field(
        default=["http://localhost:3000"],
        description="Allowed CORS origins (comma-separated or JSON array)",
    )
    cors_allow_credentials: bool = True
    cors_allow_methods: list[str] = ["GET", "POST", "PATCH", "DELETE", "OPTIONS"]
    cors_allow_headers: list[str] = ["Content-Type", "Authorization"]
 
    # -- Supabase ---------------------------------------------------------
    supabase_url: str = Field(..., description="Supabase project URL")
    supabase_service_key: str = Field(
        ...,
        description="Supabase service-role key",
        repr=False,
        alias="SUPABASE_SERVICE_ROLE_KEY",
    )
 
    # -- Auth -------------------------------------------------------------
    auth_enabled: bool = Field(default=True, description="Require authentication")
 
    # -- Logging ----------------------------------------------------------
    log_level: Literal["DEBUG", "INFO", "WARNING", "ERROR"] = "INFO"
    log_json: bool = Field(default=False, description="Emit JSON logs")
 
    @classmethod
    def _split_origins(cls, value: str | list[str]) -> list[str]:
        if isinstance(value, str):
            return [origin.strip() for origin in value.split(",") if origin.strip()]
        return value
 
    @model_validator(mode="before")
    @classmethod
    def parse_cors_origins(cls, values: dict[str, object]) -> dict[str, object]:
        raw = values.get("cors_origins") or values.get("CORS_ORIGINS")
        if raw is not None:
            values["cors_origins"] = cls._split_origins(raw)  # type: ignore[arg-type]
        return values
 
    @model_validator(mode="after")
    def enforce_production_safety(self) -> "Settings":
        """Refuse to boot production with development toggles left on."""
        if self.environment != "production":
            return self
 
        if self.debug:
            raise ValueError("DEBUG must be disabled in production")
        if self.reload:
            raise ValueError("Auto-reload must be disabled in production")
        if not self.auth_enabled:
            raise ValueError(
                "AUTH_ENABLED must be true in production — disabling it collapses every "
                "tenant into a single shared identity"
            )
        return self
 
 
@lru_cache(maxsize=1)
def get_settings() -> Settings:
    """Return the process-wide settings, constructed once."""
    return Settings()
 
 
settings = get_settings()

Two things worth pausing on.

enforce_production_safety is not boilerplate. The version of this project I'm rebuilding had this exact validator, and it guarded debug and reload — but not auth_enabled. That meant one environment variable could start a production deployment where every request resolved to the same fake development user, and every user's documents belonged to everyone. It started cleanly. It logged nothing unusual. The guard is three lines and it's the difference between a typo and an incident.

supabase_url and supabase_service_key have no defaults. The ... means required. If they're missing the app won't start, which is what you want — a database URL that silently defaults to something is a database you'll connect to by accident.

Now create .env.example. This one goes at the backend root, next to pyproject.toml — not under app/ — because env_file=".env" resolves against the directory you run from, and every command in this part runs from backend/. A .env in the wrong place isn't an error; it's silently ignored, and every setting quietly falls back to its default. Document only what the code reads:

# ─── Application ─────────────────────────────────────────────────────────────
ENVIRONMENT=development
DEBUG=false
LOG_LEVEL=INFO
LOG_JSON=false
 
# ─── Server ──────────────────────────────────────────────────────────────────
HOST=0.0.0.0
PORT=8000
RELOAD=true
 
# ─── CORS ────────────────────────────────────────────────────────────────────
CORS_ORIGINS=http://localhost:3000
 
# ─── Supabase ────────────────────────────────────────────────────────────────
SUPABASE_URL=http://127.0.0.1:54321
SUPABASE_SERVICE_ROLE_KEY=
 
# ─── Authentication ──────────────────────────────────────────────────────────
AUTH_ENABLED=false

cp .env.example .env

For now the Supabase values can stay as they are — nothing connects to a database yet. Part 1 sets up the local stack and fills them in.

Step 5 — Logging

One call at startup wires structlog and the standard library together, so libraries that use logging end up in the same stream and format as your own calls.

Create app/core/logging.py:

"""Structured logging setup."""
 
import logging
import sys
 
import structlog
 
from app.core.config import settings
 
 
def configure_logging() -> None:
    """Configure structlog and the stdlib root logger. Call once at startup."""
    level = getattr(logging, settings.log_level)
 
    logging.basicConfig(
        format="%(message)s",
        stream=sys.stdout,
        level=level,
        force=True,
    )
 
    shared_processors: list[structlog.typing.Processor] = [
        structlog.contextvars.merge_contextvars,
        structlog.stdlib.add_log_level,
        structlog.stdlib.add_logger_name,
        structlog.processors.TimeStamper(fmt="iso", utc=True),
        structlog.processors.StackInfoRenderer(),
        structlog.processors.format_exc_info,
    ]
 
    renderer: structlog.typing.Processor = (
        structlog.processors.JSONRenderer()
        if settings.log_json
        else structlog.dev.ConsoleRenderer()
    )
 
    structlog.configure(
        processors=[*shared_processors, renderer],
        wrapper_class=structlog.make_filtering_bound_logger(level),
        logger_factory=structlog.stdlib.LoggerFactory(),
        cache_logger_on_first_use=True,
    )
 
 
def get_logger(name: str) -> structlog.stdlib.BoundLogger:
    """Return a bound logger for a module. Use `get_logger(__name__)`."""
    return structlog.get_logger(name)  # type: ignore[no-any-return]

That type: ignore is the one place in this part where the type checker needs telling something it can't work out: structlog.get_logger is annotated as returning Any, and strict = true from Step 2 refuses to let an Any escape through a typed return. One suppression, at the boundary, with the reason attached — which is the shape a type: ignore should have.

Structured logging means logging key-value pairs rather than sentences:

logger.info("document_ingested", document_id=doc.id, chunks=42, duration_ms=1832)

instead of logger.info(f"Ingested {doc.id} into 42 chunks in 1832ms"). Readable in development via ConsoleRenderer, and queryable in production because it's JSON. Once there's a pipeline emitting hundreds of these per document, "show me every ingestion that produced zero chunks" is a filter rather than a regex.

Step 6 — The application

Create app/main.py:

"""FastAPI application entrypoint."""
 
from collections.abc import AsyncGenerator
from contextlib import asynccontextmanager
 
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
 
from app.core.config import settings
from app.core.logging import configure_logging, get_logger
 
logger = get_logger(__name__)
 
 
class Health(BaseModel):
    """Liveness payload."""
 
    status: str
    version: str
    environment: str
 
 
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None]:
    configure_logging()
    logger.info(
        "startup",
        app=settings.app_name,
        version=settings.app_version,
        environment=settings.environment,
    )
    yield
    logger.info("shutdown")
 
 
def create_app() -> FastAPI:
    """Build the application."""
    app = FastAPI(
        title=settings.app_name,
        version=settings.app_version,
        debug=settings.debug,
        lifespan=lifespan,
    )
 
    app.add_middleware(
        CORSMiddleware,
        allow_origins=settings.cors_origins,
        allow_credentials=settings.cors_allow_credentials,
        allow_methods=settings.cors_allow_methods,
        allow_headers=settings.cors_allow_headers,
    )
 
    @app.get("/health", response_model=Health, tags=["meta"])
    async def health() -> Health:
        return Health(
            status="ok",
            version=settings.app_version,
            environment=settings.environment,
        )
 
    return app
 
 
app = create_app()

create_app() is a factory rather than a module-level app because tests will want to build their own instance with different settings. Doing it now costs one line; retrofitting it once twenty modules import app directly costs an afternoon.

No routers are mounted. api/v1/ is empty, and the app mounts nothing from it. Each part adds its own router, so the mounted surface is always the surface that works.

Step 7 — Run it, then check it

Start the server:

uv run uvicorn app.main:app --reload

It keeps running and holds that terminal, so leave it alone and open a second terminal, also in backend/, for everything else in this step. From the second one:

curl localhost:8000/health

{"status":"ok","version":"0.1.0","environment":"development"}

Interactive docs at http://localhost:8000/docs.

Now look back at the first terminal, the one running the server. Among uvicorn's own lines you should see this one — the logger.info("startup", …) call in main.py, formatted by the logging config from Step 5. Your timestamp will differ:

2026-08-17T22:08:47.952910Z [info] startup [app.main] app='Advanced RAG' environment=development version=0.1.0

Check the production guard

Step 4 put a validator on Settings, enforce_production_safety, that refuses to let the app start in production with development settings left on. That's the guard. The point of running it is to see it refuse — a safety check you've never seen fire is one you're taking on trust.

Stop the server with Ctrl+C, then start it again pretending to be production, with authentication switched off:

ENVIRONMENT=production RELOAD=false AUTH_ENABLED=false uv run uvicorn app.main:app

Note RELOAD=false. The validator checks debug, then reload, then auth_enabled, and .env sets RELOAD=true — without overriding it you'd trip the reload check first and never reach the interesting one.

It refuses to start:

pydantic_core._pydantic_core.ValidationError: 1 validation error for Settings
  Value error, AUTH_ENABLED must be true in production — disabling it collapses every
  tenant into a single shared identity [type=value_error, ...]

The ValueError you raised comes back wrapped in a pydantic ValidationError — that's pydantic collecting every validation failure before reporting, and it's why the message is nested rather than at the top of the traceback.

Check the code

A server that starts and code that's clean are different claims. These three checks are the ones the tools in Step 2 were configured for:

uv run ruff check app/
uv run ruff format --check app/
uv run mypy app/

The first two will probably fail on a first run, unless your editor adds a final newline when it saves. Ruff reports W292 No newline at end of file on config.py, logging.py, and main.py, and ruff format --check says it would reformat the same three. That isn't a mistake in the code: copying out of a code block tends to drop the final newline. The fourteen __init__.py files are unaffected because printf supplies it. One command fixes both:

uv run ruff check --fix app/

Then run the three checks again. This time all three pass — All checks passed!, 17 files already formatted, and Success: no issues found in 17 source files. Seventeen is fourteen __init__.py files plus main.py, config.py, and logging.py.

Step 8 — Put it under version control

Everything so far lives in one folder on your machine. Before the next part starts adding to it, commit it and push it to GitHub, so each part lands as a change you can look back at.

The repository goes at the project root, advanced-rag/, not in backend/ — later parts put other things beside backend/, and they belong in the same history. So step up a level:

cd ..

First, tell git what to leave out. Create .gitignore here, in advanced-rag/:

# Python
__pycache__/
.venv/
.mypy_cache/
.ruff_cache/
 
# Local settings — may hold secrets from Part 1 onward
.env
 
# macOS
.DS_Store

This has to exist before the first commit. .venv/ is thousands of files that uv sync recreates in seconds, and .env is where your API key goes in Part 1. Once a file is committed, adding it to .gitignore later doesn't remove it from history.

Then create the repository and check what it's about to include:

git init -b main
git add .
git status

git status should list 22 files: .gitignore, everything under backend/ you wrote, and uv.lock — which uv sync generated, and which is committed so everyone installs exactly the same versions. .env.example yes, .env no, and nothing from .venv/. If .env or .venv/ appears, .gitignore is in the wrong place or misspelled; fix it before going on.

git commit -m "Part 0: foundation and folder structure"

Now put it on GitHub. With the GitHub CLI — gh auth login once if you haven't used it before:

gh repo create advanced-rag --private --source=. --push

That creates the repository on GitHub, adds it as the origin remote, and pushes main. Drop --private if you want it public.

If you'd rather stay in your editor, VS Code's Source Control panel does the same thing: Initialize Repository, commit, then Publish Branch. It reads the same .gitignore, so create that first either way.

Where we are

advanced-rag/               a git repository, pushed to GitHub
├── .gitignore
└── backend/
    ├── .env.example        what the app reads
    ├── .env                your local copy, not committed
    ├── .python-version     3.12
    ├── pyproject.toml      5 runtime dependencies, plus ruff and mypy
    └── app/
        ├── main.py         a FastAPI app with one endpoint
        ├── core/           config + logging
        └── (twelve empty packages, each labelled)

Roughly 250 lines. One endpoint, no database, no tests, no Docker.

The empty folders are the useful output. Every part that follows fills exactly one of them, and when a part is done you can point at the folder and say what's in it and why.

What's next

Part 1 — Ingestion. The local Postgres stack with pgvector, the schema, and the pipeline that turns an uploaded PDF into embedded chunks. That's where db/, ingestion/, schemas/, and the first router in api/v1/ get filled in — and where the first tests appear, because it's the first part with behaviour worth testing. It's also where supabase finally becomes a dependency, with a file you can point at.