---
title: "Build a RAG Chatbot on Netlify with AI Gateway and Netlify Database"
description: "Build a RAG chatbot with Netlify Database, pgvector, and AI Gateway for vector storage, embeddings, and chat without external API keys."
source: "https://www.netlify.com/knowledge-base/build-a-rag-chatbot-on-netlify/"
last_updated: "2026-07-24T14:06:30.000Z"
---
Retrieval-augmented generation (RAG) answers a question by first fetching the most relevant pieces of your own content, then asking a language model to answer using that content as context. It keeps answers grounded in your data instead of the model’s training set.

You can build the whole pipeline on Netlify without a separate vector database or a single provider API key. [Netlify Database](https://docs.netlify.com/build/data-and-storage/netlify-database/) stores the vectors with the pgvector extension, and the [AI Gateway](https://docs.netlify.com/build/ai-gateway/overview/) generates embeddings and chat completions through the standard OpenAI SDK, injecting the credentials for you. This guide walks the pipeline end to end: ingest your content, embed it, store the vectors, retrieve the closest matches for a question, and answer with a model.

## Prerequisites

-   A Netlify project on a credit-based plan (Free, Personal, or Pro), since Netlify Database and the AI Gateway both run on credits.
-   The [Netlify CLI](https://docs.netlify.com/cli/get-started/) installed and your project linked.
-   Node 18 or later locally.
-   The `openai`, `@netlify/database`, and `postgres` packages.

## How the pipeline fits together

Four stages, two of them at request time:

1.  **Ingest (once, or when content changes):** read your source documents, split them into chunks, embed each chunk, and store the vector.
2.  **Embed the question (per request):** turn the user’s question into a vector with the same model.
3.  **Retrieve (per request):** find the chunks whose vectors sit closest to the question vector.
4.  **Answer (per request):** pass those chunks to a chat model as context and return its reply.

## Step 1: Provision the database with pgvector

Initialize Netlify Database in your project:

```
npx netlify db init
```

That installs `@netlify/database` and scaffolds a migrations directory. Netlify provisions the Postgres instance on your next deploy and applies migrations automatically.

Enable pgvector and create the table that holds your chunks and their vectors. Add a migration under `netlify/database/migrations/`:

```
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE embeddings (  id        bigserial PRIMARY KEY,  content   text NOT NULL,  embedding vector(1536));
CREATE INDEX idx_embeddings_vector  ON embeddings USING ivfflat (embedding vector_cosine_ops)  WITH (lists = 100);
```

The `vector(1536)` type matches the 1536 dimensions that OpenAI’s `text-embedding-3-small` model returns. The IVFFlat index makes cosine-distance search fast once you hold more than a few thousand rows.

## Step 2: Embed and store your content

The AI Gateway injects `OPENAI_API_KEY` and `OPENAI_BASE_URL` into every Netlify compute context, so the OpenAI SDK works with no key configuration of your own. Write an ingestion script that reads your documents, embeds each chunk, and inserts the vector:

```
import OpenAI from 'openai';import { getConnectionString } from '@netlify/database';import postgres from 'postgres';
const openai = new OpenAI(); // credentials come from the AI Gatewayconst sql = postgres(getConnectionString()); // connection string from Netlify Database
for (const chunk of chunks) {  const { data } = await openai.embeddings.create({    model: 'text-embedding-3-small',    input: chunk,  });  const vec = `[${data[0].embedding.join(',')}]`;  await sql`INSERT INTO embeddings (content, embedding) VALUES (${chunk}, ${vec}::vector)`;}
```

Run it against your provisioned database with the CLI so the environment matches production:

```
netlify dev:exec node ingest.js
```

Chunk your content to a few hundred tokens per row. Smaller chunks retrieve more precisely; larger chunks carry more context per match. Start around 300 tokens and adjust once you see real questions.

## Step 3: Retrieve the closest chunks

At request time, embed the question with the same model, then rank stored chunks by cosine distance using pgvector’s `<=>` operator. Lower distance means a closer match:

```
const { data } = await openai.embeddings.create({  model: 'text-embedding-3-small',  input: question,});const queryVec = `[${data[0].embedding.join(',')}]`;
const matches = await sql`  SELECT content, embedding <=> ${queryVec}::vector AS distance  FROM embeddings  ORDER BY distance  LIMIT 5`;
```

Five matches is a reasonable start. Return fewer if answers drift off-topic, more if the model lacks enough context to answer.

## Step 4: Answer with the model

Concatenate the retrieved chunks into a context block and pass it to a chat model as a system message. The same AI Gateway credentials cover chat completions:

```
const context = matches.map((m) => m.content).join('\n\n');
const reply = await openai.chat.completions.create({  model: 'gpt-5',  messages: [    {      role: 'system',      content: `Answer using only the context below. If the answer isn't there, say so.\n\n${context}`,    },    { role: 'user', content: question },  ],});
return reply.choices[0].message.content;
```

The instruction to answer only from context, and to admit when the context lacks the answer, is what keeps a RAG chatbot from inventing details. Keep it explicit.

## Step 5: Serve it from a function

Put steps 3 and 4 in a [Netlify Function](https://docs.netlify.com/build/functions/overview/) so your front end can POST a question and receive an answer:

```
import type { Config } from '@netlify/functions';
export default async (req: Request) => {  const { question } = await req.json();  // embed question, retrieve matches, call the model (steps 3–4)  return Response.json({ answer });};
export const config: Config = { path: '/api/ask' };
```

## Activating the AI Gateway

The AI Gateway turns on after your first production deploy. Until then, calls have no credentials to pick up. Deploy once with `netlify deploy --prod`, and the injected variables become available in Functions, Edge Functions, and the local preview server. If you already set `OPENAI_API_KEY` at the project or team level, Netlify leaves your value in place and does not override it.

## What this costs

Both halves bill through your Netlify credits rather than a separate provider invoice. The AI Gateway converts token usage into credits at the rates on the [AI features pricing page](https://docs.netlify.com/manage/accounts-and-billing/billing/billing-for-credit-based-plans/pricing-for-ai-features), and Netlify Database draws compute and storage from the same pool. Embedding your corpus once is cheap; the recurring cost is the per-question embedding plus the chat completion. One bill, one dashboard, no keys to rotate.

## When this fits

This pipeline handles the retrieval most apps need: a corpus of documents, questions answered from them, all in one pgvector table. If you grow into a very large corpus or need advanced retrieval (hybrid keyword-and-vector search, re-ranking, or heavy metadata filtering), a single table and a plain similarity query will start to limit precision, and a dedicated retrieval layer earns the extra moving part. Start here, and reach for more when a real retrieval-quality problem shows up. One caveat to plan for early: if you later switch embedding models, the vector dimension changes, so you re-embed the whole corpus and migrate the column.

## Resources

-   [AI Gateway overview](https://docs.netlify.com/build/ai-gateway/overview/)
-   [Netlify Database](https://docs.netlify.com/build/data-and-storage/netlify-database/) and [getting started](https://docs.netlify.com/build/data-and-storage/netlify-database/getting-started/)
-   [Build a RAG application with Neon, Netlify, and OpenAI](https://developers.netlify.com/guides/build-rag-application-with-neon-netlify-openai/)
-   [pricing for AI features](https://docs.netlify.com/manage/accounts-and-billing/billing/billing-for-credit-based-plans/pricing-for-ai-features)