RAG Storage Cost Calculator

Understanding RAG Infrastructure Costs

RAG pipelines have a two-phase cost structure that catches many teams off guard. The setup phase — converting documents into embeddings — is a one-time cost that scales with your data volume. The operational phase — hosting vectors and answering queries — is a recurring monthly cost that scales with both data size and query volume.

Most developers focus only on the LLM API cost (which is visible in their billing dashboard) and underestimate the vector storage and embedding costs. For a mid-size enterprise knowledge base of 50,000 documents averaging 20 pages each, the one-time embedding cost can easily reach $500–$2,000 depending on the model chosen — and the monthly storage cost adds another $50–$200 per month before any queries are made.

Phase 1: Embedding Generation (One-Time)

Before you can search your documents, you must convert every chunk into a vector embedding using an embedding model API. The cost is: total tokens × price per million tokens. A 400-page business document contains roughly 160,000 tokens when chunked at 512 tokens with 50% overlap, producing about 625 vectors. At $0.020/M tokens (text-embedding-3-small), that single document costs about $0.003 to embed — cheap in isolation, but a 10,000-document corpus at the same density costs $30 to embed.

Once embedded, you typically don't re-embed unless the document content changes. This makes embedding a capital expenditure rather than an operating cost. The key decision is choosing between a cheap model (lower upfront cost, fewer storage bytes, slightly lower retrieval quality) and a premium model (higher upfront cost, more storage, better semantic matching).

Phase 2: Vector Database Hosting (Monthly)

Every embedding must be stored in a vector database so it can be retrieved at query time using approximate nearest neighbour (ANN) search. Storage costs depend on vector count, embedding dimensions, and the provider's pricing model:

• Pinecone Serverless: charges $0.033/GB-month for stored vectors and a small per-query fee for ANN searches. Simple to use, no cluster management required.
• Weaviate Cloud: charges per million vector-dimensions per month ($0.05/1M), making it more cost-effective for lower-dimensional embeddings like Cohere or Voyage (1,024 dims vs. OpenAI's 1,536–3,072 dims).
• Qdrant Cloud: charges ~$0.040/GB-month for stored vectors, with a generous free tier (1 GB RAM). Strong performance for large datasets due to its Rust-based implementation.
• Supabase pgvector: storage is billed as standard PostgreSQL storage ($0.125/GB-month) plus a $25/month Pro plan base. The most economical choice if you already use Supabase — the marginal cost of adding pgvector is just the storage increment.

Phase 3: Query Costs (Per Request)

Every user query triggers two billable events: a vector retrieval (small fee to the vector DB) and an LLM generation call (larger fee to the language model). The retrieval cost is typically negligible — Pinecone serverless charges roughly $0.0001 per query; Weaviate and Qdrant even less. The dominant cost is almost always the LLM generation step.

With k=5 retrieved chunks at 512 tokens each, plus a 50-token user query, each generation call requires about 2,610 input tokens. At GPT-4o pricing ($2.50/M input, $10.00/M output), a 400-token response costs about $0.0105 per query — $315/month for 1,000 daily queries. Switching to GPT-4o mini ($0.15/M input, $0.60/M output) reduces that to about $0.0006 per query — $18/month. For most Q&A and document retrieval tasks, GPT-4o mini delivers comparable quality at a fraction of the cost.

Optimising Your RAG Budget

The highest-leverage cost optimisation for most RAG deployments is choosing the right LLM for generation. A 10× cheaper model (GPT-4o mini vs. GPT-4o) typically reduces monthly costs by 80–90% with minimal quality degradation for standard document Q&A. Embedding model choice has a smaller but meaningful impact: text-embedding-3-small produces 1,536-dim vectors at $0.020/M tokens; text-embedding-3-large produces 3,072-dim vectors at $0.130/M. The storage cost difference is proportional to dimensions — text-embedding-3-large uses twice the storage at 6.5× the embedding cost.

For most teams starting a new RAG project, the recommended baseline is: text-embedding-3-small for embeddings, Pinecone or Supabase for vector storage (depending on existing infrastructure), and GPT-4o mini or Claude Haiku for generation. Upgrade to larger models only after benchmarking retrieval quality on your specific domain.