Chroma: The Vector Database You Can Learn in an Afternoon (and When to Outgrow It)
Most vector databases ask you to think about clusters, indexes and infrastructure before you store a single document. ChromaDB asks you to run pip install chromadb and write about six lines of Python. That is the entire reason the chroma vector database became the default first stop for people learning RAG.
The same design, an embedded database living inside your Python process, is also why there is a ceiling. This guide covers both halves honestly: everything you need for a persistent, queryable Chroma DB store before the afternoon ends, and the specific signals that mean it is time to move.
The project lives at trychroma.com, with documentation linked throughout. Every code block below was executed against ChromaDB 1.5.9, the latest release as of 21 August 2026.
What Is ChromaDB?
When someone asks "what is ChromaDB" or "what is Chroma DB", this is the answer:
ChromaDB is an open-source vector database designed for AI applications. It stores documents together with their embeddings and metadata in collections, generates the embeddings for you by default, and returns the most semantically similar documents to a query. It runs embedded inside your application, as a server, or as a managed cloud service.
Is ChromaDB a vector database in the strict sense? Yes, but shaped differently. Most vector databases are servers you connect to; ChromaDB is, by default, a library you import. The database runs inside your process and writes to a local directory: no cluster, no connection string, no credentials. That one decision explains both its ease and its ceiling, and the rest of this guide follows from it. (What a vector database does and why similarity search needs one is its own guide; one sentence here, then we move.)
ChromaDB also speaks its own vocabulary where other tools talk about "points" and "payloads", which trips up readers arriving from other databases:
| Term | What ChromaDB means by it |
|---|---|
| Collection | A named set of documents with their vectors, like a table |
| Document | A piece of raw text you stored |
| Embedding | The vector generated for a document or query |
| Metadata | Key-value facts attached to a document, usable as filters |
| Query (n_results) | A similarity search asking for the n closest documents |
Build an AI-First Career, Master the Complete Skillset
Choose from our industry-leading programs designed for career success
Modern Software and AI Engineering Program
Master full-stack development with AI integration
+1000 moreModern Data Science and ML with specialisation in AI
Advanced data science techniques with AI specialization
+1000 moreAdvanced AIML with Specialisation in Agentic AI
Deep dive into AIML with focus on Agentic systems
+1000 moreDevOps, Cloud & AI Platform Engineering
Build and manage AI-powered cloud infrastructure
+1000 moreAI Engineering Advanced Certification by IIT-Roorkee
Premier AI engineering certification from IIT-Roorkee
AI Forward Deployed Engineer Program
Full-stack engineering, production AI and client-facing consulting
+1000 moreIs ChromaDB Free and Open Source?
Yes. The ChromaDB engine is open source under Apache 2.0 and free to run locally or on your own server forever, with no seat or query limits. Chroma Cloud is the separate paid managed service (real rates in the pricing section), and new users get $5 in credits.
The ChromaDB source on GitHub (29,000+ stars as of August 2026) is the source of truth for the licence and the roadmap. The company behind it is venture-funded with a commercial cloud product, which for a reader means one thing: active development and a moving feature set, so date-stamp anything you read about it, including this page.
Why ChromaDB Is the Fastest Vector Database to Get Running
Because the database runs inside your Python process, there is no network hop between your code and your vectors, nothing to start before your script runs, nothing to authenticate against, and nothing to deploy for a demo. A notebook, a laptop and a folder are the entire stack. The honest flip side comes in the same breath: your database's availability is your script's availability, and two processes cannot safely share one local directory.
There are four ways to run it, and choosing between them is the first real decision:
| Mode | How you create it | Data survives restart? | Use it for |
|---|---|---|---|
| Ephemeral (in-memory) | chromadb.Client() | No; memory only, gone when the program exits | Notebook experiments and tests |
| Persistent | chromadb.PersistentClient(path="...") | Yes; writes to a local directory (.chroma by default) | Almost all local development; the sane default |
| Client-server | chroma run CLI or the official Docker image, then HttpClient | Yes | Several processes or services sharing one store |
| Chroma Cloud | CloudClient with API credentials | Yes; managed | Production without running infrastructure |
Here is the thing most tutorials omit: the chromadb.Client() used across the internet is the ephemeral one. The single most common beginner surprise with ChromaDB is "my documents disappeared", and they were never lost; they were never saved. Our quickstart uses PersistentClient from the first line.
The Rust core, correcting the record. You will still read that ChromaDB is "a thin Python wrapper over SQLite". Out of date. Since version 1.0.0 the core was rewritten in Rust, and Chroma's official announcement reports local Chroma "4x faster" overall with "3-5x faster writes" and "3-5x faster queries", plus "true multithreading" free of Python's GIL, measured on "a dataset of 1M OpenAI 1536 dimensional embeddings" using "default local Chroma settings on a 12 core M2 Macbook". The current chromadb latest version is 1.5.9 (PyPI, 21 August 2026). One honest caveat: a faster single node is still a single node. That is the subject of the outgrow section. (Under the hood sits an approximate-nearest-neighbour index; how those work is a sibling topic.)
The Afternoon Quickstart: ChromaDB in Python, Five Steps
Follow these five blocks in order and you end with a persistent, queryable database. Verified by execution against the official getting-started guide and a live 1.5.9 interpreter on 21 August 2026.
Step 1: Install.
pip install chromadb
The install pulls a fair amount, including the default embedding model's dependencies, so give a slow connection a few minutes. There is no separate Chroma DB download to manage afterwards.
Hit ModuleNotFoundError: No module named 'chromadb' anyway? You installed into a different interpreter than the one running your script. Use python -m pip install chromadb inside the right environment; if Python environments are the part tripping you up rather than ChromaDB itself, firm up the Python fundamentals first.
Step 2: Create a persistent client.
import chromadb
Contrast this with chromadb.Client(), which is memory-only. The moment this line runs you get a real directory on disk containing chroma.sqlite3 and the index data.
Step 3: Create a collection.
Use get_or_create_collection() rather than create_collection() in any script you will run twice: calling create_collection() on an existing name raises an error (an InternalError on 1.5.9), and this papercut catches every beginner on their second run.
Step 4: Add documents.
Notice what you did not pass: embeddings. ChromaDB silently downloaded a model, embedded both documents and stored the vectors, which is exactly why it feels easy. Add metadatas=[...] from day one (the next subsection shows why), and reach for upsert() when you want "insert or update" behaviour on re-runs.
Step 5: Query.
The returned dictionary confuses everyone on first contact, so here it is explained rather than just printed:
- documents, ids, metadatas and distances are lists of lists: one inner list per query string you passed. With one query and n_results=2, results["ids"] is [["id1", "id2"]].
- embeddings is None unless you ask for it with include=["embeddings"].
- distances are distances, not similarity scores: lower is closer. In our executed run the pineapple document scored 1.0404 against 1.2431 for oranges, so it ranked first.
Add Metadata and Filter Your Search
Both filters above were executed and returned exactly the expected document. Metadata filtering is what turns a demo into something usable: filter by source, user, language or date instead of searching everything. How filtering is designed inside a vector search engine is covered by the Qdrant deep dive; the mechanism is its own topic.
How Scaler Transformed Careers in Different Fields
Scaler learners achieved 2.5x salary growth with average post-Scaler CTC reaching ₹23L.
Google
Amazon
Microsoft
Flipkart
Adobe1200+ moreInspect What You Stored
Three calls beginners rarely discover: collection.count() for the document total, collection.peek(5) for a quick look at the first entries, and collection.get(ids=["id1"]) to fetch exact records. For a ChromaDB UI, community and emerging official tools let you browse collections visually; confirm what is current in the docs at publish time, since this corner of the project moves quickly.
Embeddings Without the Setup, and How to Swap the Model
The magic in Step 4 has a precise name. When you create a collection without specifying an embedding function, ChromaDB uses all-MiniLM-L6-v2 from Sentence Transformers, and it runs locally on your machine (our test environment downloaded a 79 MB ONNX model on first use). That is why the quickstart needed no API key, no account and no internet after the first download. Practically: free, private, offline-capable and fast on CPU, but a small general-purpose English model.
Swap it when you hit a concrete trigger: your content is not English (a live concern for Hindi, Tamil or code-mixed Indian-language corpora, where a multilingual model changes retrieval quality dramatically); your domain is specialised, like legal or medical text; your documents are long enough that the default's small context window truncates them; or you must match embeddings another system already produces.
from chromadb.utils.embedding_functions import OpenAIEmbeddingFunction
The embedding function applies automatically on add, upsert and query, so query text is embedded the same way as documents; that consistency is the correctness point beginners miss. ChromaDB's docs list around a dozen providers, including OpenAI, Cohere, Hugging Face, Jina AI, Mistral, Together AI, Google Generative AI and Cloudflare Workers AI (current list).
One trap worth a weekend of your life: the embedding function is bound to the collection. Change models later and your old vectors are incomparable with new ones; you must re-embed everything into a fresh collection. Decide the model before you ingest a large corpus, not after, and if you are unsure which model fits your content, that choice deserves its own guide.
Turn Learning into Career Growth
From Quickstart to a Working RAG App
The pipeline: load documents, split them into chunks, add() them to a persistent collection, query() with the user's question, and pass the retrieved chunks to an LLM as context. ChromaDB occupies exactly one box in that pipeline, and the parts people get wrong usually sit on either side of it. How you split documents affects retrieval quality more than which vector database you chose; how to split documents into retrievable chunks is the natural next read.
If you want to see the whole pipeline running rather than just the vector-store step, Scaler's walkthrough covers it end to end:
Framework integrations. ChromaDB is a first-class chroma vector store in LangChain and LlamaIndex, and Spring AI supports it for JVM teams (genuinely useful in India's many Java shops). The LangChain ChromaDB integration lives in the langchain_chroma package, verified today:
Three field-tested tips: always set an explicit persist_directory with a framework wrapper, because wrapper defaults differ from raw ChromaDB; batch your add() calls instead of adding documents one at a time; and store the source filename and chunk index in metadata from day one, because the first time someone asks "where did this answer come from", you will need them. When the app works and you want it seen, our list of generative AI projects worth building for a portfolio shows how projects like this read to a recruiter.
When to Outgrow ChromaDB: the Honest Signals
ChromaDB is genuinely good at what it is designed for, and "you outgrew it" is a success condition, not a mistake. Most projects never need to leave. But the ceiling is real, specific and measurable, and nobody else on this SERP draws the line. Drawn from the project's own architecture, here is where it sits:
| Signal | What it looks like | Why ChromaDB strains | Next step |
|---|---|---|---|
| Dataset size | Millions of vectors no longer comfortable on one machine | Single-node design; no sharding | A distributed engine (Qdrant, Milvus) or managed service |
| Concurrency | Several processes or real simultaneous web traffic | Embedded mode is single-process; server mode is still one node | An engine with pooling and replicas |
| Multi-tenancy | Many customers needing isolated, filtered data | Collection-per-tenant stops scaling; coarse filter isolation | Indexed payload filtering and tenant-aware sharding |
| Operations | You need HA, replication, backups, an SLA | No built-in replication or failover when self-hosted | Chroma Cloud, or a database with clustering |
| Query complexity | Heavy filtering, hybrid keyword plus semantic search, reranking | Simpler filtering than purpose-built engines | Qdrant's filtering, or a hybrid search stack |
| Relational gravity | Vectors must join with data already in Postgres | Separate store, backup and consistency story | pgvector inside your existing Postgres |
| Cost shape | Cloud bills climbing with query volume | Usage pricing scales with reads | A self-hosted alternative on a fixed-cost VM |
If you arrived searching chromadb vs qdrant, pinecone vs chromadb, faiss vs chromadb, milvus vs chromadb or pgvector vs chromadb, that table is the honest answer: framed around your situation, not a feature grid.
Just as important, here is what is not a reason to leave: a few hundred thousand documents is fine; "slow queries" that are actually slow embedding calls are not a database problem; and "someone on Reddit said it does not scale" is not a measurement. Measure first: collection size on disk, p95 query latency, and where request time actually goes. Knowing when to scale, and when not to, is itself one of the AI engineering skills that come after your first RAG project.
What migration actually costs. Nobody publishes this part. First, you usually re-embed everything: embeddings are model-specific and rarely portable, so the switch costs API spend and wall-clock time. Second, you rewrite the ingestion and retrieval layer, though a framework abstraction reduces that to a swap. Third, you now operate a service, with backups, upgrades and monitoring. Fourth, you re-tune, because filter and index behaviour differ. A small corpus is an afternoon; a large multi-tenant system is a project. And keeping ChromaDB for local development while running something bigger in production is a completely legitimate end state that many teams land on.
ChromaDB Cloud: Pricing and the Managed Middle Ground
Chroma Cloud keeps the API you already learned and removes the single-node problem: the lowest-friction escape hatch for teams who like ChromaDB and simply need it to be someone else's operational problem.
Published rates (Chroma Cloud pricing, verified 21 August 2026): 2.50 per logical GiB via add, update or upsert (logical GiB means the raw uncompressed data you send, billed once per write); storage 0.0075 per TiB queried plus $0.09 per GiB returned; no stated minimum spend. The query accounting is unusual enough to spell out: one vector similarity search counts as one query, each metadata or full-text predicate adds one, and full-text or regex filters bill as (N minus 2) queries where N is the search string's length in characters. A 40-character full-text filter therefore bills like 38 queries: heavy regex filtering can dominate a bill in a way pure vector search does not.
Worked example, labelled an estimate with stated assumptions (rerun it against the live pricing page; INR at ≈ ₹96/USD, 20 Aug 2026):
| Line item | Rate | Our assumption | Estimate |
|---|---|---|---|
| Writes, one-time | $2.50 per logical GiB | 3 GiB ingested once | $7.50 |
| Storage | $0.33 per GiB per month | 2 GiB stored | $0.66 per month |
| Queries | $0.0075 per TiB queried | 50,000 simple searches vs 2 GiB | $0.73 per month |
| Data returned | $0.09 per GiB | 5 chunks of 10 KB per query | $0.21 per month |
| Monthly total | month 1 ≈ KaTeX parse error: Expected 'EOF', got '₹' at position 9: 9.10 (≈ ₹̲875), then ≈ 1.60 (≈ ₹155) |
One footnote that matters to Indian teams: because self-hosted ChromaDB is just a container writing to a directory, a team with data-localisation obligations can run it in an Indian cloud region on its own account, no vendor region list required. That optionality is a quiet advantage of the open-source-first model; confirm your own legal requirements rather than treating this as legal advice.
If you would rather learn the production side of this stack in a guided sequence than by trial and error, our roundup of structured generative AI courses covering RAG and deployment is the pragmatic next step.
FAQs
What is the ChromaDB vector database?
An open-source vector database for AI applications. It stores documents with their embeddings and metadata in collections, generates embeddings automatically, and returns the most semantically similar documents to a query. It runs embedded in your app, as a server, or in the cloud.
Is Chroma vector DB free?
Yes. The ChromaDB engine is open source under Apache 2.0 and free to run locally or self-hosted, with no query or seat limits. Chroma Cloud is the separate paid managed service; new users get $5 in credits (as of August 2026).
Is Chroma a good vector database?
For learning, prototyping and small-to-mid production apps, yes: it is the fastest vector database to get running. It strains when you need millions of vectors, real concurrency, multi-tenancy or high availability, at which point a distributed engine or managed service fits better.
Can ChromaDB be used in production?
Yes, within limits. Self-hosted ChromaDB is a single node with no built-in replication or failover, so it suits internal tools and modest traffic. For high availability or heavy concurrency, use Chroma Cloud or a clustered vector database.
Does ChromaDB store data permanently?
Only if you ask it to: chromadb.Client() is in-memory and loses everything when the process ends. Use chromadb.PersistentClient(path="...") for disk persistence, or run client-server or cloud mode.
What embedding model does Chroma use by default?
all-MiniLM-L6-v2 from Sentence Transformers, running locally on your machine, which is why the quickstart needs no API key. You can swap in OpenAI, Cohere, Hugging Face, Jina, Mistral and others, per collection.
How do I install ChromaDB?
Run pip install chromadb. If you then see ModuleNotFoundError: No module named 'chromadb', you installed into a different interpreter than the one running your script; use python -m pip install chromadb in the right environment.
What are the top 5 vector databases?
There is no single ranking. ChromaDB leads on ease of getting started, Qdrant and Milvus on self-hosted scale and filtering, Pinecone on fully-managed operations, and pgvector for teams already on Postgres. The right pick depends on scale, ops capacity and where your data lives.
Conclusion
An afternoon really is enough to learn this chroma vector database, and that is a genuine engineering virtue, not a compromise: the fastest path from an idea to a working retrieval system is worth a great deal when you are learning. The same in-process design that makes it effortless is also what puts a ceiling on it, and that ceiling is specific: dataset size, concurrency, tenancy, operations. Know the signals, measure before you move, and let the tool do its job until it cannot.
Knowing when a tool stops fitting is a large part of engineering judgement. If you would rather build that judgement across RAG, embeddings and deployment as a structured path with mentorship than piece it together from tutorials, explore Scaler's AI & Machine Learning program.
