Qlaudia

Dark mode

Chroma

Artificial Intelligence
Technology & Computing

Store and retrieve high-dimensional embeddings for semantic similarity search using vector indexing and approximate nearest-neighbour algorithms. Chroma simplifies building Retrieval-Augmented Generation (RAG) applications by handling embedding storage, HNSW indexing, and metadata filtering in a lightweight, deployable package.

What you'll learn

CollectionChroma's primary organizational unit, a named container that holds embeddings, associated documents, and metadata for items retrieved by vector similarity rather than exact matching.
Similarity Search (Query)A retrieval method that finds records in a collection whose semantic meaning is closest to a given query by comparing high-dimensional embedding vectors rather than matching exact words.
Vector EmbeddingA representation of text, images, or other data as an ordered list of numbers positioned in a high-dimensional space where semantically similar items cluster nearby.
Embedding FunctionA callable that takes raw data—typically a list of strings—and returns a corresponding list of vectors for use in Chroma's vector operations.
HNSW IndexA hierarchical graph-based approximate nearest neighbor index that Chroma uses internally to efficiently answer similarity queries by organizing vectors in multiple layers with short traversal paths.
Chroma Python ClientThe official Python library that provides a unified interface for creating collections, ingesting documents, and running similarity queries against Chroma whether running locally or as a remote server.
Chroma Vector DatabaseA lightweight, open-source vector database built to store embeddings and retrieve items by semantic similarity with minimal infrastructure overhead.
Metadata FilteringA retrieval technique that constrains vector search results to only records whose metadata satisfies specified field conditions, combining semantic similarity with contextual relevance.
Managed Vector Database (e.g., Pinecone)A cloud-hosted service that stores embedding vectors and handles infrastructure concerns like scaling, replication, and availability through a simple API.
Retrieval PipelineThe four-stage sequence of chunking raw text, embedding pieces into vector space, storing vectors in a database, and querying to find relevant results.
Document StorageChroma's mechanism for storing the original raw text of each document alongside its embedding vector so that search results include human-readable content.
Persistent StorageA configuration that writes Chroma's collections and embeddings to disk so they survive application restarts, rather than existing only in memory.
Retrieval-Augmented Generation (RAG)An architecture that combines a vector database for retrieving relevant external documents with a language model for generating responses grounded in that retrieved information.
Distance MetricA mathematical function that measures how close two vectors are to each other, used by Chroma to rank search results from most to least similar.
Add Documents OperationThe primary method for inserting data into a Chroma collection by supplying documents, embeddings, metadata, and IDs as parallel lists in a single call.
Cosine SimilarityA metric that measures vector similarity by computing the cosine of the angle between two vectors, which makes it effective for comparing text embeddings since it depends only on direction, not magnitude.
Chroma JavaScript ClientThe official TypeScript/JavaScript library for interacting with a Chroma server from Node.js or browser environments as a pure HTTP client over REST.
Collection Metadata SchemaThe set of structured key-value pairs attached to individual documents in a collection, used for exact matching and range filtering separate from semantic search.
LangChain IntegrationNative support for Chroma as a LangChain vector store that implements the VectorStore interface and enables seamless embedding, storage, and retrieval of documents within LangChain applications.
Sentence Transformers IntegrationA locally run embedding library that maps semantically similar text to nearby points in vector space without requiring cloud APIs or external services.
Semantic SearchA retrieval method that finds documents based on conceptual meaning using vector embeddings rather than exact keyword matching.
Chunking StrategyA set of decisions governing how documents are split into smaller, self-contained segments before ingestion into a vector store, affecting the granularity at which meaning can be retrieved.
In-Memory StorageA Chroma operating mode where all data is held entirely in RAM and discarded when the process ends, with no persistence to disk.
Prototyping vs. Production ScaleA vector store designed for rapid development without infrastructure complexity can introduce trade-offs when scaling to production environments.
Client-Server ModeAn architecture where Chroma runs as a standalone server process that multiple client applications connect to over a network through an HTTP REST API.
Delete OperationA Chroma operation that permanently removes one or more documents from a collection by targeting them via explicit IDs, metadata filters, or both.
LlamaIndex IntegrationA Python framework integration where Chroma serves as a native vector store backend for LlamaIndex to persist and query embedded document indexes.
Upsert OperationAn idempotent database operation that inserts a new record if its ID does not exist, or overwrites an existing record if the ID is already present.
k-Nearest Neighbors (kNN)A retrieval strategy that finds the k stored vectors closest to a query vector according to a distance metric, serving as the core operation for semantic search in vector databases.
Get OperationA retrieval method that fetches documents, embeddings, and metadata from a collection by exact lookup using IDs or metadata filters, without performing vector similarity search.
OpenAI Embedding IntegrationSupport for using OpenAI's family of embedding models as pluggable functions that convert text into dense numerical vectors for Chroma's similarity search.
Where Filter ClauseA structured dictionary expression that narrows a query or retrieval operation to only consider documents whose metadata fields satisfy specified conditions.
Where Document Filter ClauseA filtering mechanism that restricts query results based on whether the raw text content of documents contains or excludes specified keywords or phrases.
Embeddable In-Process ModeA deployment mode where the entire Chroma database runs inside the same process as the application, with no separate server or network overhead required.
Approximate Nearest Neighbor (ANN)An indexing and retrieval strategy that locates vectors close to true nearest neighbors by trading a small risk of imperfect results for dramatically faster query speeds on large collections.

About this course

Chroma Overview

Chroma is an open-source vector database built specifically to help developers store, search, and retrieve high-dimensional embeddings with as little friction as possible. It occupies a particular niche in the AI tooling ecosystem: it is the go-to starting point for teams building Retrieval-Augmented Generation (RAG) applications who want to move from zero to a working prototype in minutes rather than days. Understanding what Chroma is, what problem it solves, and where its boundaries lie will give you a clear map of the territory before diving into any individual feature.

The Problem Chroma Solves

Modern language model applications rarely rely on the model's parametric memory alone. Instead, they retrieve relevant context from an external knowledge base at query time and pass that context to the model. The retrieval step requires a very different kind of search from traditional keyword matching: you need semantic similarity search, where you find documents whose meaning is close to the query, not just documents that share the same words.

Semantic similarity is measured over embedding vectors — dense numerical representations of text (or images, audio, etc.) produced by an encoder model. A query is embedded into the same vector space as the stored documents, and the database returns the documents whose vectors are nearest to the query vector under some distance metric such as cosine similarity or L2 distance.

$\text{cosine\_similarity}(\mathbf{u}, \mathbf{v}) = \frac{\mathbf{u} \cdot \mathbf{v}}{\|\mathbf{u}\| \|\mathbf{v}\|}$

Building this infrastructure from scratch — managing embedding storage, indexing those vectors for fast approximate nearest-neighbour (ANN) search, filtering by metadata, and persisting everything to disk — is non-trivial. Chroma packages all of this into a single, lightweight library so you can focus on your application instead.

Core Architecture

Chroma's architecture is organised around a small number of concepts that work together.

Figure 1: How the main Chroma components relate — your application talks to a Client, which manages Collections backed by an HNSW vector index, with an embedding function converting raw text into vectors.

Collections

A collection is the fundamental organisational unit in Chroma, analogous to a table in a relational database. Each collection holds a set of items, where each item consists of:

A document — the raw text (or other modality) being indexed.
An embedding — the vector representation of that document, either provided by you or generated automatically.
Metadata — a key-value dictionary for structured filtering (e.g., {"source": "arxiv", "year": 2024}).
An ID — a unique string identifier for the item.

Embedding Functions

Rather than requiring you to embed documents separately before inserting them, Chroma accepts a pluggable embedding function at collection creation time. When you add raw text, Chroma calls this function internally to produce the vectors. The default uses the all-MiniLM-L6-v2 sentence-transformers model locally; you can swap in OpenAI embeddings, Cohere, HuggingFace, or any callable you define.

The HNSW Index

Under the hood, Chroma stores vectors in an HNSW (Hierarchical Navigable Small World) index via the hnswlib library. HNSW is a graph-based approximate nearest-neighbour algorithm that achieves sub-linear query time by navigating a multi-layer proximity graph. The trade-off versus exact search is a small, configurable recall loss in exchange for dramatically faster queries as the collection size grows.

Deployment Modes

One of Chroma's defining characteristics is that it offers three deployment modes with a largely consistent API:

Mode	How it runs	Persistence	Typical use
In-process / ephemeral	Embedded in your Python process, data in memory	❌ Lost on exit	Unit tests, quick experiments
In-process / persistent	Embedded, data written to a local directory	✅ Survives restarts	Local development, single-machine apps
Client-server	Chroma runs as a separate HTTP server	✅	Multi-process access, staging environments

The ability to start with zero infrastructure — just pip install chromadb and a few lines of Python — is the primary reason Chroma is the most common first vector database teams reach for.

Key Operations

Working with Chroma comes down to four operations on a collection:

add — Insert documents (and optionally pre-computed embeddings and metadata).
query — Given one or more query texts or vectors, return the $k$ most similar items, optionally filtered by metadata.
update / upsert — Modify existing items or insert them if they don't exist.
delete — Remove items by ID or metadata filter.

The query call is where Chroma does its core work: it embeds the query text, searches the HNSW index for the nearest vectors, applies any metadata filters, and returns ranked results with distances.

Ecosystem Integrations

Chroma has first-class integrations with the two dominant LLM orchestration frameworks:

LangChain — Chroma is available as a VectorStore implementation, meaning any LangChain retriever or RAG chain can swap Chroma in with one line of configuration.
LlamaIndex — Chroma is a supported VectorStore in LlamaIndex's storage layer, compatible with its VectorStoreIndex and query engines.

These integrations mean that prototype code written against Chroma can often be migrated to a different vector store later by changing a single constructor call.

Where Chroma Fits — and Where It Doesn't

Chroma is deliberately scoped. It excels at:

Rapid prototyping of RAG and semantic search pipelines.
Local development where spinning up infrastructure is unnecessary overhead.
Educational contexts where the goal is understanding the retrieval pattern, not operating a database cluster.

It is not designed for:

High-throughput production workloads with millions of vectors and concurrent users — teams typically migrate to managed services such as Pinecone, Weaviate, or Qdrant at that stage.
Advanced filtering on complex structured queries — its metadata filtering is simple key-value matching, not a full query language.
Multi-tenancy and access control — Chroma has no authentication layer in its default deployment.

Understanding these boundaries early saves the common mistake of architecting a production system around Chroma only to hit scaling constraints later. Use it to learn the patterns and prove your retrieval logic; treat it as a prototype accelerator rather than a permanent foundation.