Qlaudia

Dark mode

Haystack

Artificial Intelligence
Technology & Computing

Assemble LLM-powered applications using typed components and explicit directed graph pipelines that enforce data contracts and enable modular testing. Haystack integrates retrievers, generators, document stores, and external services into production-ready systems with built-in observability and reproducibility.

What you'll learn

Component ModelThe fundamental unit of computation in Haystack, implemented as a Python class decorated with @component that takes typed keyword arguments as inputs and returns a typed dictionary as outputs.
Query PipelineA Haystack pipeline dedicated to the read path of RAG systems, which takes a user question, embeds it, retrieves relevant document chunks from a store, and generates a grounded answer using a language model.
Pipeline ConstructionThe process of instantiating components, registering them as named nodes, and declaring typed edges that define data flow between them.
Pipeline TestingA systematic approach to verifying that Haystack pipelines work reliably in production by testing components in isolation and end-to-end, leveraging typed inputs and outputs to mock external dependencies and handle non-determinism.
Custom Component CreationA way to encapsulate any transformation or side-effect as a first-class Haystack component by decorating a class with @component and defining a run() method that accepts typed inputs and returns a dictionary of typed outputs.
Retrieval-Augmented GenerationA technique that grounds language model outputs in external knowledge by combining a retrieval system—which finds relevant documents from a knowledge base—with a generation system that produces answers based on those documents rather than parametric memory alone.
Component Inputs and OutputsTyped data sockets through which Haystack components communicate with the rest of the pipeline, declared via the run() method signature for inputs and the @component.output_types decorator for outputs.
Component RegistryHaystack's internal catalogue that maps string names to component classes, enabling pipelines serialized to YAML or JSON to be reconstructed by looking up components during deserialization.
Pipeline SerializationThe process of converting a fully constructed pipeline into a portable text format and reconstructing it identically from the saved file.
Directed Graph ExecutionThe execution model where Haystack pipelines are structured as directed acyclic graphs with components as nodes and typed data flowing along directed edges, enabling deterministic and parallel execution.
Document StoreThe persistence and indexing layer in Haystack that stores documents with their content, metadata, and optional embeddings while exposing a query interface for retrievers.
Retriever ComponentA Haystack component that queries a Document Store and returns documents most relevant to an input query, forming the first stage of RAG pipelines where retrieval quality directly determines whether the generator receives the right context.
Agentic WorkflowsA pipeline pattern where a language model reasons about what action to take next and routes itself dynamically through different tools or branches based on intermediate outputs rather than following a fixed sequence.
LLM IntegrationTyped connectors to external language model providers that function as proper Haystack components with declared inputs, outputs, and pipeline participation.
Deepset Cloud IntegrationThe managed platform by Deepset that enables Haystack pipelines to be deployed, scaled, and monitored in production without requiring teams to manage their own infrastructure.
Indexing PipelineA Haystack pipeline that processes raw documents offline through conversion, preprocessing, embedding, and writing stages to populate a document store for retrieval tasks.
Evaluation PipelineA structured Haystack pipeline that measures the quality of another pipeline using test questions and quantitative metrics to detect retrieval and generation failures.
Haystack 2.0 ArchitectureA ground-up redesign of Haystack using typed components, explicit directed graphs, and strict serialization to make AI pipelines maintainable and debuggable at production scale.
Embedder ComponentA Haystack component that converts text into a dense numerical vector encoding semantic meaning, enabling vector-based search and retrieval.
BM25 RetrieverA Haystack component that ranks documents using classical information retrieval statistics based on term frequency and inverse document frequency without requiring machine learning models or embeddings.
Vector / Embedding RetrieverA Haystack component that finds documents by comparing semantic meaning in a high-dimensional vector space rather than by matching keywords, enabling retrieval of semantically relevant documents even without word overlap.
Haystack vs LangChainHaystack optimizes for time to diagnose failures in production systems through explicit typed graphs, while LangChain optimizes for rapid prototyping through high-level composable abstractions.
Metadata FilteringThe process of restricting document retrieval to only those documents satisfying logical conditions on structured key-value fields stored alongside content.
Generator ComponentA Haystack component wrapping a large language model that synthesizes text output from a prompt built from retrieved documents and user queries.
Document PreprocessingThe pipeline stage that transforms raw heterogeneous source material like PDFs and HTML into clean, consistently structured Document objects ready for embedding and retrieval.
Document SplitterA Haystack component that breaks long documents into shorter, semantically focused chunks that fit within model context windows and improve retrieval relevance.
Router ComponentA Haystack component that inspects incoming data and directs it to one of several downstream branches based on the content, enabling conditional logic and multi-path execution in pipelines.
Prompt Builder ComponentA Haystack component that constructs the final text prompt for a Generator by templating dynamic values—such as retrieved documents and user queries—using Jinja2 placeholder syntax.
Pipeline DebuggingThe systematic technique of isolating and testing individual components outside the pipeline to locate where failures originate.
Conversation MemoryThe mechanism by which a chat application retains and injects the history of prior exchanges into each new prompt so the language model can answer follow-up questions and maintain conversational context.
Pipeline VisualizationA built-in Haystack feature that renders a pipeline's directed graph structure as a human-readable Mermaid diagram to communicate architecture, identify structural mistakes, and understand data flow between components.
Hybrid RetrievalA search strategy that combines keyword-based sparse retrieval and vector-based dense retrieval, merging their results into a single ranked list to leverage the strengths of both approaches.
Tool Use in AgentsThe mechanism that allows an agent to access external capabilities by emitting tool calls, which the framework executes and returns as observations back to the LLM.

About this course

Haystack Overview

Haystack is an open-source AI framework developed by Deepset for building production-ready applications powered by large language models (LLMs). It provides the scaffolding to assemble retrieval-augmented generation (RAG) pipelines, conversational agents, document processing workflows, and other AI-driven systems — all from a consistent set of composable building blocks.

Understanding what Haystack is for requires understanding the problem it addresses. When you build an LLM application, you rarely talk to a model in isolation. You retrieve documents, embed queries, rank results, format prompts, call external APIs, route inputs based on conditions, and stream outputs to users. Each of these steps is a distinct computational unit, and the challenge is wiring them together reliably, testing them individually, and diagnosing failures when something goes wrong in production. Haystack's design philosophy is built entirely around making that challenge tractable.

Historical Context: The 2.0 Rewrite

Haystack's original version was primarily focused on question-answering pipelines over document stores. With the explosion of LLM capabilities, Deepset undertook a full architectural rewrite — Haystack 2.0 — to generalise the framework to arbitrary AI workflows. The 2.0 redesign introduced a strict component model with typed inputs and outputs, and replaced opaque abstraction chains with explicit directed graph pipelines that can be visualised, validated, and debugged node by node.

This stands in deliberate contrast to frameworks that hide orchestration logic behind deep call stacks. In Haystack 2.0, every connection between components is declared explicitly, every data contract is typed, and the pipeline itself is an inspectable data structure — not a runtime side effect.

Core Concepts at a Glance

Haystack's architecture is built around five interlocking ideas:

Components — The atomic units of computation. A component is a Python class with a run() method and explicitly declared typed inputs and outputs. Examples include OpenAIGenerator, InMemoryBM25Retriever, SentenceTransformersDocumentEmbedder, and PromptBuilder.

Pipelines — Directed acyclic (or cyclical, for agents) graphs of components. A pipeline defines how data flows from one component to the next. Pipelines are assembled programmatically and can be serialised to YAML for reproducibility.

Document Stores — Backend storage systems for documents and their embeddings. Haystack integrates with in-memory stores for development and production stores like OpenSearch, Weaviate, Qdrant, and Elasticsearch.

Integrations — First- and third-party connectors that expose external services as Haystack-compatible components. This includes LLM providers (OpenAI, Cohere, Mistral, HuggingFace), embedding services, rerankers, and tool-calling wrappers.

Agents and Tools — For workflows that require iterative reasoning or external action, Haystack supports tool-calling loops where an LLM can invoke registered tools repeatedly until a stopping condition is reached.

A simplified RAG pipeline in Haystack: a query triggers the Retriever (backed by a Document Store) and the PromptBuilder in parallel; their outputs feed a Generator whose reply is returned as the answer.

Why Haystack for Production?

Several design choices make Haystack particularly well-suited for production engineering teams:

Type safety at connection time. When you connect two components, Haystack validates that the output type of the upstream component matches the input type of the downstream component. Mismatches are caught before the pipeline ever runs.
Serialisation. Pipelines can be written to and loaded from YAML, enabling version control, reproducibility, and configuration-driven deployment.
Isolated unit testing. Because each component is an independent Python class with a run() method, you can unit test any component in isolation — passing synthetic inputs and asserting on outputs — without running the entire pipeline.
Observability. The directed graph structure means you always know which component produced which output, making logging, tracing, and debugging far more straightforward than in frameworks with hidden orchestration layers.

What Haystack Is Not

Haystack is not a managed cloud service, a prompt engineering UI, or a model fine-tuning toolkit. It is a framework for pipeline orchestration — it gives you the wiring, the components, and the contracts. The models, embedders, and vector databases are plugged in via integrations.

Summary

Haystack provides a principled, architecturally transparent way to build LLM-powered applications. Its component-and-pipeline model enforces explicit data contracts, enables modular testing, and produces systems that are debuggable by design. For teams building RAG systems, agents, or document processing pipelines who need confidence in what their code is doing at every step, Haystack offers a level of structural clarity that is rare in the LLM tooling ecosystem.