简体中文
EnglishWebsite review
This review may include AI-generated content and is not individually verified. Domain, certificate and other facts come from the observed data on this page.
What is Haystack Documentation?
Haystack Documentation is the official documentation site for Haystack, an open-source framework from deepset for building production-ready LLM applications, particularly search, question answering, and retrieval-augmented generation (RAG) pipelines. It is a technical reference, not a product you buy: it explains how to install the framework, assemble components into pipelines, connect data stores and model providers, and deploy the result.
What you'll find there
- Concept and architecture guides covering pipelines, components, document stores, retrievers, readers/generators, and agents.
- Tutorials and walkthroughs that build working systems step by step, from a first RAG pipeline to more advanced routing and evaluation setups.
- API reference for classes, methods and parameters, useful once you know which component you need.
- Integration pages for model providers, vector databases, and file converters.
- Deployment and evaluation material, including guidance on serving pipelines and measuring retrieval or answer quality.
Who it's for
| Reader | Typical use |
|---|---|
| Python developers new to LLMs | Follow a tutorial to get a minimal RAG pipeline running, then adapt it |
| Engineers moving a prototype to production | Look up deployment patterns, scaling, and observability guidance |
| Teams comparing frameworks | Read the architecture and component model to judge fit and lock-in |
| Researchers and data scientists | Check evaluation and pipeline-composition options before writing custom code |
Trade-offs to expect
Haystack's documentation assumes comfort with Python and with concepts like embeddings and vector search; it is not a from-zero machine-learning course. Because the framework is modular, the docs are spread across many component and integration pages, so finding the right page often depends on already knowing the component name. Version changes can also mean older blog posts and Stack Overflow answers diverge from current docs, making the official site the safer reference.
A practical next step
If you are evaluating Haystack for a document-QA project, start with the "get started" or first-pipeline tutorial, run it against a small set of your own PDFs, and only then read the component API pages for the parts you need to change. That order keeps the breadth of the reference from becoming overwhelming. For background on the project itself, see Haystack and the maintainer, deepset.
How do I install Haystack to build my first question-answering pipeline?
Install Haystack with pip, then build a minimal pipeline that retrieves documents and passes them to a reader model. The library is a Python framework for search, retrieval-augmented generation and question answering, so the install step is short — the real work is choosing components and data.
Install
pip install haystack-ai
Use a virtual environment (python -m venv .venv) to keep dependencies isolated. If you plan to run local models rather than API-backed ones, install the matching integration package for your model provider or inference framework at the same time; Haystack itself stays lightweight and pulls in components as needed.
A first question-answering pipeline
The typical shape is: documents go into a document store, a retriever fetches the passages most likely to contain the answer, and a reader (or a generator, for RAG-style answers) extracts or writes the answer.
from haystack import Document, Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.document_stores.in_memory import InMemoryDocumentStore
store = InMemoryDocumentStore()
store.write_documents([
Document(content="Haystack is an open-source framework for building NLP pipelines."),
Document(content="A retriever finds candidate documents for a query."),
])
pipe = Pipeline()
pipe.add_component("retriever", InMemoryBM25Retriever(document_store=store))
result = pipe.run({"retriever": {"query": "What is Haystack?"}})
print(result["retriever"]["documents"])
Add a reader or generator component and connect retriever.documents to its input to turn retrieved passages into an actual answer. For a first project, an in-memory store plus BM25 avoids API keys and setup, so you can confirm the wiring works before swapping in a vector store or hosted model.
Practical choices
- Prototype first: in-memory document store, small sample corpus, one retriever.
- Then scale: swap in a persistent store and add embedding-based retrieval when keyword search misses paraphrased questions.
- Answers vs. passages: a reader extracts spans; a generator produces fluent answers and needs a prompt. Pick based on whether you need citations or conversational output.
Official references worth bookmarking: Haystack Documentation for component APIs and tutorials, and Haystack for project-level guides. If you prefer a hosted alternative for comparison, LangChain covers similar pipeline territory with a different abstraction style.
Next step: run the snippet above, then replace the sample documents with a handful of your own and watch which queries the retriever gets wrong — that tells you whether you need better chunking, embeddings or a different reader.
Which components can I use to connect Haystack to my own document store or vector database?
Haystack connects to external storage through its DocumentStore abstraction. Rather than wiring a database directly into a pipeline, you choose a DocumentStore class that matches your backend, and the rest of the pipeline talks to that interface. The practical question is whether your backend already has a supported class or whether you need to write one.
If your backend is already supported
Use the built-in DocumentStore for that system. Common examples include Elasticsearch, OpenSearch, Weaviate, Qdrant, Pinecone, Milvus, Chroma, Marqo, PGVector and FAISS. Each class handles the backend-specific details: connection setup, schema or index creation, embedding storage, filtering syntax, and retrieval methods such as BM25Retriever, EmbeddingRetriever or MultiModalRetriever.
This is the lowest-effort path. You get tested behavior, documented parameters and predictable filter support. The trade-off is that you inherit the class's assumptions about fields, distance metrics and query capabilities.
If your backend is not supported
Write a custom DocumentStore by subclassing DocumentStore from haystack.document_stores. In practice you implement the methods your pipeline actually uses:
- Writing:
write_documentsanddelete_documents - Reading:
filter_documents(or the olderget_all_documentsstyle access) - Retrieval: a
runmethod on a retriever component that queries your store, if you also need custom search - Index handling:
count_documentsand any index or collection setup your backend requires
Haystack's documentation includes a custom DocumentStore tutorial that walks through this, and the base class defines which methods are expected. If your database has a Python client, you typically wrap that client inside the DocumentStore rather than reimplementing network calls.
A quick comparison
| Situation | Component to use | Main trade-off |
|---|---|---|
| Backend has a built-in store | That specific DocumentStore class | Fast setup, but limited to supported features |
| Backend is proprietary or unusual | Custom DocumentStore subclass | Full control, but you own correctness and maintenance |
| You only need keyword search | Built-in store plus BM25Retriever |
No embedding pipeline, weaker semantic matching |
| You need vector search | Built-in store plus EmbeddingRetriever |
Requires embeddings and a compatible index |
A concrete decision rule
Start by checking whether your database appears in Haystack's supported DocumentStore list. If it does, use it and confirm that it supports the filters and retrieval mode you need. If it does not, estimate the work: if your backend exposes a Python client with basic CRUD and search, a custom DocumentStore is usually a contained task; if it only offers a REST API with unusual semantics, budget more time for testing write, delete and filter behavior.
For background, the Haystack Documentation covers both the supported stores and the custom store tutorial, and deepset provides the surrounding platform context.
How can I deploy a Haystack pipeline to production with an API?
Haystack's own documentation covers building pipelines, but it is not a deployment guide: it explains components and pipeline wiring, not how to run them behind a production API. For that, you wrap a pipeline in a web framework and add the operational layer around it.
The practical pattern
Expose your pipeline through a REST endpoint, keep the pipeline object loaded once at process start, and treat each request as a call into it.
- Framework: FastAPI, Flask, or Django REST Framework. FastAPI is the common choice because request and response models map cleanly to typed pipeline inputs and outputs.
- Endpoint shape: a
POST /queryor/pipeline/runroute that accepts a JSON body, callspipeline.run(...), and returns the answer plus any source documents. - Startup: build or load the pipeline once when the app starts, not per request. Re-initialising retrievers, embedders, or model clients on every call is the most common cause of slow, expensive deployments.
- Concurrency: sync pipelines block the worker. Either run them in a thread pool or use an async server setup so one slow query does not stall others.
- Secrets: keep API keys for model providers in environment variables or a secret manager, never in the repository.
Deployment options
| Option | Good for | Trade-off |
|---|---|---|
| Container on a VM or managed container service | Small teams wanting full control | You handle scaling, restarts, and monitoring |
| Serverless functions | Sporadic traffic, low cost at rest | Cold starts hurt large models; request time limits |
| Kubernetes | High, variable traffic; multiple services | Real operational overhead |
| Managed inference platforms | Teams avoiding infrastructure work | Less control over pipeline internals |
Before going live
Add health and readiness endpoints, structured logging of query latency and failures, request timeouts, and a rate limit. Cache embeddings or repeated queries where the data allows it. Version your pipeline config so you can roll back a prompt or model change without redeploying code.
For the pipeline-building side, start with Haystack Documentation. FastAPI's official docs at FastAPI show the request/response pattern this deployment needs.
A useful decision criterion: if your queries take more than a second or two, favour a long-running container over serverless, and load models at startup.
What are the differences between Haystack 1.x and Haystack 2.x for migrating my existing project?
Haystack 2.x is a ground-up redesign rather than an incremental upgrade, so migrating from 1.x usually means rewriting pipeline construction, node interfaces, and document-store wiring rather than tweaking imports. The core shift is from a fixed, opinionated pipeline object with predefined node types to a composable graph where you connect components explicitly and each component declares its inputs and outputs.
What actually changed
| Area | Haystack 1.x | Haystack 2.x |
|---|---|---|
| Pipeline model | Pipeline class with add_node and typed node roles (retriever, reader, generator) | Graph-based Pipeline where you connect named component sockets |
| Components | Nodes with fixed roles and implicit connections | Components with explicit run() inputs/outputs, reusable and testable in isolation |
| Retrievers | Tied closely to document stores | Decoupled; document stores and retrievers are separate, swappable pieces |
| Document stores | Store-specific classes with their own query methods | Common interface, so retrievers and writers work across backends |
| Custom code | Subclass a node base class and fit the expected signature | Write a component with a declared run signature; no central node registry to satisfy |
What this means for a migration
The practical consequence is that your retrieval and generation logic usually survives conceptually but not literally. A question-answering pipeline in 1.x that chained retriever to reader becomes an explicit graph in 2.x: a retriever component, an optional prompt builder, a generator, and an output adapter, each connected by name. If you wrote custom nodes, expect to rewrite them as components with explicit inputs and outputs, which is more work up front but makes them independently testable.
A useful decision criterion: if your project is small, stable, and pinned to a 1.x version, migrating buys you little. If you need new models, new document stores, or components you want to reuse across projects, 2.x is the direction the ecosystem is moving, and the graph model is what makes that reuse possible.
A concrete next step
Inventory your current pipeline as a list of steps and their data types, then map each step to a 2.x component before writing code. Start with a minimal retrieval-only graph to confirm your document store connects, then add generation. Keep the 1.x version running in a branch until the new graph produces comparable answers on a handful of known queries.
For the authoritative component list and migration notes, see Haystack Documentation. The broader project context lives at Haystack.
How do I fine-tune or evaluate a retrieval model within Haystack?
Haystack handles retrieval-model training and evaluation as two related but separate workflows: you fine-tune with a training pipeline, then measure the result with an evaluation pipeline that scores retrieval on labeled data.
Fine-tuning a retrieval model
The general pattern in Haystack is:
- Collect labeled data — query–document pairs, ideally with hard negatives (documents that look relevant but are not). The quality of these negatives usually matters more than the size of the dataset.
- Build a training pipeline — typically a
SentenceTransformersTraineror an equivalent trainer component wired to a document store and a dataset. - Run training — the trainer updates the embedding model (bi-encoder) or cross-encoder weights.
- Save and reload — write the fine-tuned model to disk and point your query and document embedders at that path in your inference pipeline.
Two model families are worth separating:
| Approach | What it trains | Best for |
|---|---|---|
| Bi-encoder (dense retrieval) | Query and document embedders | Fast first-stage retrieval over large corpora |
| Cross-encoder (reranking) | A joint query–document scorer | High-precision reranking of a short candidate list |
A common production setup fine-tunes a bi-encoder for recall and a cross-encoder for precision, then combines them.
Evaluating retrieval
Evaluation uses an evaluation pipeline with labeled queries and a set of metrics. In Haystack the standard building blocks are the EvaluationRunResult and evaluators such as DocumentMAP, DocumentMRR, DocumentRecall, and ContextRelevance.
A practical workflow:
- Hold out a portion of your labeled queries as a test set.
- Run the pipeline in evaluation mode so it records retrieved documents per query.
- Compute metrics across queries, then compare runs before and after fine-tuning.
Choosing a metric
- Recall@k — did the relevant document appear in the top k? Use this when a downstream reranker or reader will handle precision.
- MRR — how high did the first relevant document rank? Good for single-answer question answering.
- MAP — balances precision and recall across the ranking. Useful when multiple documents are relevant.
- Context relevance / NDCG — better when relevance is graded rather than binary.
A concrete decision path
If you have fewer than a few hundred labeled queries, start by evaluating an off-the-shelf model and tuning retrieval parameters (top-k, chunk size, hybrid weighting) before committing to fine-tuning. If you have thousands of labeled pairs with hard negatives and metrics plateau, fine-tune the bi-encoder, then evaluate again with the same test set to confirm the gain is real.
For exact component names, trainer arguments, and evaluator signatures, work from the official documentation at Haystack Documentation; for the underlying training library, see Sentence Transformers.
Amazon Route 53
Vercel
Google Workspace
User reviews (0)