How to Get Started with Ragas

To get started with Ragas, install the package, run the Quick Start, then follow the tutorial that matches what you are building: a prompt, a RAG system, an AI workflow, or an AI agent. Ragas is an evaluation framework for AI applications, so the fastest path is to evaluate one small, real system you already have rather than reading the whole documentation first. The steps below follow the structure of the official docs at docs.ragas.io.

Step 1: Install Ragas

Ragas is a Python package. Install it in the environment where your AI application runs, then confirm the import works before writing any evaluation code.

pip install ragas

import ragas
print(ragas.__version__)

Expected result: the version prints without errors. If the import fails, check that you installed into the same Python environment your application uses (a common issue when a virtual environment is active in one terminal but not another).

Step 2: Run the Quick Start

The docs place Get Started → Installation → Quick Start at the front of the navigation. Work through the Quick Start before the tutorials: it shows the smallest complete evaluation loop — supply a dataset, choose a metric, run the evaluation, read the scores. Once that loop runs end to end, every later page is a variation on it.

Step 3: Pick the tutorial that matches your application

The Tutorials section is organized by application type. Choose based on what you are actually evaluating:

Start with the one closest to your current project. Running one tutorial against your own data teaches more than reading all four.

Step 4: Learn the core concepts

The Core Concepts section explains the three things every Ragas evaluation combines:

• Datasets — the inputs, contexts, responses, and reference answers your evaluation runs on.
• Metrics — the scoring functions. The docs group them into families: RAG metrics (context precision, context recall, noise sensitivity, response relevancy, faithfulness), agent and tool-use metrics (topic adherence, tool call accuracy, tool call F1, agent goal accuracy), natural language comparison (factual correctness, semantic similarity), traditional non-LLM metrics (BLEU, ROUGE, CHRF, exact match, string similarity), SQL metrics, and general-purpose metrics (aspect critic, rubric-based scoring).
• Experimentation — running evaluations repeatedly to compare versions of your application.

Read the Metrics Overview and Available Metrics pages with your own use case in mind. For a RAG system, faithfulness and response relevancy are the natural starting points; for an agent, tool call accuracy and agent goal accuracy.

Step 5: Use the how-to guides when you hit a specific need

The How-to Guides are task-oriented and are best consulted on demand rather than read front to back. Common entry points:

• Customise models and LLM Adapters — point Ragas at the model provider you already use.
• Run Config and Caching — control how evaluations execute and avoid repeating expensive calls.
• Modify Prompts and Adapt Metrics to Languages — adjust metric behavior, including for non-English applications.
• Testset Generation — generate evaluation data, including single-hop and multi-hop queries and non-English testsets.
• Prompt Evaluation and Optimization — iterate on prompts systematically rather than by trial and error.

Practical tips

• Evaluate a small sample first. A handful of examples is enough to confirm your dataset schema and metric choices are right before running a full evaluation.
• Match metrics to your architecture. Applying RAG retrieval metrics to a system with no retrieval step produces scores that do not mean anything.
• Treat scores as comparisons, not absolutes. The Experimentation concept exists because the useful signal is usually the change between two versions of your application.

The docs also include benchmarking guides (evaluating a new LLM, aligning an LLM as a judge, evaluating a text-to-SQL agent) and a RAG evaluation guide for evaluating and improving a RAG app — useful once the basics are working.