A standard vocabulary for retrieval
Ragas metrics such as faithfulness, context precision and context recall are widely used to describe how well retrieval-backed answers hold up.
SERVICES / OPEN SOURCE
Retrieval and agent evaluations with open-source Ragas.
Independent design and implementation of evaluations with the open-source Ragas library: faithfulness and context metrics for retrieval-backed agents, plus tool call and goal accuracy for agent behavior.
Where open-source Ragas fits
Agent and tool metrics
Tool call accuracy, tool call F1, agent goal accuracy and topic adherence extend evaluation from answers to agent behavior.
Self-run and open source
Ragas is an open-source Python library under the Apache-2.0 license, run in your own environment with your choice of judge model.
Testing agents with Ragas
Is the answer supported?
Faithfulness checks each claim against retrieved context, and context metrics show whether retrieval found what the answer needed.
Did the agent use tools correctly?
Tool call metrics compare the agent's calls and arguments with expected calls, in order or as a set, depending on what your workflow requires.
Did it reach the goal?
Goal accuracy judges whether the conversation achieved what the user wanted, with or without a reference outcome.
How we help
Build reviewed test sets
Ragas can generate retrieval test questions from your documents. We review and correct them, and add expected tool calls and outcomes by hand.
Fit metrics to your rules
Exact tool-argument matching can be too strict or too loose. We decide where exact matching, rubric scoring or outcome checks give the right answer.
Calibrate the judges
LLM-based metrics depend on the judge model and prompt. We pin both and compare scores with expert labels before they inform decisions.
Make it practical
Version-aware setup
Ragas's interfaces have changed between releases. We pin versions and document how your evaluations are configured and rerun.
Combined with outcome checks
Metrics are paired with direct checks of the resulting system state, so a high score means the work actually happened.
Settings agreed with you
Library settings, including usage analytics, are configured to match your data-handling policies.
Related services
Evaluation Assessment
A baseline for one Ragas agent or workflow, with prioritized findings and a release recommendation.
Explore serviceHallucination Testing
Measure how often your Ragas agent invents facts, policies or actions, and which fixes reduce it.
Explore serviceManaged Evaluations
Recurring tests as your Ragas agent, prompts, models and data change, with a monthly quality report.
Explore serviceFrom scope to handover
Start with your inputs
Representative tasks, approved documents, expected outcomes, agent access and a domain reviewer.
Keep reusable assets
Reviewed test sets, metric configurations, calibration results, a baseline report, prioritized findings and a documented rerun process.
Support after handover
We can retest fixes, extend coverage to new Ragas workflows or move to recurring Managed Evaluations as your agent changes. Platform usage and licenses stay in your account.
Make your next decision with evidence.
Start with one workflow, clear success criteria, and an evaluation plan your team can use.