Agents on your lakehouse
Agents built with Agent Bricks or the agent framework that answer questions and take actions over data governed in Unity Catalog.
SERVICES / DATA PLATFORMS
Evaluate Databricks agents with judges you can trust.
Independent testing and evaluation for AI agents built on Databricks with Agent Bricks and MLflow: retrieval, tool calls, data answers and LLM judges calibrated to your experts.
Common Databricks agent patterns
Retrieval over your documents
Agents grounded in vector search indexes. Index freshness, chunking and retrieval quality directly shape answer quality.
Data and SQL answers
Agents that query tables through Genie or Unity Catalog functions. A confident number from a wrong join is still a wrong answer.
Quality issues we look for
Retrieval misses
Relevant documents not retrieved, stale indexes preferred, or answers that blend retrieved facts with unsupported claims.
Wrong data, right format
Queries with incorrect joins, filters or metric definitions that return plausible but wrong figures.
Judges that disagree with experts
LLM judges that pass answers your subject-matter experts would fail, or the reverse, giving false confidence in a release.
What we assess
End-to-end task completion
Whether the agent reaches the intended outcome across your workflows, tools and data, verified against expected answers and results.
Retrieval and grounding
Whether retrieved context is relevant and sufficient, and whether answers stay faithful to it, scored claim by claim.
Quality, latency and cost
Model, prompt and tool changes compared on the same evaluation dataset, with latency and cost per successful task alongside quality.
Working with native Databricks evaluation
MLflow evaluation
Where they fit, we use MLflow's GenAI evaluation, built-in judges and custom guidelines judges, run on evaluation datasets reviewed with your team.
Expert review
We use the Review App to collect labels from your subject-matter experts and use those labels to calibrate automated judges.
Traces as evidence
MLflow traces show each retrieval, tool call and model step, so findings point to the exact component that failed.
Related services
Evaluation Assessment
A baseline for one Databricks agent or workflow, with prioritized findings and a release recommendation.
Explore serviceHallucination Testing
Measure how often your Databricks agent invents facts, policies or actions, and which fixes reduce it.
Explore serviceManaged Evaluations
Recurring tests as your Databricks agent, prompts, models and data change, with a monthly quality report.
Explore servicePlan the engagement
Your inputs
A workspace with agreed access, agent and model versions, evaluation data or approved samples, business rules and a domain reviewer.
Your deliverables
An agreed benchmark, calibrated judges, prioritized findings, reusable evaluation datasets and recommendations for the next iteration.
Support after handover
We can retest fixes, extend coverage to new Databricks workflows or move to recurring Managed Evaluations as your agent changes. Platform usage and licenses stay in your account.
Make your next decision with evidence.
Start with one workflow, clear success criteria, and an evaluation plan your team can use.