Tests as configuration
Promptfoo describes test cases and expectations in configuration files that live in your repository and run from the command line.
SERVICES / EVALUATION PLATFORMS
Repeatable comparisons with Promptfoo.
Independent planning and implementation for teams considering or using Promptfoo: assertion-based tests and side-by-side prompt and model comparisons built into your development workflow.
Where Promptfoo fits
Side-by-side comparisons
The same tests run across multiple prompts, models or settings, showing results in a comparison matrix.
Deterministic and model-graded assertions
Exact checks such as JSON validity or required content sit alongside rubric-based, model-graded assertions.
How we help
Write assertions that reflect your rules
Turn business requirements into specific assertions, so a passing test means the behavior is actually right.
Build comparison suites
Create reviewed test sets for prompt changes and model selection, including edge cases and multi-turn scenarios.
Calibrate model-graded checks
Validate rubric assertions against expert labels before using them as release gates.
Make it routine
CI integration
Run agreed suites on pull requests or before releases, with critical assertions failing the build.
Results people can read
Summaries that highlight regressions and severe failures, not just pass counts.
Team enablement
We train engineers to add cases and maintain assertions as the product evolves.
Related services
Using the open-source CLI only?
If your team runs the open-source Promptfoo CLI without a commercial plan, see our open-source Promptfoo services for agent test suites and CI gates.
Explore serviceEvaluation Assessment
A baseline for one Promptfoo agent or workflow, with prioritized findings and a release recommendation.
Explore serviceHallucination Testing
Measure how often your Promptfoo agent invents facts, policies or actions, and which fixes reduce it.
Explore serviceFrom scope to handover
Start with your inputs
Representative tasks, expected outcomes, repository access, model access and a domain reviewer.
Keep reusable assets
Versioned test configurations, assertions, a baseline comparison, prioritized findings and a documented rerun process.
Support after handover
We can retest fixes, extend coverage to new Promptfoo workflows or move to recurring Managed Evaluations as your agent changes. Platform usage and licenses stay in your account.
Make your next decision with evidence.
Start with one workflow, clear success criteria, and an evaluation plan your team can use.