AI evaluation & QA
Every prompt, model and release, measured before it ships.
For teams that build their own AI. We design the test set and the rubric, score every change, run safety checks, and put a gate in your pipeline. You keep the release decision. You get the evidence.
AI features fail quietly.
- A prompt tweak breaks answers that worked last month.
- A model upgrade changes behaviour overnight, on the provider’s schedule, not yours.
- Nobody owns evaluation, so releases ship on a handful of manual checks.
Ordinary QA was not built for software that can answer differently every time. This is the gap we fill.
What we test
Each check is written as a test case with an expected result, so that a score means something.
Quality and groundedness
Is the answer correct, and does it come from the source it cites?
Safety and tone
Does it refuse what it should, and speak the way your brand does?
Robustness
Prompt injection, jailbreaks and data leakage, tested in an agreed scope.
Regression
Does last month’s good answer survive this week’s prompt change or model upgrade?
Agents and tools
Does the agent stay within its permissions, and does it stop when it should?
Arabic and bilingual
Dialects, Arabic-English mixing and tone, checked by native speakers.
Start free. Commit after you have seen the work.
Three steps, each one optional until you have seen the previous one.
01Free AI Quick Check
We use your public AI feature as an ordinary user and send a private one-page memo with three to five findings. No call needed.
02AI Release Readiness Audit
Ten business days, one AI feature, fixed price. You get a risk map, a rubric, a test set, baseline scores, robustness checks, a release-gate design and a 30-day fix plan.
03Monthly release assurance
Every prompt, model or retrieval change gets an evaluation run within two business days. A monthly safety sweep, a monthly report and a quarterly review. After the first three months, cancel with 30 days’ notice.
What the audit delivers
Seven deliverables in ten business days, ending with a readout call.
Risk map
The feature’s user journeys, failure modes and business impact, ranked.
Evaluation rubric
Quality, groundedness, safety and tone criteria, agreed with you.
Test set, version 1
100 to 200 cases, including edge cases and adversarial inputs.
Baseline scores
Where the feature stands today, with evidence for each failure mode.
Robustness checks
Prompt-injection, jailbreak and data-leakage tests within an agreed scope.
Release-gate design
How the evaluations run in your CI on every prompt, model or retrieval change.
Action plan
The top ten issues and a 30-day fix plan, presented in a readout call.
Evidence you can act on
- A score for every release, before it reaches customers.
- Gates in your CI that stop regressions automatically.
- A test set and a rubric that your team owns.
- A monthly report your leadership can read in five minutes.
- Clear answers when enterprise buyers ask how you test your AI.
Free
AI Quick Check
Send us a link to your public AI feature. We use it the way an ordinary user would, and send you a private one-page memo with three to five findings.
Include the link, what the feature is meant to do, and, if you like, what worries you most about it.
Request a Quick Check- No call needed, and no obligation.
- We use your product only as an ordinary user would: no jailbreaks, prompt injection or data extraction without your written permission.
- We never publish what we find.
- We take on a few each week, so there may be a short wait.
Questions we get asked
Do you need our code or production data?
No. We work in a test environment, with accounts you provide. An NDA and a data-processing agreement are standard.
Which evaluation tools do you use?
Your existing platform, or open-source frameworks such as promptfoo, DeepEval and Ragas. Gates run in GitHub Actions or GitLab CI. You own the suite.
What happens when we switch models?
That is when the suite pays off. We run it against the new model and show you what changed before you switch.
Can you evaluate agents, not only chat features?
Yes. For agents we also test permissions, tool use, stopping behaviour and what gets logged.
Do you also test the rest of the product?
Yes. Software QA covers regression, automation and release gates for the product around the AI, under the same monthly report.
Software QADo you guarantee bug-free releases?
No one honestly can. We give you evidence and a go or no-go recommendation. The release decision stays yours.
Find out how your AI holds up. Free.
Send us a link to your AI feature and get a private one-page memo. No call needed.
Founding clients get a lower price on their first project.