About Evalban
AI software can get worse without anyone touching the code.
A prompt tweak, a model upgrade or new data can quietly make answers worse, slower or more expensive. We build the place where AI teams plan their work and see, on the same board, whether that work made the model better.
Why we built it
Most teams shipping AI features plan in one tool, run evals in another, and find out about regressions from a screenshot in Slack, or from a customer. The ticket says Done. The model says otherwise.
Evalban closes that gap. Your eval scripts send their scores to the board. When a change makes quality, cost or speed slip below what you agreed on, Evalban files the bug with the failing examples attached, holds the issue out of Done, and tells the team.
"Done" should mean the work shipped and the model is still good. That's the whole idea.
What we believe
Quality is part of done.
An AI feature isn't finished because the code merged. It's finished when its evals pass. Evalban builds that into the workflow instead of leaving it to memory.
Show the evidence.
Every regression comes with the examples that failed, the version that caused it and the run that caught it. Less arguing about vibes, more fixing.
Fit the way teams already work.
Keep your eval code, your GitHub and your Slack. Two lines send results from Python or CI. We don't ask you to adopt a new framework.
Your data is yours.
We use what you send us to run Evalban for you, and nothing else. We don't train AI models on your content.
What's shipped so far
Evalban is new and growing in the open with early teams. Here's what works today.
- PlanningBacklog, sprints, a board with an Evaluating column, releases, reports, wiki, and several projects per workspace.
- Evals on the boardBaselines, slices, cost and latency budgets, run comparisons and a timeline of every change to each model.
- Automatic bugsRegressions file a bug with failing examples. Fixed failures that come back reopen it.
- IntegrationsGitHub pull requests move issues, merges are held until evals pass, and Slack hears about it.
- For developersThe evalban Python package, a GitHub Action, a REST API and a feedback widget for 👍/👎 from real users.
- For adminsRoles, guests, an audit log, Google sign-in and a weekly email summary.
Want to try it, or work with us?
Get started and we'll set up your workspace. Questions, ideas or partnerships: [email protected].