evalban

Changelog

What's new in Evalban.

New features and improvements, newest first. Subscribe with RSS.

Sample project, docs and a clearer audit log

  • New
    Sample projectNew workspaces can add a sample support-bot project in one click: issues in every column, a sprint, a model whose eval regressed, the bug Evalban filed for it, and a wiki page. Remove it any time from Settings.
  • New
    Developer docsA full guide to the Python SDK, command line, GitHub Action, REST API and feedback widget at evalban.com/docs.
  • Improved
    Audit log wordingSprint moves, pull requests and eval events now read in plain words, with links to the issue and pull request.

Projects, audit log, Google sign-in and pricing

  • New
    Multiple projectsEach workspace can have several projects, each with its own board, backlog, sprints, releases and issue keys. Switch from the sidebar.
  • New
    Audit logAdmins can see who changed what, filter it by kind or person, and download it as CSV.
  • New
    Continue with GoogleSign in with your Google account, alongside the email code.
  • New
    Pricing announcedFree for up to 3 members; Team will be $25 per member per month when paid plans open. Workspaces from early access stay free.
  • New
    Compare runsPick any two runs of a model to see score, cost and speed changes, slice by slice, and which examples broke or got fixed.
  • New
    Model timelineEvery run, version, promotion, rollback, baseline change, merge and release for a model, in one list by day.
  • New
    Feedback widgetTwo lines of code add 👍/👎 under your AI answers. A spike in 👎 files a bug.
  • New
    Weekly emailEvery Monday: what shipped, what regressed, and how each model's score and cost moved. Turn it off in your profile.
  • New
    Notifications, search and importA notification bell, Ctrl K search across issues, models, releases and pages, and CSV import from Jira or Linear.
  • New
    Wiki editorA toolbar, keyboard shortcuts, images by paste or drag, and edit messages in the page history.

Early access

  • New
    Evals on the boardBaselines, slices, budgets, verdicts, an Evaluating column, and bugs filed automatically with the failing examples.
  • New
    GitHub and SlackPull requests move issues and merges wait for evals; alerts go to a Slack channel.
  • New
    Python SDK and GitHub Actionpip install evalban and evalban-dev/eval-action@v1.
  • New
    Releases, reports and wikiRelease notes and readiness, velocity and quality reports, and a team wiki.