Skip to main content
Test Sets are how you curate a stable list of real requests so you can repeatedly benchmark changes (configuration changes, prompt changes, and models) in a consistent way. In the Portal, Test Sets are organized by Intent Group. If you want the full reference docs for Test Sets, start here:

​
Where to find Test Sets

You can access Test Sets from two places:
  • Global view: Test > Test Sets (shows all Test Sets across Intent Groups)
  • Intent Group view: open an Intent Group and use its Test Sets tab

​
Create a Test Set (wizard)

  1. Click New Test Set.
  2. Select an Intent Group (a Test Set belongs to exactly one Intent Group).
  3. Add a name (required) and optional description.
  4. Select requests from the Eligible Requests table.
    • Use View to inspect a request before selecting.
    • Add tags to help you filter and analyze later.
  5. Review and optionally mark the Test Set as Golden.
Select Requests
Don’t worry about perfect coverage on day one — you can add more requests as you discover new edge cases.

​
Add a request to a Test Set while you’re debugging

The fastest way to build a Test Set is to “promote” interesting requests you’ve already observed.

​
From a Request page

  1. Open a request (for example from an Intent Group’s Requests tab or from Test > Evaluations).
  2. Click Add to Test Set.
  3. Pick an existing Test Set (or create a new one inline).
  4. Add tags and submit.
Add to Test Set Request Overview

​
From a Session view

  1. Find the request row you care about.
  2. Click the + action to open the Add to Test Set menu.
  3. Pick a Test Set, add tags, submit.
Add to Test Set Modal
In the Add-to-Test-Set menu, the Portal enforces a maximum of 5 tags for that flow. If you need richer tagging, you can tag in the Test Set request tables after the request is added.

​
Correct the “expected” response (when the original response is wrong)

If the model response in production is wrong (but you know what it should have said), update the Test Set’s expected response by editing the request’s final assistant message. You can do this either:
  • Before adding (from the Eligible Requests table during Test Set creation), or
  • After adding (from the Test Set’s request tables, using the request edit action)
This is described in detail in Test Set Creation.