Test Sets allow you to monitor improvements between iterations of model configurations (new models, parameter changes, etc). We encourage building at least one “golden” Test Set of correct examples that Maitai can use to automatically test newly fine-tuned models. You can also execute Test Runs yourself against these Test Sets to validate incremental changes you’ve made.

​
Creating a Test Set

  1. Navigate to the Overview page for your Intent in the Portal
  2. Click Test Sets in the upper right
Test Sets Navigation
  1. On the Test Sets page, you’ll see your existing test sets for this Intent. Click + New Test Set in the upper right corner
New Test Set Button
  1. Define your Test Set:
    • Provide a name and description
    • Review the full list of requests belonging to the Intent (shown in reverse chronological order)
    • Select requests to include by hovering over ‘View’ to review message content
    • Click Create Test Set in the lower right corner
Select Requests
Don’t worry about missing requests - you can always add more requests to a Test Set after it’s created.

​
Adding Requests to Existing Test Sets

You can add requests to Test Sets in two ways:

​
From the Session Overview Modal

  1. Find your desired request and click the + icon in the upper right (near View Request)
  2. In the context menu, you’ll see:
    • Available test sets for this intent
    • Which test sets the request already belongs to
    • Option to create a new test set
Add to Test Set Modal
  1. Select your target Test Set
  2. Add any optional tags
  3. Click submit

​
From the Request Overview Page

  1. Click the + icon in the upper right
  2. Follow the same steps as the Session Overview modal:
    • Choose your Test Set
    • Add optional tags
    • Submit
Add to Test Set Request Overview
Creating a new Test Set directly from the request can sometimes be easier than going through the formal Test Set creation flow.