Create Test Sets

Test Sets allow you to monitor improvements between iterations of model configurations (new models, parameter changes, etc). We encourage building at least one “golden” Test Set of correct examples that Maitai can use to automatically test newly fine-tuned models.

You can also execute Test Runs yourself against these Test Sets to validate incremental changes you’ve made.

​

Creating a Test Set

1. Navigate to the Overview page for your Intent in the Portal
2. Click Test Sets in the upper right

3. On the Test Sets page, you’ll see your existing test sets for this Intent. Click + New Test Set in the upper right corner

4. Define your Test Set:
   • Provide a name and description
   • Review the full list of requests belonging to the Intent (shown in reverse chronological order)
   • Select requests to include by hovering over ‘View’ to review message content
   • Click Create Test Set in the lower right corner

​

Adding Requests to Existing Test Sets

You can add requests to Test Sets in two ways:

​

From the Session Overview Modal

1. Find your desired request and click the + icon in the upper right (near View Request)
2. In the context menu, you’ll see:
   • Available test sets for this intent
   • Which test sets the request already belongs to
   • Option to create a new test set

3. Select your target Test Set
4. Add any optional tags
5. Click submit

​

From the Request Overview Page

1. Click the + icon in the upper right
2. Follow the same steps as the Session Overview modal:
   • Choose your Test Set
   • Add optional tags
   • Submit