BetaTest and improve your customer agent with batch testing

With batch testing, admins can test their customer agent at scale before deploying it to live channels. Create your own test questions or generate them from existing customer conversations. After running a test, review the customer agent's responses to evaluate their quality, identify improvement areas, and coach the agent.

Before you get started

Please note: if you're a Super Admin, you can opt your account into the Customer Agent Batch Testing beta.

Run a new test

Please note: you can create up to 8 test sets. 

1. In your HubSpot account, click More, then navigate to Service > Customer Agent. If More doesn't appear in your account, navigate to Service > Customer Agent directly.

2. In the left sidebar menu, click Test > Batch testing. 
3. To generate questions from live conversations: 
   
   • In the top right of the table, click + New test set.
   • Select From live conversations.
   • In the dialog box, select the checkbox next to each theme that you want to test.
   • Click Generate questions. A list of commonly asked questions is generated for each selected theme.
     
     

     Please note: 5 or more conversations on a similar topic are required for a theme to appear.

     
     
     
     

3. • To add a question to the test set, click Add a question.
4. To create your own test questions: 
   
   • In the top right of the table, click + New test set.
   • Select Blank test.
   • In the Question field, enter your question.
   • Click Save.
     
5. If you've opted into the Customer Agent Segments beta, you can test a specific segment. Click Add segment, then select a segment.
   
   
   
6. In the top right, click Run test. 

The customer agent will generate a response for every question in the test set. 

Review answers and coach the customer agent

After running a test, you can evaluate the agent's performance and adjust its configuration:

1. Under Not reviewed, click a question to preview the customer agent’s response.
   
   
   
2. Mark the response as Looks good or Looks bad. 
3. If you marked the response as Looks bad:
   • To provide specific feedback about how the customer agent could improve, click Write note.
   • To coach the agent, click Coach agent.
   • In the right panel, navigate between the Knowledge, Actions, Handoff, and Experience tabs to edit how the agent should respond or behave. For example, you can add a handoff trigger or create a short answer.

Run and review existing test sets

Test sets and their results are saved automatically. You can rerun tests at any time to monitor the quality of your customer agent.

1. In your HubSpot account, click More, then navigate to Service > Customer Agent. If More doesn't appear in your account, navigate to Service > Customer Agent directly.

2. In the left sidebar menu, click Test > Batch testing. 
3. To run an existing test set, click Run. 
4. To clone the test, click the three horizontal dots and select Duplicate. 
5. To delete the test, click the three horizontal dots and select Delete. 
   
   
   
6. To review test results, click the Test results tab.