Creating a New Test
Overview
AB Testing allows you to make data-based decisions when you are considering making changes to a chatbot. This support page explains how to create A/B tests for chatbots.
Creating a new test
You can create a new bot test from two different places in Signals:
- From the Testing tab
- Directly from the bot builder
Note: tests created from the bot builder will automatically select the chatbot you're currently editing for the bot test.
Test Builder
- Select a bot
- Edit the bot
Pro Tip: This step creates the B variant that you will be testing. Take good notes of the changes made at this step, since this builder view will not be available to view or edit while the test is running.
- Select Success Criteria
Conversations: Track the conversion rate from "bot prompt" to "conversation." This is the best success metric to measure the effectiveness of the very first question in a chatbot.
Emails captured: Track the conversion rate from "conversation" to "email capture." Choose this option when testing the messaging in email capture skills or the placement of email capture in the chatbot flow.
Meetings booked: Track the conversion rate from "conversation" to "meeting booked." Choose this option when testing the messaging in calendar skills or the placement of calendars in the chatbot flow.
- Select Duration
Note: hover your mouse over the information bubbles for more information about each of these options.
- Set Schedule
Note: If you selected "Automatic" at Step 4, you only need to specify a start date.
- Promotion
Note: hover your mouse over the information bubbles for more information about each of these options.
- A brief description
Pro Tip: Include specific notes about changes made in Step 2.
- Summary
Click "finish" in the summary modal or "next" at the top right of the screen to create the test.
