A/B testing with Mopinion

CRO specialists and Growth Marketers put heavy emphasis on experimentation and A/B testing. A/B testing, or split testing, is a way to test two versions of content and compare their performance. The goal of an A/B test is often to see which version leads to more conversions and a higher CTR. Qualitative data often doesn’t get as much attention as the raw numbers. For example, a specific A/B test can lead to more conversions or clicks on a certain button, but the users’ experience or sentiment might not be better. The click on the button might have happened because you weren’t clear about what would happen if your visitor clicked on the button.
That is why we believe it important to collect feedback for your A/B tests and capture qualitative data as well.

In this article, we will show two practical ways you can conduct A/B tests with Mopinion.

1. Using feedback forms to measure how users experience two different versions of an A/B test.
2. Testing two different feedback forms to measure which one has a higher response rate.

Online customer feedback can also help you form hypotheses that are not just based on assumptions. Based on the feedback you collect, you can identify problems your customers have on your online channels. You can use this information as input to test different hypotheses. But in this article, we will focus on the two points above.

No time to read the entire article? Watch the video where I (Growth Marketer at Mopinion) explain how you can A/B test with Mopinion.

What is A/B testing?

A/B testing lets you test two versions of content to see which one performs better. You serve 50% of your audience one version of the content, variant A, while you serve the other 50% the other version of the content, variant B. Distribution should be random.

Next, you measure the conversion rate, click through rate or customer experience to see which version of your test is performing better. With a large enough sample size, you will get a statistically significant result and know which version leads to the intended goal and thus performs better. That will be the version you will actually implement and bring your desired improvement.

What can you A/B test?

A/B tests are often done on digital channels such as websites pages, in-app, and in emails. On (landing) pages on your websites or in-app, you can test many variables, such as:

• Titles and subtitles
• Body copy
• Colours
• Content type (video, image, text, as seen in image below)
• Lay-out
• Design
• Call-to-actions (CTA’s)
• Navigation
• Feedback forms

With A/B testing in emails, you can test the subject line, lay out, length of the email and even the time of day you send out the email.

A/B testing is not limited to your own channels. You can also test Social Ads and Search Ads to see which ones drive the most traffic to your website and which ones are most likely to convert visitors. Here you can also play around with colours, design and CTAs.

Feedback forms are also potentials for A/B testing. Wouldn’t you want to know what kind of feedback form gives you the most insights? Within a feedback form, there are again many things to test such as lay-out, copy, design, question-order, type of question, type of feedback form, how and when it’s triggered and much more. Changing one of the variables might have a positive impact on the response rate of your feedback forms.

Why should you A/B test (with Mopinion)?

With A/B tests, you’ll be able to make meaningful and data-driven changes to your website, app, email or other digital channel that actually lead to your intended improvements. This can be anywhere between higher ROI, more conversions, better customer experience and more. Testing takes out the guesswork and assumptions.

Now let’s focus on why you should start A/B testing with Mopinion specifically. We’ll focus on the two ways to A/B with Mopinion as stated above.

1. Using feedback forms to measure how users experience two different versions of an A/B test.

Measuring how your audience experiences the two versions of your A/B test will give you qualitative data. It provides you with insight on the sentiment and opinion of your audience. For example, you’re doing an A/B test in your email with different types of content. Your audience can give a rating and leave an open comment on a feedback form in your email. Your A/B test can lead to different ratings of the two versions. This impacts your CSAT, NPS, or whatever else you’re measuring and provides you with insights into the sentiment of your audience. The open comments will tell you more about why a certain rating is given and why there could be a difference for your two versions. The open comments also inform you on the general opinion of your audience. Next to quantitative goals based on CTR and (sales) conversions, you will know what your audience thinks of the two versions of your email.

Although it’s not common to have goals that are not directly related to raw sales numbers, it could be that your focus is on creating a better user experience. An improved experience can and often does positively impact sales metrics. We also encourage you to collect feedback in that case as well, take a look at the following example.

Let’s say you want to improve the user experience on a specific website page. You want to measure time on page, number of pages visited or scroll depth. Your experiment variant (B) leads to more pages visited and longer time on the page, which could indicate a better website experience. However, it can also be a sign that visitors couldn’t find the information they were looking for so they needed more time or visited more pages to find it. You will need to collect qualitative data to know whether your change had an actual positive impact on the experience.

2. Testing two different feedback forms to measure which one has a higher response rate.

For the second way to A/B test with Mopinion – when you conduct A/B tests with your feedback forms, such as question order or form design – you will be able to determine which feedback form gives you the most responses or open comments and thus the most insights.

Two ways of A/B testing with Mopinion

1. Using feedback forms to measure how users experience two different versions of an A/B test

Example:
You’re doing an A/B test on a landing page on your website. You’ve changed the lay-out of your page as seen in the image above. You’re measuring Customer Satisfaction and want to know if the average rating per variant is different. After conducting the test, your original variant (A) has an average rating of 3/5, while the experiment variant (B) has an average rating of 5/5. With a large enough sample size and a significant result, you’re able to say that the difference in rating is due to the changes you made in your B variant. It’s a clear indication that your visitors are more satisfied with the experiment variant and you should implement the changes on the page.

How to do this with Mopinion?

The implementation of a feedback form on your A/B test isn’t too difficult, you only need 1 form. You can keep the original form if you already had one live on the page you’re testing, or you make a new feedback form and put it live on the page. You do have to make sure it’s available to 100% of your audience.

All you need to do then is to put a data element in your feedback form. The data element needs to be able to indicate which variant the feedback was given in – A or B. See the image below of how this will look like in your feedback form*.


Mopinion feedback form design – A/B test data element

The next image shows you how it will look like in the feedback item in the inbox. Every feedback that is given, will send the variant through to Mopinion and show up in the individual feedback items in the inbox. This way you know which feedback item belongs to which test variant.


Mopinion feedback inbox – A/B Test variant visibility

You can easily track your experiments in your dashboard. You can do this by building a chart and filtering the rating based on the variants (with the data element). See the screenshot below of what it looks like in the dashboard. You can see that the average rating for the original variant A is 3, while the rating for the experiment variant B is 5. You can also see whether this changes over time or per audience segment when building your chart, there are endless possibilities.


Mopinion graph – A/B test average rating per variant

When you see a difference in your dashboard per condition, you should dive deeper into the feedback items per variant and read the open comments. Perhaps you will find a reason for the difference in rating there and get a taste of the sentiment and thoughts of your audience.

In our knowledge base, you can find a step-by-step guide on how to conduct an A/B test with Mopinion. We have written a guide for three different A/B testing tools: Google Optimize, Optimizely, and AB Tasty. Of course our support team is here to help if you’re using another A/B testing tool and you want to collect feedback in your A/B tests.

* The screenshot with the data element is based on Google Optimize

2. Testing two different feedback forms to measure which one has a higher response rate.

Example:
You want to test two different feedback forms on a webpage and see which one gets more responses from your audience. You want to know whether your conversational feedback form gets more responses than your classic Mopinion form. After conducting the test, you have 75 responses on your conversational form and 50 responses on your classic form, while the forms are opened just as often. With enough responses and enough views on the feedback form, you can say with a certain amount of confidence that the Conversational feedback form gets more responses and thus gives you more insights.

How to do this with Mopinion?

It’s quite easy to do this with Mopinion, as you can deploy two different forms on the same web or app page or email. You will need to show your forms to the same number of people for a fair test. Each variant will be shown to half or your audience so the division should be 50/50. The first form (variant A) is shown to 50% of the visitors. The other form (variant B) is shown to 100% of the visitors who did not see the first form. Also, a 1 second delay helps to ensure that both forms won’t be shown at the same time.


Mopinion deployment – Two forms on the same page to A/B test

Once feedback starts coming in, you can compare the feedback between the two conditions, such as the response rate, rating (e.g. CSAT, NPS) and of course the open comments. You can easily keep track of your experiment by adding both forms in the same graph and comparing the number of responses.

Let’s take you through a real-life example of an A/B test that I’ve done with Mopinion forms.

How Mopinion uses Mopinion to conduct AB tests

In October 2022, I tested the passive feedback form on our homepage, a conversational feedback form versus a classic feedback form. The goal of my A/B test was to understand which feedback form gets more responses and which one gets a higher rating. My hypothesis was that Conversational Feedback forms will get more responses and a higher rating than classic feedback forms.


Mopinion dashboard – Feedback form A/B test responses


Mopinion dashboard – Feedback form A/B test average rating

Here you can see the results of my experiment and how it looks in my personal dashboard. My first hypothesis was supported (not significantly**) as the conversational feedback form received 11 more responses than my classic form. My second hypothesis was not supported (again not significantly) as the classic form had an average rating of 3.8 which is 0.2 points higher than my conversational form.

By conducting A/B tests with your feedback forms, you learn which feedback form gives you the most responses and most insights.

If you want to start testing different Mopinion feedback forms yourself, head over to our knowledge base where you’ll find our step-to-step guide.

** The test wasn’t done long enough to get a significant effect, but with more feedback items you might be able to get a significant effect

Start A/B testing with the help of Mopinion

To conclude, Mopinion is a great help for your A/B tests. It helps you get qualitative data on A/B tests and get an understanding of the sentiment and experience of your audience. It enriches the quantitative data of your tests with qualitative data. On top of that, it helps you understand what kind of feedback form will get you the most responses and insights.

We wish you the best of luck for your A/B tests!