We know just as well as the next SaaS company how important it is to move with speed, agility and customer focus. That is why when Mopinion decided to launch its new user interface Mopinion Raspberry, we wanted to use every tool in our toolbelt to ensure a swift and effective release. This translated to a SaaS software testing strategy bolstered by user feedback.
In this article, we will address how SaaS companies can leverage user feedback in their software testing by sharing our own user testing journey prior to and during the beta release of Mopinion Raspberry.
And as for whether or not user feedback is the missing ingredient in SaaS software testing? We’ll let you decide after reading this article.
The Mopinion Raspberry Project
At the beginning of 2020, Mopinion beta launched its new user interface – Mopinion Raspberry. This next generation user interface not only simplifies user flows and makes it easier for users to reach their goals, but it also delivers a more consistent user experience.
As part of this release, the development team optimised five main aspects of the user interface: the overall functionality of the Navigation (i.e. menus and configuration), the process for creating surveys (the Form Builder), the Feedback Inbox used for browsing through feedback responses, setting up Dashboards and lastly, Data Exploration and trends; a rather large undertaking, which required rigorous testing.
With all hands on deck, beta testing for Mopinion Raspberry commenced in November 2019. Here is a first-hand look at how we leveraged user feedback to reinforce our beta testing efforts.
The initial approach: identifying the user journey
The easiest way to get started with both optimising and testing is to first identify the user journey and establish (if you haven’t already) a clearcut onboarding process for your software. For example, many SaaS companies operate using a certain flow that consists of tasks the customer needs to complete successfully in order to move on to the next step. Mopinion refers to these steps as ‘modules’ in its software.
Note: It’s important to mention that depending on certain factors such as the size of your company and the time frame you’re working with, a new software or product release can be released in one of the two different ways. You can develop and release an entire product all at once and start testing it altogether, or break the process down into phases, testing different modules as you go. The latter, which is typically the route most SaaS companies go, is the approach Mopinion followed as well.
Free White Paper: The future of Online Customer Feedback
Learn how you can leverage the power of online customer feedback to optimise websites and apps.
We divided the user journey up into different modules, starting with the Form Builder: the module used to create and customise user feedback forms. For our software, this is the first step in the onboarding process. After all, the customer needs to know how to create a feedback form before they can even think about deploying it or collecting any feedback data.
Once our plan of action was established, the next step was to get our feedback forms up and running within the platform.
Creating effective feedback forms for testing
When creating feedback forms for testing purposes, it’s important to identify which form elements you will need. This is especially true if you expect a high volume of responses (i.e. if you have a large development team or big batch of external beta testers) as you’ll need to find ways of – for example – categorising the incoming data so that it is easily digestible and filterable.
Here are a few examples of elements we used in our feedback forms:
Visual Feedback
Visual feedback is a particularly interesting element to incorporate into your feedback forms as this type of feedback gives you direct insights (via screenshots) into issues such as bug reports and design errors.
The screenshot (rendered via the software) is then submitted alongside other feedback and metadata such as the operation system (OS), browser and more, enabling the user to quickly recreate and resolve the issue.
Feedback Categories
Feedback categories are also an ideal element to use in feedback forms. Collecting categories enables users to filter through their feedback data and prioritise accordingly. For example, if you are working with a tight time frame, it’s much more effective to prioritise items such as bugs and content before you start looking into more minor tweaks.
During testing, Mopinion applied these categories to its feedback forms as well. We used the following categories: ‘Bug’, ‘Suggestions’, ‘Content’, ‘Compliment’ and ‘Other’.
Categorising our feedback helped us pick apart the most important items and gave us some focus in terms of what needed optimising first.
Anwar Jebali, Senior Web Developer and Front End Team Lead, Mopinion
(Internal) beta testing begins…
Once a strategy was mapped out and feedback forms were set, it was time to really start testing. To get multiple perspectives, we decided to break our internal testing panel down into three groups:
Group 1: Development & Support Teams
The first testing group consisted of the two teams within Mopinion who work most closely with the platform – the Development and Customer Support teams. We started with this group as these team members are very familiar with all of the ins and outs of the software from a technical perspective. This group tested each of the modules individually in a span of two weeks per module.
Having helped customers utilise every feature in every configuration, it was great to try and find combinations that might serve up an error. It was also exciting to see that features now have a more consistent interface and are exactly where you expect them to be. Not to mention, hearing all the compliments from customers who’ve already started using it as much as possible makes me proud to be part of the first tests.
Guus Linzel, Customer Success Team, Mopinion
Group 2: All other Departments
This second group was intended to give us the ‘end user testing effect’. We asked internal team members from Marketing and Sales to run through all the modules from an end user perspective. This means they had no prior experience with the new release and would be looking at it with a blank slate.
Being in a client facing role we have to demo the Mopinion platform capabilities quite frequently. This often means showcasing certain features in the simplest and quickest way possible. Having the opportunity to dummy run our usual demo presentation on Raspberry was exciting. Following the same demonstration format as our old platform enabled us to find quirks and workflows that were perhaps a little ‘clunky’ when tested in the real world.
Luke Price, Mopinion Sales Manager
Group 3: Volunteer beta testers
The third group (ongoing) consists of volunteer beta testers (customers only) who are keen to give us their input – via the feedback forms – on the flow and overall user experience of Mopinion Raspberry. We’ve asked this group to notify us of any small glitches or bugs they come across when carrying out certain actions such as building a feedback form, deploying a feedback form, writing up notes and sending email alerts.
In addition to these testers, we’ve also held on-site beta testing sessions, which have enabled us to observe how our customers click through the platform and immediately get feedback on which issues they’re running into.
Performing effective action management
All throughout testing, the Mopinion development team has been analysing incoming feedback data and continuously iterating in order to meet the quality standards that we set out for ourselves from the beginning.
Within the Mopinion software itself, the team systematically filters through feedback using a variety of tags. For example, during each phase of module testing, the team labels their feedback accordingly (i.e. Form Builder, Dashboard, Data Explorer, etc.).
Below is a selection of items that include issues from the Form Builder:
Additionally, we use custom filters for feedback categories:
The value? Using tagging and custom filters, we were able to quickly and easily prioritise our data. Once we addressed the issues we needed to fix, we employed project management tools such a JIRA to keep a record of all the actions we needed to run through.
The Beta Launch
Releasing a product in beta is a pivotal stage. This is the stage when actual users start trying out the software and reporting issues they come across and suggestions they might have. In other words, it’s when your little cocoon of a product transforms into a beautiful butterfly!
In the words of Reid Hoffman (founder of LinkedIn):
If you are not embarrassed by the first version of your product, you’ve launched too late.
At the end of January (just three months after testing began), Mopinion beta launched Mopinion Raspberry. We gave our users notice that they can switch back and forth between the classic version and the new user interface with the click of a button. This gave early adopters, who were keen to test Raspberry the opportunity to dive in using their own data.
Note: the user interface is what is called ‘backwards compatible’, meaning our system offers interoperability with the classic user interface. Users are not forced to switch over but they can test it out in a live environment.
During this period, it is important to collect as much feedback as possible so that you can continuously enhance functionality and quality leading up to the official launch of your product.
Insights and Learnings
All in all, there has been a lot to take away from the testing process. From testing panels to the beta launch, here are some of our learnings…
For one, we found that it was especially effective for us to test using different user types and levels of knowledge of the system. This ensures that various use cases are tested. For example, most bugs tend to occur in settings and/or configurations that the developer or development hasn’t thought of.
Anwar Jebali shares our insights,
Most of the feedback we received was centered around bugs. This feedback was very valuable in that it has helped us track down different parts that weren’t working properly.
There were, for example, plenty of bugs that affected the usability of the software such as frozen functionalities, unclickable buttons, sizing and formatting issues and many more.
We also received some particularly interesting feedback regarding the type views presented which we – as developers – hadn’t previously considered. For instance, one of our testers reported that he didn’t necessarily prefer having the name of the survey respondent highlighted, seeing as how they weren’t requesting contact details in their feedback. This was a really good insight that sparked a great idea. As a result, we plan to implement a way to customise certain parts of lists through list options.So for example, in the feedback inbox, a user would be able to select if they want to see a customer-centered feedback list or a list centered around category values.
We also received a number of interesting design insights. For example, one of our users reported feedback around the use of card views on the feedback forms page, sharing that they preferred having a display option that would enable their team to switch from card view to list view.
As you can see, by holding these test sessions with team members from other departments as well as external beta testers, we have gained a great deal of insight into the usability of the user interface.
Coming soon: the official launch
Keep in mind that the duration of a beta launch will vary from SaaS to SaaS depending on factors such as the size of the user base and the magnitude of the release. Either way, continuous feedback collection and optimisation should be a priority in this phase. Once this period is over, you’ have reached the final phase, the official launch.
Later this year, Mopinion will officially launch the new user interface for all customers. In the meantime we will continue iterating on the release and gearing it up for the launch.
Stay tuned for insights on how to test post launch!
Ready to see Mopinion in action?
Want to learn more about Mopinion’s all-in-1 user feedback platform? Don’t be shy and take our software for a spin! Do you prefer it a bit more personal? Just book a demo. One of our feedback pro’s will guide you through the software and answer any questions you may have.
