There is no finish line in website experimentation. And that's a good thing. Our customers' needs, wants, and preferences are always evolving. It’s our job to keep up and consistently deliver relevant experiences that compel them to convert. To do so, we must approach website experimentation as an ongoing, iterative process. How we execute is key.
Keep reading to learn how to run a website experiment in five clear steps.
1. Research
Research is the foundation of the experiments you run, but oftentimes the research process itself is not well thought out. When you're not properly conducting research, you won't have insight into user behavior and will be running your experiments purely based on assumptions rather than data. Not to mention, you won't be able to identify and solve for problems or friction points on your website that are stopping visitors from converting.
To ensure you have a solid research process in place, leverage both qualitative data (e.g. customer surveys, direct interviews, user testing) and quantitative data (e.g. analytics, previous experiments) that provides a full picture of the customer journey.
2. Hypothesize and Prioritize
Website experimentation is a powerful way to help your business drive more conversions and revenue. At the same time, it should focus on finding a solution to a problem so you can optimize the customer experience. In order to do this, you need a strong, research-backed hypothesis for each and every experiment you run.
After you've defined your hypothesis (aka your problem statement, inputs, and expected outcome), you need to determine how this particular experiment will fit into your current plans using your chosen prioritization framework. This is a critical step, yet about one third of experimenters don’t have a process in place to prioritize their experiments. Because time is our most valuable resource, we need prioritization to ensure that it's spent on the experiments with the highest probability of impact. So, if you determine that your experiment will have the biggest impact out of all your ideas, you're ready to start building it.
3. Configure, Build, and QA the Experiment
Before you can build an experiment, you first need to figure out if it's even worth running using standards such as statistical significance and minimum detectable effect (MDE) calculations. For example, if you find that an experiment will take months to reach stat sig or will achieve little-to-no lift, it should be deprioritized.
When you do determine that an experiment is worth running, you're ready to turn it over to your engineering resource to build and QA it. The worst thing you can do is spend days building an experiment only to have it break post-launch or disrupt something on your website.
4. Launch and Monitor the Experiment
Once your experiment passes QA, you’re ready to launch it in your optimization tool. While you’ll want to monitor your experiment to ensure it's not disrupting the customer experience and your analytics data is flowing properly, it's important to let it run its expected duration so you can reach a large enough sample size to declare a real winner.
5. Analyze Results
No matter how much work you put into the first four steps, an experiment is meaningless without proper analysis. A thorough post-experiment analysis is crucial for unveiling insights about your audience and their behavior and, of course, iterating on your experimentation efforts.
To gain in-depth learnings, don't just look at one top-level metric (e.g. conversions) in your analysis—look at other relevant metrics like micro-conversions and customer lifetime value (CLTV) to better understand an experiment's impact on customer behavior. Then, apply these learnings to either iterate on the same experiment or reprioritize your existing experimentation plan. Treating experimentation as a cyclical process will set you up for success by allowing you to continually resonate with your audience in the moment.
When you take an interactive approach to website experimentation using these five steps you can continuously improve the customer experience and, in turn, drive better results for your business.
Get a summarized version of our five step website experimentation strategy in our nifty infographic, 5 Steps to Effectively Execute Your Website Experiments.
