Top 5 myths about website personalization

The perception is you need lots of planning and effort to start personalizing your site, but in reality, it’s best to keep it simple to start.

There is broad agreement among marketers that website personalization drives higher conversion rates and more revenue than showing the same site to everyone. However, the majority of marketers have yet to fully realize the potential of personalization. We regularly see misperceptions about personalization from some marketers, even while others are enjoying great gains. We’d like to shine some light on a few of the most common myths about personalization.


Myth 1: It will take me a long time to get started with website personalization

Personalization needn’t take a long time to put in place. For example, most of Intellimize’s customers began their first campaign within days of becoming a customer.

The perception is often that you need lots of planning and effort, but in reality, it’s best to keep things simple to start. In fact, we recommend you start by identifying three elements of a landing page or homepage to personalize (ones that you believe will have an impact) and test three different versions of each element at the same time. This allows you to get up and running quickly and helps you identify the elements of the page are having the biggest impact on performance.

Real-world Example: Perkville, an all-in-one referral and rewards program to help businesses drive customer loyalty and grow revenue, went from ideation to setting their first campaign live in days. Within three months they had tested 39 different ideas and saw 70% lift across their send referral flow.


Myth 2: My audience segments have to be rigorously defined to use personalization

Predictive personalization does not require you to predefine segments, unless you want to. Unlike rules-based personalization, predictive personalization automatically discovers segments by observing how your ideas perform for different audiences. These AI-based systems explore all possible combinations of visitor attributes to discover which of your messages perform best for each. There is no need to pre-define segments and specify messaging for each segment (unless you optionally want to). Predictive personalization often delivers new insights about your audience.

Real-world Example: Chime, an online bank that helps its members achieve financial wellness, learned that people responded to headlines differently on different devices. The predictive personalization system automatically discovered this and adjusted traffic to reflect this insight. Chime did not need to set up any audience rules about device type.

Different Headlines Performed Best Based on Device Type

Best Performer: Desktop Best Performer: Mobile Best Performer: Tablet
44% lift over
existing headline
13% lift over
existing headline
623% lift over
existing headline


Myth 3: I need first-party data to begin personalizing my site

First party data can almost always help improve performance, but it’s not required. Contextual data such as geography, device type, time of day, first visit/repeat visit, day of week, and device type is available on every visit to your site. It’s easy to overlook the value of this contextual data because it seems so basic on the surface; however it’s common for these attributes to enable double digit lift for marketers.

Real-world example:

A client of ours recently tested pre-populating city and state information in online order forms to see if prepopulating improved conversion rates. It turns out that during the day pre-populating the information generated more lift. During the early evening, which are a busy time for this client, hiding these fields (and only asking for zip code) performed better. Most marketers would not have thought that there would be a correlation with time of day for this kind of experiment.


Myth 4: Website personalization requires technical and engineering expertise

This myth is partly true. Building your own personalization engine in-house is resource intensive and requires specialized expertise. Integrating your internal systems and programming machine learning algorithms can be time consuming.

On the other hand, some third-party solutions can be implemented by adding a single line of code to your site. With this approach, getting started with personalization can be as simple as providing creative assets like headlines, images, and body copy variations to a trusted partner. If you have bigger, more involved ideas, most agencies and some vendors will also provide coding support to bring those ideas to life.


Myth 5: Website personalization will create inconsistency in our messaging

Your messaging needn’t be monotonal to be consistent. Personalizing your site experience enables you to engage each of your customers in ways that are relevant to them individually. Each potential customer has different questions and considerations in their mind before making a conversion decision. Personalization enables you to take a consistent message and focus on the aspects of that message that are more impactful to each individual visitor. Most importantly, you define all of the variations a visitor could possibly see, so you have complete control over your messaging and how it will be delivered via personalization.

We see real success among conversion rate optimizers that use personalization. We hope we’ve demystified personalization some and helped you think about how you can apply use it to achieve your marketing goals.

How can marketers balance learning and performance?

Marketers who are able to strike an optimal balance between learning and performance gain an advantage over their competition. 

When marketers look to optimize conversion rates on their websites, they face a practical tradeoff. On one hand, there is a need to learn by testing new ideas to see whether they perform better than existing ideas. On the other hand, there’s a need to drive performance by applying the best of what’s been learned to deliver results. Statisticians and engineers will often refer to this tension as the explore-exploit tradeoff.

This tradeoff sometimes leaves marketers between a rock and a hard place because visitor traffic is finite. If you allocate more traffic to driving lift based on previous learnings, you are limiting the speed of learning on new ideas. If you allocate more traffic to learning about new ideas quickly, you miss some upside in performance. Marketers who are able to strike an optimal balance between these competing goals gain an advantage over their competition. These marketers work with and act on better information and use that learning to get better results.

A common approach marketers take to solving this problem is A/B testing. Marketers allocate a small amount of traffic toward learnings, testing one idea at a time, and then take action by applying what they’ve learned from their tests to all of their visitors. Unfortunately, this approach has three key weaknesses:

  • It’s slow. For the large majority of websites, A/B tests take a long time to yield results, in part because small portions of traffic are used for testing.
  • The loss in upside while testing can be large. A/B tests aren’t considered complete until they achieve statistical significance which can take weeks or longer. Throughout the testing period, the marketer is allocating 50% of their test traffic to one idea that is underperforming.
  • It’s static. After the tests are complete and the winning idea is getting all the traffic, the learning stops. The market might change, users might change, and the competitive space might change. However, to recognize and adapt to those changes, an entire new testing cycle would be needed.

So what should marketers do?

To balance driving conversions from your site now and delivering the best possible results, you need to do three things well:

  1. Automatically adjust exploration to find the right explore – exploit balance
  2. Handle changes to audience traffic and changes to variations at any time
  3. Personalize what is shown at the individual level

We think it is possible to do all three of these things well and in the right balance, and we believe one approach that does all three of these well is predictive personalization. These systems automatically optimize the balance between learning and performance while enabling you to test many ideas simultaneously. They continuously measure performance and reallocate traffic to the then-best ideas to deliver results. We believe predictive personalization systems are the way to make the right tradeoff for marketers.

Is AI a threat to marketers’ jobs?

Machine Learning Personalization

When you introduce AI to marketing, you need more creativity and insight, not less.

It seems like articles about artificial intelligence (AI) changing the world are in vogue, and it is easy to see how AI can be a disruptive force in many industries. Will AI replace us marketers, leaving us out of work? In short, I think not.

Artificial intelligence is becoming widely used in marketing and advertising. In some areas, AI is automating manual, rote work we have done before. For example, paper insertion orders (IOs) in paid advertising used to take a long time to process and are now effectively executed in milliseconds with programmatic ad buying. Marketers and their agencies remain as important as ever in paid advertising, now driving more sophisticated strategies more quickly.

Another area AI is impacting is the optimization of conversion rates for web sites. There are three broad functions required to successfully optimize the conversion rate of a web site:

  • Customer intimacy: Understanding prospects and existing customers. Internalizing a day in their life and how they could use our product or service
  • Ideation: The creative work of designing new messaging, new content, and new assets to connect our product or service to prospects’ day to day needs. The goal is to persuade our prospect to buy / use our product or service
  • Experiment management: The mechanical work of building experiments, observing prospects’ behavior, analyzing the resulting data, and implementing changes to our websites based on the results

Marketers and machines each excel in different parts of the process. Marketers are uniquely qualified to understand prospects and customers and to develop impactful ideas that influence and persuade those prospects and customers to buy.

Machines are exceptional observers, able to watch 24×7, ingest lots of data, and draw quantitative conclusions at a scale no marketer ever would. For example, one of Intellimize’s clients worked on a campaign that tested 4.5 billion possible versions of a single page. No marketer would ever monitor that many versions of a page, but a machine can.

So what impact will AI have on marketers? We will test more ideas, more quickly and learn more than we could have before. Successful marketers will learn to use AI as an effective tool and will drive more conversions while spending far less time on the mechanics and process of experiments.

We can instead invest more time in knowing our customers by visiting them, observing their behavior, and talking to them. We can focus more energy ideating, creating, and developing new ideas to persuade our prospects to take the actions we want of them. In short, we can do the things we, as marketers, are uniquely great at doing.

When you introduce AI to testing, you need more creativity and insight, not less.

As with other roles, the adoption of AI has implications for the skills marketers may be valued for in the future. Our unique ability to intimately understand our target audiences and to communicate with those prospects in a compelling and converting way will probably be worth more. The rote work of setting up and managing experiments will probably matter less. If there is a threat AI poses to marketers, it may be that AI will push our creative abilities and our need to directly connect with prospects – and perhaps push us out of our comfort zone.

How marketers can test, learn & grow quickly with low web traffic

Think about the bold ideas that have the potential to move the needle enough to make a bigger splash.

Most marketers don’t have the luxury of millions of pageviews on their sites each month. As a result, it can take weeks or months for an A/B test to reach statistical significance for these sites. Testing ideas often becomes time consuming and costly, driving marketers to create long backlogs of ideas they want to test and debating how to prioritize those lists. Worse, marketers may end up giving up on data-driven testing altogether, costing them potential conversions on their website.

What can you do to test, learn, and grow quickly on 1,000 pageviews a day? We have three suggestions:

Focus on incremental conversions

If you are only testing a handful of ideas every year with a typical site’s traffic, waiting for statistical significance may work for you. However, if your marketing strategy would benefit from dozens of tests a year, A/B testing will usually not yield results fast enough for you to meet your goals.

We suggest asking yourself why statistical significance is important. We believe most marketers want significance so that you can generate incremental conversions from your website and know the conversions aren’t due to chance.

If incremental conversions are the point, then we suggest focusing on them. More conversions means more revenue, customers, etc.

We further suggest keeping a holdback group, often also called the control group. This group will see your existing website rather than the tests you’re running, enabling you to measure apples-to-apples lift of your ideas over time.

Test your biggest ideas and spread your bets

If your daily pageviews are in the hundreds of thousands, a fraction-of-a-percent improvement can mean significant growth. However, if you’re working with smaller numbers, any individual small win has less value. For example, if you drive a 0.15% increase in conversion rate by changing a button color and your site has 2,000 visitors a day, that means 3 more conversions per day. For most companies that won’t move the needle.

So test your biggest ideas now. Think about the bold ideas that have the potential to move the needle enough to make a bigger splash. A bunch of small ideas can also add up to something big, and we suggest starting with the bigger ideas.

Then spread your bets. Instead of trying ten variations of a single page element (such as 10 different headlines), try a couple variations for five different elements on your page (such as two headlines, two call to action texts, two hero images, two layouts, and two abandon modal ideas). Your site visitors will tell you which elements to invest more in through their behavior. Once you see lift by changing one element of the page, create similar, derivative variations of those high performers to drive even more lift.

Use predictive personalization to automate and accelerate your testing

Predictive personalization enables marketers to test more ideas with less work. Marketers with low traffic often test many ideas at once and see results in days rather than months. This is possible because predictive personalization automates the process of allocating traffic to your best performing ideas.

One client working with Intellimize was able to test 39 ideas (which yield a total of 633K possible versions of the page) on less than 2,000 daily pageviews and saw a 70% lift in referrals overall, with a 27% lift in conversions within the first month alone. With a traditional A/B test this would have taken 11 months to reach statistical significance and they would have lost out on conversions while waiting for their tests to end.

Looking forward

Even without a large volume of traffic, marketers can use data driven approaches to optimize their conversion rate quickly. We suggest:

  1. Focusing on incremental conversions to optimize for what matters
  2. Testing big ideas to improve the likelihood that your tests will have a material impact
  3. Using predictive personalization to test more, learn what works more quickly, and automatically allocate more traffic to your best ideas

Instead of trying to fit your company to a testing model that doesn’t suit it, focus on these three ideas to drive better performance more quickly.

When should I end my A/B test?

Your site’s audience is not static.

It’s risky to assume the A/B test sample you run today will be representative of your audience tomorrow.

A/B tests take a lot of ongoing management from marketers. You need to monitor your experiment, ensure variations are working, and—perhaps most nerve wracking of all—decide when to call a winner and end the test. A lot has been written about ending A/B tests (like this article and this post), and there are many opinions about the best approach.

We believe this is a question you shouldn’t need to answer in the first place. Why? First, your audience isn’t static. The “right answer” for today may be wrong for tomorrow’s audience. Second, A/B tests rely on samples that may not be representative of your entire audience.

Most A/B tests aren’t representative because your audience is constantly changing

A/B tests are designed to make inferences about audience behaviors. They optimize for your site’s audience during the time of the test. However, your audience changes constantly and is influenced by your marketing efforts, your competitors, seasonal effects, and other random factors. An A/B test takes a snapshot of your audience without regard for how the behavior of your audience might change over time. At best, you answer the question “what was the best option then?”

A/B tests also rely on the assumption that the audience sampled during the test is representative of your audience outside the scope of the test. Taking a valid sample of your audience requires attention to detail and a degree of judgement. For example, while you can calculate the sample size required for a valid test, your sampling window should include at least a full business cycle (such as a full week) to ensure you include all types of behavior (such as weekday and weekend behavior). However, you don’t want your sampling period to be too long because it increases the likelihood of including nonrandom bias, like a change to your marketing campaign. Every sample has some risk of bias.

3 ways predictive personalization eliminates the sampling problem

  • First, predictive personalization ensures that ideas are tested continuously. The system reacts automatically to changes in your audience’s behavior over time. Predictive personalization will optimize for the best performance even when your audience and the optimal answer keep changing. You do not need to constantly monitor your experiment or regularly intervene, freeing you up to conduct more experiments, learn and iterate to accelerate performance improvements.
  • Second, your ideas are run with your entire audience instead of a sample. This approach minimizes sampling errors because you are continuously sampling all visitors to your site. A predictive personalization system optimizes for conversions rather than attaining statistical significance with a sample.
  • Finally, while testing ideas randomly is the right approach when you’re optimizing for statistical significance, predictive personalization will intentionally adjust traffic to your higher performing variations. Your performance will be driven by a real time view across your entire audience of which ideas are working better. Those better performing ideas will be shown more often, automatically.

Instead of asking when you should end your A/B test, ask yourself the question, “Is my audience really static?” If you’re not sure, or the answer is “no,” predictive personalization may help you achieve better results than A/B testing.

Why continuous web testing yields better and clearer results

The tests you ran three months ago aren’t measuring the right answer today. Predictive personalization gives you the peace of mind of knowing your site is being continuously optimized.

When was the last time you re-ran an A/B test for your web site after you achieved a statistically significant result? When was the last time you wanted to? I’m willing to bet you’ve never wanted to do that. A/B tests can be expensive and time consuming, on one hand. And on the other, the idea of repeating a test evokes an uneasy feeling that you may discover that the old results (and all the work you did with that info) aren’t working anymore.

When was the last time you re-ran an A/B test?

The problem with this widely used approach is that your audience makeup and behavior change regularly, and one A/B test does not capture that change over time.

A/B tests optimize for a particular moment by understanding how one idea affects your site’s conversion rate at that moment. When you change your paid marketing campaign, when visitor behavior changes, or when someone makes a change elsewhere on your site, the previous results may not be the best answer, and acting on those results may even cause harm. The tests you ran three months ago aren’t measuring the right answer today. If you re-ran the same A/B tests constantly to stay up to date, you’d be testing the same thing over and over, preventing you from testing other ideas.

Customer behavior changes over time. Websites should too.

Predictive personalization tests all of your ideas continuously, observing how your audience responds now, and making adjustments automatically. If your visitors’ behavior changes, predictive personalization will change their experiences accordingly.

Think about all the factors that influence your audience’s behavior on your mobile and desktop websites. It’s a long list that includes your marketing efforts, your competitors’ marketing efforts, seasonality, and even random, unpredictable occurrences.

Recently, one of our clients promoted their anniversary with a giveaway using social channels. This promotion created a significant spike in traffic and had a sharp effect on the audience makeup of their site. The homepage headline that performed best changed within hours, impacting their customer acquisition flow. Intellimize automatically detected this change in audience behavior and allocated more traffic to the new best performing variations. Within hours, the new top performing headline was receiving more traffic than any other variation. Two weeks later, as the promotional effect subsided, Intellimize automatically shifted traffic again based on a new best performers.

Without continuous testing, our client would have lost thousands of conversions. Even worse, they would not have been aware of what they were missing. They would have, in fact, cheered the large spike in traffic, unaware they were missing out on many new customers.

Nothing beats knowing

Even if the changes in your audience profile aren’t as dramatic as that example, predictive personalization gives you the peace of mind of knowing your site is being continuously optimized. Automating the testing, analysis, and rebalancing of ideas eliminates the challenges marketers face when they optimize for a point in time. Marketers no longer need to cross their fingers and hope that the right answer ‘back then’ remains the right answer ‘right now.’ Predictive personalization helps ensure that you won’t be taken by surprise when things change, which they always do.

Why predictive personalization is replacing A/B testing

Website Personalization

Personalization correctly applied always outperforms “one size fits all” approaches.

Performance marketers today face a conundrum. On the one hand, you’re tasked with creating marketing campaigns that appeal to wide—and sometimes very different—groups of prospects. On the other, those campaigns and efforts must deliver results. Why is it a conundrum? Because a single strategy, no matter how refined and researched, cannot optimally appeal to all of your visitors.

Performance marketers have turned to A/B testing to identify the single best page for all visitors. While A/B testing has helped marketers better quantify the impact of their ideas, trying to find the single “one size fits all” best page to show all of your visitors leaves money on the table and wastes time and effort.

Let’s explore an example to illustrate this. Say you are responsible for optimizing the conversion rate of a website and the current messaging that performs best (which we’ll call the baseline) converts 3% of your visitors. You decide you want to see if different messages can perform better, so you create two new test variations. In this example, you have three different, equally sized audience segments visiting your site: existing customers, prospective customers and competitors’ customers.

Conversion Rates for 3 Different Variations to 3 Audience Segments (with equal traffic) 

Baseline 1 2
Audience Segments A: Existing Customers 3% 2% 1%
B: Prospective Customers 2% 4% 1%
C: Competitors’ Customers 1% 3% 5%
Average conversion rate 2.0% 3.0% 2.3%

Pick one to show to all: 3% conversion rate overall

Personalize to each segment: 4% conversion rate overall (33% lift)

If you could only show a single variation to every site visitor, you would select Variation 1 because it delivers the best overall conversion rate across all customers (3%). However, if you could personalize the experience for each group and show each group the variation that performs best for them, you would show Baseline to existing customers (3%), Variation 1 to prospective customers (4%) and Variation 2 to competitors’ customers (5%). This personalized approach would result in a 4% conversion rate from the same ideas and the same group of visitors — a 33% improvement in performance.

No matter how individual variations perform personalization correctly applied always outperforms “one size fits all” approaches.

While rules based personalization is better than a “one size fits all” approach, you are required to set up static rules to deliver specific experiences to predefined segments of your audience. Predictive personalization uses machine learning to present the best experiences to each individual visitor to your site. Additionally, both A/B testing and rules based personalization optimize for a point in time, irrespective of how your visitors’ behavior changes or how your marketing efforts change in the future. Predictive personalization automatically adjusts to changes in visitor behavior over time, shifting traffic to your best performing experiences.

There are three advantages predictive personalization offers marketers over A/B testing:

Faster Results: Predictive personalization begins optimizing without waiting weeks or months for statistical significance, unlike A/B testing. Predictive personalization also allows you to test more experiences and variations at the same time than A/B testing so you can see results across more ideas sooner.

Better Results: Personalization outperforms “one size fits all” approaches by serving the best performing experiences to each user. Additionally, as visitor behavior changes over time, predictive personalization adjusts accordingly to deliver the best performing experience. A/B testing, on the other hand, picks a winner once and does not adjust again.

Less Work: A/B testing requires your ongoing attention, monitoring experiments, deciding when to call winners, and managing potentially large matrices of separate test cells. Predictive personalization automates experiment management and execution, freeing marketers to spend more of their time understanding prospects and developing new ideas to drive conversion.

In the year since we’ve launched our predictive personalization system, we’ve seen that these benefits are real. Perkville, a referral and rewards program increased customer referrals by 70% in its first few months. Chime, a bank focused on helping consumers lead healthier financial lives, saw a 79% lift in new account signups four weeks after introducing their second round of ideas.

Any performance marketer who has a frustratingly long backlog of ideas to test and wants to see results quickly should be investigating predictive personalization. A/B Testing, gave us a way to understand the best single experience for all visitors at one moment in time, and predictive personalization now gives us a way to deliver the best experiences for each and every visitor as visitor behavior changes.