This is a follow up post regarding my presentation from the Multivariate Testing and Conversion Tweaking panel at last week's PubCon in Las Vegas, where I presented alongside Vanessa Fox and Glenn Alsup. The deck from the presentation can be found on slideshare.
As a web designer, I wanted to share my thoughts on a shift in website design that I've seen referred to as "performance-based design" or "data-driven design." Essentially, this shift is about relying on user data and testing to drive design decisions because at the center of this movement is the idea that we can no longer assume that what we think as designers is best for users. This is an idea that I'm comfortable with as a designer who has become deeply entrenched in the online marketing world. The web is the most measurable and testable form of design in history and there are a number of reasons why now is the perfect time to take advantage of that fact.
The two biggest reasons are a down (although recovering) economy and free technology. As we slowly come out of the economic downturn, the lessons that businesses learned will guide their decisions for years to come. Getting "more for less" was how many companies survived and even excelled during these hard times. This involved finding and addressing inefficiencies, which is at the core of landing page testing and conversion tweaking. Free technologies such as Google Analytics and Website Optimizer, although not technically new, have created opportunities for even the smallest businesses to learn more about their visitors and test different approaches in their web design and content strategies.
I predict that in the coming year there will be noticeable effects of this shift towards performance-based design. We'll see a dramatic increase in investment in post-click optimization in order to drive greater ROI. With this will come a hunger for more data and with that, a need for new and meaningful ways to collect and analyze that data. I foresee the core focus of this new data collection and analysis to be social media.
Technology for Multivariate Testing
Multivariate testing is about finding the (sometimes very small) synergies that exist between numerous variables on your page that maximize customer throughput - those little things that when combined in the right way persuade visitors to convert. To do this, you need really complex statistical analysis. The good news is that technology has removed that complexity making it possible to focus on what to test, not how to test.
There are a variety of options and costs out there for managing multivariate testing. Many require some sort of JavaScript tagging to be embedded on your page, while others offer a full DNS-based solution that routes traffic to your site through a testing server first, allowing for fast and easy deployment of tests. Make sure to get your IT department involved. Often times the "yay" or "nay" decision on a technology comes down to them, so it’s best that they aren’t the last people you consult. And my advice is to go with a nimble solution, whatever that means for your specific situation. Success in testing is often the product of momentum and building on the successes of previous testing rounds. It is very difficult to gain that momentum when it takes 6 weeks for a page to be tagged or a new headline to be uploaded.
What should you test?
The simple answer to this is "everything." The problem with this is that if you were to test literally everything, you would find yourself in the middle of a giant mess. So, the challenge becomes knowing where to start.
To figure out where to start, I categorize a page into one of two categories:
How I categorize pages largely relies on intuition and experience, so this is an area where the data might not help.
If a page is a disaster, then no amount of tweaking is ever going to give you worthwhile results. These pages first need help to make them more sticky, so you need to consider concepts like first impressions, user experience, and general layouts. These are the types of things that we consider when we think like designers.
Since tweaking won’t work, I tend approach these projects by doing a radical redesign, or a completely new and different take on the subject. For this I might test different page layouts, fonts, and colors while maintaining a consistency in the messaging. Since I might be testing radically different looks, I like to use A/B testing at this stage.
Once the stickiness of a page is maximized, then you have to start thinking like a marketer. You’ve captured a visitor's attention and now they might actually pay attention to what you are saying! This is where "tweaking" comes into play. Now you have to test specific messaging to figure out what resonates with the user - what persuades them convert. What are the different messages you could convey through your headline or your hero shot? This is the perfect scenario for multivariate testing where you have individual elements throughout the page that each have multiple options.
Specific Elements to Tweak
I wanted to provide at least a bare minimum of what everyone should be testing (tweaking) on their landing pages. The elements listed below are common to most landing pages, but they also can have a big impact on the conversion potential of your page.
The last piece of this puzzle is figuring out what the variables should be for your test. Here, when I say "variables," I'm talking about the different options to test within the individual page elements, so what are the different headlines you should try or the different button texts.
This is where data on your site visitors and customers can be very useful. Data can fall into two categories: quantitative and qualitative.
Quantitative data is the information that numbers tell us. This can be information from your site analytics about the number of users with a certain language setting or the number of referrals from a certain keyword, and you could use that data to influence whether you offer translations of your content or what keywords you use in your headlines. Quantitative paid search metrics on ad copy testing can help you determine what messaging resonates well with users.
Qualitative data tells you about your customers in a way that has emotion and attitude and a sort of raw honestly. This type of data can come from customer feedback directly to your customer service department or by researching what your customers (and maybe more importantly detractors) are saying about you in social media. Personas (which were covered by Vanessa Fox on our panel) are another great source of qualitative data.
Caveat: There is an important caveat as it relates to data, and I find this to be true anytime you are dealing with data, not just online. Beware of making decisions based on aggregate or average data. Tools like Google Analytics make it easy to focus in on the averages, because they are right there at the top of the data columns in larger, pretty type. But consider a test that you’ve design to reduce the bounce rate on one of your landing pages. After running the test long enough, you are able to reduce the bounce rate by 25%. That's a success right? Well, yes and no. If you were to segment the data by the source of the traffic you may find that, overall, you reduced your bounce rate by 25%, but you increased the bounce rate on your display traffic by 50%! It's a smaller channel in terms of the overall traffic volume, so it gets lost in the aggregate data.
Likewise, you might run a multivariate test on one of your landing pages and find that conversions increased a modest 0.5%. That is probably not the amazing results you were looking for. However, if you take a closer look at the data, you might find that for a certain, small set of related niche keywords, the conversions skyrocketed, while conversions for the other keywords driving traffic to the landing page fell, resulting in the overall increase of only a half a percentage point. This gives you a valuable learning, which is: use the winning landing page for the small set of keywords it performed well on and then rerun your multivariate test without those keywords to pollute the testing process and see if a different combination of variables can increase conversions.
Segmenting Your Data for Meaningful Results
There are a myriad of ways to segment your data, but the 3 primary ones I focus on are:
Take-Aways
These are some of the key takeaways that I summarized at the end of my presentation:
Jason Cooper
Director, Value Added Services
As a web designer, I wanted to share my thoughts on a shift in website design that I've seen referred to as "performance-based design" or "data-driven design." Essentially, this shift is about relying on user data and testing to drive design decisions because at the center of this movement is the idea that we can no longer assume that what we think as designers is best for users. This is an idea that I'm comfortable with as a designer who has become deeply entrenched in the online marketing world. The web is the most measurable and testable form of design in history and there are a number of reasons why now is the perfect time to take advantage of that fact.
The two biggest reasons are a down (although recovering) economy and free technology. As we slowly come out of the economic downturn, the lessons that businesses learned will guide their decisions for years to come. Getting "more for less" was how many companies survived and even excelled during these hard times. This involved finding and addressing inefficiencies, which is at the core of landing page testing and conversion tweaking. Free technologies such as Google Analytics and Website Optimizer, although not technically new, have created opportunities for even the smallest businesses to learn more about their visitors and test different approaches in their web design and content strategies.
I predict that in the coming year there will be noticeable effects of this shift towards performance-based design. We'll see a dramatic increase in investment in post-click optimization in order to drive greater ROI. With this will come a hunger for more data and with that, a need for new and meaningful ways to collect and analyze that data. I foresee the core focus of this new data collection and analysis to be social media.
Technology for Multivariate Testing
Multivariate testing is about finding the (sometimes very small) synergies that exist between numerous variables on your page that maximize customer throughput - those little things that when combined in the right way persuade visitors to convert. To do this, you need really complex statistical analysis. The good news is that technology has removed that complexity making it possible to focus on what to test, not how to test.
There are a variety of options and costs out there for managing multivariate testing. Many require some sort of JavaScript tagging to be embedded on your page, while others offer a full DNS-based solution that routes traffic to your site through a testing server first, allowing for fast and easy deployment of tests. Make sure to get your IT department involved. Often times the "yay" or "nay" decision on a technology comes down to them, so it’s best that they aren’t the last people you consult. And my advice is to go with a nimble solution, whatever that means for your specific situation. Success in testing is often the product of momentum and building on the successes of previous testing rounds. It is very difficult to gain that momentum when it takes 6 weeks for a page to be tagged or a new headline to be uploaded.
What should you test?
The simple answer to this is "everything." The problem with this is that if you were to test literally everything, you would find yourself in the middle of a giant mess. So, the challenge becomes knowing where to start.
To figure out where to start, I categorize a page into one of two categories:
- It's a disaster and needs help with broader design based concepts
- It's not bad and could benefit from some tweaking or fine-tuning
How I categorize pages largely relies on intuition and experience, so this is an area where the data might not help.
If a page is a disaster, then no amount of tweaking is ever going to give you worthwhile results. These pages first need help to make them more sticky, so you need to consider concepts like first impressions, user experience, and general layouts. These are the types of things that we consider when we think like designers.
Since tweaking won’t work, I tend approach these projects by doing a radical redesign, or a completely new and different take on the subject. For this I might test different page layouts, fonts, and colors while maintaining a consistency in the messaging. Since I might be testing radically different looks, I like to use A/B testing at this stage.
Once the stickiness of a page is maximized, then you have to start thinking like a marketer. You’ve captured a visitor's attention and now they might actually pay attention to what you are saying! This is where "tweaking" comes into play. Now you have to test specific messaging to figure out what resonates with the user - what persuades them convert. What are the different messages you could convey through your headline or your hero shot? This is the perfect scenario for multivariate testing where you have individual elements throughout the page that each have multiple options.
Specific Elements to Tweak
I wanted to provide at least a bare minimum of what everyone should be testing (tweaking) on their landing pages. The elements listed below are common to most landing pages, but they also can have a big impact on the conversion potential of your page.
- Brand Presentation – Visitors need to trust your brand. So brand presentation deals with how you present your brand on the page, whether that’s through a simple logo placement or a short introductory paragraph explaining your value proposition. As a general rule, the more well-known a brand is, the less effort you have to put into the brand presentation.
- Call to Action – Every landing page needs one - it’s the action you want your visitors to take. Important things to test about your call to action are placement, emphasis and wording.
- Product Image – How you present your product can influence a visitor's perception of its value. A perfect example comes from digital software downloads. Pages that show a physical package (even if none exist and was just made up by the landing page designer) tend to convert better even though the user downloads the product and never receives a tangible product. The illusion of something tangible adds value in the user's mind.
- Hero Shot – The simple thought here is that people want to see themselves using the product or service they are considering purchasing.
- Buttons – Don't overlook or underestimate the influence of your buttons. If you have a button on your page, it is probably directly related to the call to action. I suggest never using standard browser buttons and instead opting to create visually stunning image buttons. Also, test the text that you include in your button (hint: it should never say "Submit"). Tell the user what they are going to get when they click on that button.
- Incentives – Offering the right incentive can make or break a page, especially in highly commoditized goods markets. Free shipping is almost always the best incentive you can offer.
- Headlines – If a user has made the decision to stick around on your page for longer than 2 seconds, then the headline is probably going to be the first thing that they read, so make sure that your headline is conveying an important message. To get an idea of how important a headline can be, check out this post about how 37signals improved conversions by 30% with simple headline testing.
The last piece of this puzzle is figuring out what the variables should be for your test. Here, when I say "variables," I'm talking about the different options to test within the individual page elements, so what are the different headlines you should try or the different button texts.
This is where data on your site visitors and customers can be very useful. Data can fall into two categories: quantitative and qualitative.
Quantitative data is the information that numbers tell us. This can be information from your site analytics about the number of users with a certain language setting or the number of referrals from a certain keyword, and you could use that data to influence whether you offer translations of your content or what keywords you use in your headlines. Quantitative paid search metrics on ad copy testing can help you determine what messaging resonates well with users.
Qualitative data tells you about your customers in a way that has emotion and attitude and a sort of raw honestly. This type of data can come from customer feedback directly to your customer service department or by researching what your customers (and maybe more importantly detractors) are saying about you in social media. Personas (which were covered by Vanessa Fox on our panel) are another great source of qualitative data.
Caveat: There is an important caveat as it relates to data, and I find this to be true anytime you are dealing with data, not just online. Beware of making decisions based on aggregate or average data. Tools like Google Analytics make it easy to focus in on the averages, because they are right there at the top of the data columns in larger, pretty type. But consider a test that you’ve design to reduce the bounce rate on one of your landing pages. After running the test long enough, you are able to reduce the bounce rate by 25%. That's a success right? Well, yes and no. If you were to segment the data by the source of the traffic you may find that, overall, you reduced your bounce rate by 25%, but you increased the bounce rate on your display traffic by 50%! It's a smaller channel in terms of the overall traffic volume, so it gets lost in the aggregate data.
Likewise, you might run a multivariate test on one of your landing pages and find that conversions increased a modest 0.5%. That is probably not the amazing results you were looking for. However, if you take a closer look at the data, you might find that for a certain, small set of related niche keywords, the conversions skyrocketed, while conversions for the other keywords driving traffic to the landing page fell, resulting in the overall increase of only a half a percentage point. This gives you a valuable learning, which is: use the winning landing page for the small set of keywords it performed well on and then rerun your multivariate test without those keywords to pollute the testing process and see if a different combination of variables can increase conversions.
Segmenting Your Data for Meaningful Results
There are a myriad of ways to segment your data, but the 3 primary ones I focus on are:
- Demographic – This is the type of information that is related to personas (age, gender) and those segments that you probably never use on Google Analytics (language setting, country of origin, connection speed, etc.)
- Psychographic – This can be a tough one to nail down because it can deal with complicated measurements like what phase is the decision-making process someone is in regarding a purchasing decision. This is a focus of retargeting efforts in search and display. But you can also get certain psychographic segmented data from you analytics, like repeat (loyal) versus new customers.
- Source – This type of segmenting is very easy to do in your analytics program and can provide a lot of insight. Look for how visitors from different channels like paid search, natural search, affiliates and social media interact with your site and ultimately convert. Most likely, you’ll find it valuable to create different landing pages for different channels.
Take-Aways
These are some of the key takeaways that I summarized at the end of my presentation:
- Doubling your conversion rate is far easier (and cheaper) than doubling your traffic (directly relates to the down economy being a driving force behind testing and post-click optimization)
- Get to know all of your data intimately so you can uncover relationships
- Segment and combine your data for more meaningful interpretation
- Test everything until you know what matters
Jason Cooper
Director, Value Added Services
