Wikipedia Entry

I just created LPO / MVLPO page on Wikipedia. Everybody is welcome to contribute:
http://en.wikipedia.org/wiki/Landing_Page_Optimization

axg

Landing Page Optimization / Multivariate Landing Page Optimization (modified draft)

I have modified the draft based on Avinash Kaushik's comments and posts in his blog. Please, comment.
Alex
~~~~~~~~~~~~


See also: [Search Engine Optimization], [Social Media Optimization]

Definition of Term

Landing Page Optimization (LPO, also known as WebPages Optimization) is the process of improving a visitor’s perception of a website by optimizing it’s content and appearance in order to make them more appealing to the target audiences as measured by target goals such as conversion rate or other.

Multivariate Landing Page Optimization (MVLPO) is Landing Page Optimization based on an experimental design.

Background

A recent study by researchers in Canada showed that the snap decisions Internet users make about the quality of a web page have a lasting impact on their opinions. They also reported that impressions were made in the first 50 milliseconds of viewing[1]. These findings underscore the importance of creating the most appealing landing pages for ROI.

In addition to obvious targets such as home pages, other parts of a website may also be affecting the goals such as conversion rate. According to MarketingSherpa data, the average ecommerce shopping cart has a 59.8% abandonment rate[2]. Many website designers do not consider these pages important. A simple improvement to this infrequently changed area (vs. the front page) could bring a substantial improvement to revenue per visitor (RPV) and ROI in general[3].

Description

LPO can be achieved through targeting and experimentation.

There are three major types of LPO based on targeting:

Associative Content Targeting also called ‘rules-based optimization’ or ‘passive targeting’). Modifies the content with relevant to the visitors information based on the search criteria, source, geo-information of source traffic or other known generic parameters that can be used for explicit non-research based consumer segmentation.

Predictive Content Targeting (also called ‘active targeting’). Adjusts the content by correlating any known information about the visitors (e.g., prior purchase behavior, personal demographic information, browsing patterns, etc.) to anticipated (desired) future actions based on predictive analytics.

Consumer Directed Targeting (also called ‘social’). The content of the pages could be created using the relevance of publicly available information through a mechanism based on reviews, ratings, tagging, referrals, etc.

There are two major types of LPO based on experimentation:

Close-Ended Experimentation exposes consumers to various executions of landing pages and observes their behavior. At the end of the test, an optimal page is selected that permanently replaces the experimental pages. This page is usually the most efficient one in achieving target goals such as conversion rate, etc. It may be one of tested pages or a synthesized one from individual elements never tested together. The methods include simple A/B-split test, multivariate (conjoint) based, Taguchi, Total Experience testing, etc.

Open-Ended Experimentation is similar to Close-Ended Experimentation with ongoing dynamic adjustment of the page based on continuing experimentation.
This article covers in details only the approaches based on the experimentation. Experimentation based LPO can be achieved using the following most frequently used methodologies: A/B split test, Multivariate LPO and Total Experience Testing. The methodologies are applicable to both – close-ended and open-ended types of experimentation.

A/B Testing (also called ‘A/B Split Test’): a generic name of testing a limited set (usually 2 or 3) of pre-created executions of a web page without use of experimental design. The typical goal is to try, for example, three versions of the home page or product page or support FAQ page and see which version of the page works better. The outcome in A/B Testing is usually measured as click-thru to next page or conversion, etc. The testing can be conducted sequentially or concurrently. In sequential (the easiest to implement) execution the page executions are placed online one at a time for a specified period. Parallel execution (‘split test’) divides the traffic between the executions.

Pro’s of doing A/B Testing:

- Inexpensive since you will use your existing resources and tools

- Simple –no heavy statistics involved

Con’s of doing A/B Testing:

- It is difficult to control all the external factors (campaigns, search traffic, press releases, seasonality) in sequential execution.

- The approach is very limited, and cannot give reliable answers for pages that combine multiple elements.

MVLPO structurally handles a combination of multiple groups of elements (graphics, text, etc.) on the page. Each group comprises multiple executions (options). For example, a landing page may have n different options of the title, m variations of the featured picture, k options of the company logo, etc.

Pro’s of doing Multivariate Testing:

- The most reliable science based approach to understand the customers mind and use it to optimize their experience.

- It evolved to a quite easy to use approach in which not much IT involvement is needed. In many cases, a few lines of javascript on the page allows the remote servers of the vendors to control the changes, collect the data and analyze the results.

- It provides a foundation for a continuous learning experience

Con’s of doing Multivariate Testing:

- As with any quantitative consumer research, there is a danger of GIGO (‘garbage in, garbage out’). You still need a clean pool of ideas that are sourced from known customer points or strategic business objectives.

- With MVLPO, you are usually optimizing one page at a time. Website experiences for most sites are complex multi page affairs. For a e-commerce website it is typical for a entry to a successful purchase to be around 12 to 18 pages, for a support site even more pages.

Total Experience Testing (also called 'Experience Testing') is a new and evolving type of experiment based testing in which the entire site experience of the visitor is examined using technical capabilities of the site platform (e.g., ATG, Blue Martini, etc.) [5].

Instead of actually creating multiple websites, the methodology uses the site platform to create several persistent experiences and monitors which one is preferred by the customers.

Pro’s of doing Experience Testing:

- The experiments reflect the total customers experience, not just one page at a time.

Con’s of doing Experience Testing:

- You need to have a website platform that supports experience testing, (for example ATG supports this)

- It takes longer than the other two methodologies.

MVLPO can be executed in a Live (production) Environment (e.g., Google Website Optimizer[4], Optimost.com, etc.) or through a Market Research Survey / Simulation (e.g., StyleMap.NET[5]).

In Live Environment MVLPO Execution, a special tool makes dynamic changes to the web site, so the visitors are directed to different executions of landing pages created according to an [experimental design]. The system keeps track of the visitors and their behavior (including their conversion rate, time spent on the page, etc.) and with sufficient data accumulated, estimates the impact of individual components on the target measurement (e.g., conversion rate).

Pro’s of Live Environment MVLPO Execution:

- This approach is very reliable because it tests the effect of variations as a real life experience, generally transparent to the visitors.

- It has evolved to a relativley simple and inexpensive to execute approach (e.g., Google Optimizer)

Con’s of Live Environment MVLPO Execution (applicable mostly to the tools prior to Google Optimizer):

- High cost

- Complexity involved in modifying a production-level website

- Long time it may take to achieve statistically reliable data caused by variations in the amount of traffic, which generates the data necessary for the decision.

- This approach may not be appropriate for low traffic / high importance websites when the site administrators do not want to loose any potential customers.

Many of these drawbacks are reduced or eliminated with the introduction of the Google Website Optimizer – a free DIY MVLPO tool that made the process more democratic and available to the website administrators directly.

Simulation (survey) based MVLPO is built on advanced market research techniques. In the research phase, the respondents are directed to a survey, which presents them with a set of experimentally designed combinations of the landing page executions. The respondents rate each execution (screen) on a rating question (e.g., purchase intent). At the end of the study, regression model(s) are created (either individual or for the total panel). The outcome relates the presence/absence of the elements in the different landing page executions to the respondents’ ratings and can be used to synthesize new pages as combinations of the top-scored elements optimized for subgroups, segments, with or without interactions.

Pro’s of the Simulation approach:

- Much faster and easier to prepare and execute (in many cases) compared to the live environment optimization.

- It works for low traffic websites.

- Usually produces more robust and rich data because of a higher control of the design.

Con’s of the Simulation approach:

- Possible bias of a simulated environment as opposed to a live one

- A necessity to recruit and optionally incentivise the respondents.

MVLPO paradigm is based on an [experimental design] (e.g., [conjoint analysis], [Taguchi method], etc.) which tests structured combination of elements. Some vendors use full factorial approach (e.g., Google Optimizer that tests all possible combinations of elements). This approach requires very large sample sizes (typically, many thousands) to achieve statistical importance. Fractional designs typically used in simulation environments require the testing of small subsets of possible combinations. Some critics of the approach raise the question of possible interactions between the elements of the web pages and the inability of most fractional designs to address the issue.

To resolve these limitations, an advanced simulation method based on the [Rule Developing Experimentation paradigm] ([RDE])[6] has been introduced. [RDE] creates individual models for each respondent, discovers any and all synergies and suppressions between the elements, uncovers attitudinal segmentation, and allows for databasing across tests and over time.

History

The first application of an experimental design to website optimization was done by Moskowitz Jacobs Inc. in the autumn of 1998 in a simulation demo-project for www.Lego.com site (Denmark). MVLPO did not become a mainstream approach until 2003-2004.

Some of the companies currently providing MVLPO in one form or another:

• Google
  
• Offermatica
  
• Optimost
  
• SiteSpect
  
• Mmetrics
  
• Widemile.
  
• Moskowitz Jacobs Inc. (RDE based).
  

References

[1] Lindgaard G., Fernandes G. J., Dudek C. & Brown J. Behav. Inf. Technol., 25. 115 - 126 (2006).

[2] Can Multivariate Tests Reduce Your Shopping Cart Abandons? Real-Life Results... MarketingSherpa, October 3, 2006 (https://www.marketingsherpa.com/barrier.html?ident=29725)

[3] Andy Theekson. Rocket Conversion Rates With Multivariate Testing. (www.ezinearticles.com/?Rocket-Conversion-Rates-With-Multivariate-Testing&id=554332)

[4] Google Website Optimizer ( http://services.google.com/training/websiteoptimizeroverview/#slide=1)

[5] Avinash Kaushik. Experimentation and Testing: A Primer.
(www.kaushik.net/avinash/2006/05/experimentation-and-testing-a-primer.html)

[6] Howard Moskowitz and Alex Gofman. Selling Blue Elephants: How to make great products that people want BEFORE they even know they want them. Wharton School Publishing, 2007.

External Links

http://services.google.com/training/websiteoptimizeroverview/#slide=1

http://www.the-dma.org/cgi/dispnewsstand?article=5275

http://www.stylemap.net/

http://www.websiteoptimization.com/speed/tweak/blink/

http://www.optimizeandprophesize.com/

http://ezinearticles.com/?Rocket-Conversion-Rates-With-Multivariate-Testing&id=554332

Categories

Internet advertising and promotion Internet terminology Search engine optimization Internet marketing by method

~~~~~~~~~~~~~
Draft prepared by Alex Gofman

Landing Page Optimization
Multivariate Landing Page Optimization

To my surprise, there is no entry for LPO or MVLPO in Wikipedia. Here is a rough draft. Please, comment.
Alex

See also: [Search Engine Optimization], [Social Media Optimization]

Definition of Term
Landing Page Optimization (LPO, also known as WebPages Optimization) is the process of improving a visitor’s perception of a website by optimizing it’s content and appearance in order to make them more appealing to the target audiences as measured by target goals such as conversion rate or other.

Multivariate Landing Page Optimization (MVLPO) is Landing Page Optimization based on an experimental design.

Background
A recent study by researchers in Canada showed that the snap decisions Internet users make about the quality of a web page have a lasting impact on their opinions. They also reported that impressions were made in the first 50 milliseconds of viewing^[1]. These findings underscore the importance of creating the most appealing landing pages for ROI.

In addition to obvious targets such as home pages, other parts of a website may also be affecting the goals such as conversion rate. According to MarketingSherpa data, the average ecommerce shopping cart has a 59.8% abandonment rate^[2]. Many website designers do not consider these pages important. A simple improvement to this infrequently changed area (vs. the front page) could bring a substantial improvement to revenue per visitor (RPV) and ROI in general^[3].

Description
In a wide interpretation, there are five major approaches to LPO:

Associative Content Targeting

(also called "rules-based optimization" or “passive targeting”). Provides relevant information to the visitors based on the search criteria, source, geo-information of source traffic or other known generic parameters that can be used for explicit non-research based consumer segmentation.

Predictive Content Targeting

(also called “active targeting”). Correlates any known information about the visitors (e.g., prior purchase behavior, personal demographic information, browsing patterns, etc.) to anticipated (desired) future actions based on predictive analytics.

Consumer Directed Targeting

(also called “social”) allows the consumers to adjust the relevance of publicly available information through a mechanism based on reviews, ratings, tagging, referrals, etc.

Close-Ended Experimentation

exposes consumers to various executions of landing pages and observes their behavior. At the end of the test, an optimal page is selected that permanently replaces the experimental pages. This page is usually the most efficient one in achieving target goals such as conversion rate, etc. It may be one of tested pages or a synthesized one from individual elements never tested together. The methods may include simple A/B-split test, multivariate (conjoint) based, Taguchi, etc.

Open-Ended Experimentation

is similar to Close-Ended Experimentation with ongoing dynamic adjustment of the page based on continuing experimentation.

This article covers in details only the last two of the approaches, while the first three entries could be better classified as targeting methods rather than optimization.

Simple LPO involves a series of one or more disconnected A/B tests, each representing a “slot” on a page template where content can be placed. This approach generally is very limited, and cannot give reliable answers for pages that combine multiple elements.

MVLPO handles a combination of multiple groups of elements (graphics, text, etc.) on the page. Each group comprises multiple executions (options). For example, a landing page may have n different options of the title, m variations of the featured picture, k options of the company logo, etc.

MVLPO can be executed in a live (production) environment (e.g., Google Website Optimizer^[4], Optimost.com, etc.) or through a market research survey/simulation (e.g., StyleMap.NET^[5]).

In Live Environment MVLPO, a special tool makes dynamic changes to the web site, so the visitors are directed to different executions of landing pages created according to an experimental design. The system keeps track of the visitors and their behavior (including their conversion rate, time spent on the page, etc.) and with sufficient data accumulated, estimates the impact of individual components on the target measurement (e.g., conversion rate). This approach is very reliable because it tests the effect of variations as a real life experience, generally transparent to the visitors. The drawbacks of the approach are the typically high cost, the complexity involved in modifying a production-level website, and the long time it may take to achieve statistically reliable data caused by variations in the amount of traffic, which generates the data necessary for the decision. This approach may not be appropriate for low traffic / high importance websites when the site administrators do not want to loose any potential customers. Many of these drawbacks are reduced or eliminated with the introduction of the Google Website Optimizer – a free DIY MVLPO tool that made the process more democratic and available to the website administrators directly.

Simulation (survey) based MVLPO is built on advanced market research techniques. In the research phase, the respondents are directed to a survey, which presents them with a set of experimentally designed combinations of the landing page executions. The respondents rate each execution (screen) on a rating question (e.g., purchase intent). At the end of the study, regression model(s) are created (either individual or for the total panel). The outcome relates the presence/absence of the elements in the different landing page executions to the respondents’ ratings and can be used to synthesize new pages as combinations of the top-scored elements optimized for subgroups, segments, with or without interactions. This survey approach using statistically designed combinations turns out to be much faster and easier to prepare and execute in many cases compared to the live environment optimization. It also addresses the issue of low traffic websites. Furthermore, the survey method may produce more robust and rich data because of a higher control of the design. The drawbacks of the approach include the possible bias of a simulated environment as opposed to a live one, and a necessity to recruit and optionally incentivise the respondents.

MVLPO paradigm is based on an experimental design (e.g., conjoint analysis, Taguchi method, etc.) which tests structured combination of elements. Some vendors use a full factorial approach (e.g., Google Optimizer that tests all possible combinations of elements). This approach requires very large sample sizes (typically, many thousands) to achieve statistical importance. Fractional designs typically used in simulation environments require the testing of small subsets of possible combinations. Some critics of the approach raise the question of possible interactions between the elements of the webpages and the inability of most fractional designs to address the issue. To resolve these limitations, an advanced simulation method based on the [Rule Developing Experimentation paradigm] ([RDE])^[5] has been introduced. [RDE] creates individual models for each respondent, discovers any and all synergies and suppressions between the elements, uncovers attitudinal segmentation, and allows for databasing across tests and over time.

History
The first application of an experimental design to website optimization was done by Moskowitz Jacobs Inc. in the autumn of 1998 in a simulation demo-project for www.Lego.com site (Denmark). MVLPO did not become a mainstream approach until 2003-2004.

Some of the companies currently providing MVLPO in one form or another:
Google
Offermatica
Optimost
SiteSpect
Mmetrics
Widemile.
Moskowitz Jacobs Inc. (RDE based).

References
^[1] Lindgaard G., Fernandes G. J., Dudek C. & Brown J. Behav. Inf. Technol., 25. 115 - 126 (2006).

^[2] Can Multivariate Tests Reduce Your Shopping Cart Abandons? Real-Life Results... MarketingSherpa, October 3, 2006
(https://www.marketingsherpa.com/barrier.html?ident=29725)

^[3] Andy Theekson. Rocket Conversion Rates With Multivariate Testing.
(www.ezinearticles.com/?Rocket-Conversion-Rates-With-Multivariate-Testing&id=554332)

^[4] Google Website Optimizer ( http://services.google.com/training/websiteoptimizeroverview/#slide=1)

^[5] Howard Moskowitz and Alex Gofman. Selling Blue Elephants: How to make great products that people want BEFORE they even know they want them. Wharton School Publishing, 2007.


External Links
http://services.google.com/training/websiteoptimizeroverview/#slide=1
http://www.the-dma.org/cgi/dispnewsstand?article=5275
http://www.stylemap.net/
http://www.websiteoptimization.com/speed/tweak/blink/
http://www.optimizeandprophesize.com/
http://ezinearticles.com/?Rocket-Conversion-Rates-With-Multivariate-Testing&id=554332


Categories:
Internet advertising and promotion
Internet terminology
Search engine optimization
Internet marketing by method

Essay on the history of the first use of Multivariate Landing Page Optimization

Who is it that deserves more credit for an invention? Is it the actual inventor, spending countless nights thinking, drawing, building and all too frequently failing repeatedly before shaping the idea? Or is it the astute businessman noticing someone else’s wild idea and seeing potential, taking a financial risk to get the rewards of the invention? Or perhaps the merchandiser that makes it to a commodity available to everyone?

We, in general, do not appreciate those out-of-nowhere troublemakers who are crying “I’ve been there first”. Whenever news breaks out about a patent infringement suite from an unknown company against an industry leader seeking untold riches in damages, the first idea that comes to our mind is “O, boy, another vulture”.

But try to look at the other side of the story, at someone’s feeling, the one’s who did in fact invent something but never patented it or even put real efforts to bring it to fruition. He might thought “I am sure someone else has done it before – I can’t be the smartest in the world - If nobody did it until now, it might not be that great idea after all”. Or any of many other explanations just to keep status quo. And a few years later to read about a phenomenal success story of ACME Corporation (or John Doe) that “got that crazy idea” and made that ingenious new gadget that nobody would even think about a few year ago.

But enough said. The story of multi-variate landing page optimization (LPO) is one of smaller opportunities “I’ve-done-that-N-years-ago –I-was-there-first!” I will not talk about the larger opportunities I’ve missed in my life – you will not believe me anyway.

So, without further ado, let’s rewind back the clock to 1998. The Internet craze was going out of control, sign on bonuses for startups were so ridiculous that everyone felt they were from another planet (perhaps because I did not get any?)

Software and hardware had finally reached the level that made creating dynamic web pages easy and displaying them in a sequence fast enough that the users did not to feel they could have a cup of coffee between the screens. After many years of successfully using conjoint analysis (a form of multivariate testing) on desktops around the world including with graphical variables, we had started working on a Web version of Ideamap, our flagship software. While the software was still in beta, suddenly we got a call from Copenhagen, from one of the longest standing licensees of Ideamap, Lene Hansen (GfK Denmark). Lego, a client of hers, was attempting to improve their website to make it stickier to the visitors. And Lene got the idea – could Ideamap be utilized to answer Lego’s question regarding the research-based Website optimization?

Although our Danish colleagues had not known yet precisely what they were searching for to make the website better, they had realized the need to use advanced customer research to achieve that goal. This was an incredibly critical thinking and a break-through – Lene’s and Lego’s realization that WebPages could be treated the same way as printed copy and optimized based on the consumer research. The rest was easy.

For us it was just another application of the approach we used for package optimization (dynamic graphical overlays based on an experimental design and a predefined template). It just has to be done online. In a few days I put online a demo (see picture below) utilizing a brand new Visual Basic functionality for web applications that allowed for systematic variation of the elements of the front page and presentation them in a sequence to respondents for their rating.

I arrived to Copenhagen on a cold gloomy day right after the New Year of 1999. The cultural experience of that city, from the colorful plums of the nobility and officers visiting the royal palace reception to the infamous Friday nights (it’s a kind of ‘happy hours’ on steroids that extends to early Saturday morning without the restrictions imposed by lack of a designated driver) deserves a separate story.

The next day we were in the airport quite early with Lene to catch our flight to Bellund, the headquarters of Lego. The only formality to get onboard was showing your ticket (no ID, metal detector or X-ray were needed to get to the jet for the 20 minutes flight).

Something was telling me (was it my lavish Danish breakfast?) that the 737 was not specifically designed to fly such short distances and at such low altitudes. I was very glad to land in what once was Lego corporate airport but later donated to the city.

Two Lego employees met us at the gate and whisked the small Opel to the sprawling campus nearby. We were a few minute late and the meeting was already running. Nobody seemed to notice or at least acknowledge our quiet entrance through the side door, and the meeting continued without hiccup. The only change was that the presenter switched to flawless English halfway thorough a phrase and the remainder of the meeting was as if I had never left New York (except for my heavy accent, which was the only noticeable one in the room).

It was the first (at least, as far as we know it now) case of using conjoint analysis approach for LPO. My search and interviews of industry leaders did not yield other contenders for the title yet.

Unfortunately, we have never really capitalized on that early experience. We were swamped with a multitude of ideas waiting to be implemented – multivariate video ad optimization, an ‘innovation machine’, establishing new science Mind Genomics, applying our approach to presidential elections, public policies, stock markets, etc... And so LPO remained in the virtually exclusive domain of web designers and webmasters.

A few years later several startups jumped on the idea, but it still lingered as a novelty until recently when Google marked this approach as mainstream by entering the field with its Google Optimizer.

So, who in fact deserves the credit for multivariate LPO? Is it the first inventors that did it mostly unbeknownst to the world? Or is it those startups that made it available to the public albeit on a very limited basis? I vote for Google – they made it readily available to everybody!


Good job, guys!

Alex Gofman


Figure 1. Two sample screens from the demo project with Lego.

Welcome to my blog!

Welcome to my blog!

Although I am not a complete novice in writing, blogging is new to me. For long time it reminded me digital photography - a temptation to just keep clicking instead of carefully crafting every shot. But the quality and depth of the blogs that I read lately have changed my mind and finally convinced me to jump the wagon. Hope, my potential readers will not regret.

Alex