Improving the "Stickiness" of Your Website Further:
Part 2: If they like A and B, would they like A+B?
Alex Gofman,
Vice President, Moskowitz Jacobs Inc.
Interactions in consumer research: searching for a needle in the hay
A few years ago, Heinz introduced quite weird Funky Fries – chocolate flavored and blue-colored fries. Heinz bet was on combining some highly popular ideas. Huge army of the consumers loves fries. Even bigger (arguably) crowd is sucker for chocolate. And kids love color.
As you can guess (or already know), the product has failed miserably. The ideas were so divergent that there was no synergy between them in the eyes of the consumers. Quite opposite, by putting the conflicting ideas together they lost appeal of both fries munchers and chocolate connoisseurs producing a negative effect (Bhatnagar, 2003).
In the marketing lexicon, the situation when reaction of consumers (their liking scores, purchase intent, etc.) to the messages (or ideas, elements of a package or a web page, etc.) combined together are not equal to the sum of their individual ratings, is called an interaction. A positive interaction (when customers' liking of the combined offer is higher than the sum of individual items scores) is called synergism. If customers like the combined idea less than the sum of individual liking scores of the components, then it is called a suppression (a negative interaction).
The problem lies in the shear number of possible pairs of elements. For example, if we have six placeholders on a webpage with six possible alternatives for each one, there are 540 possible pairs of elements.
This should explain why until very recently, the effect of interactions either was ignored or considered a middle ground between art and heavy statistics. In latter case, it required an expert guess about possible significant pairs. Such several 'alleged' (guessed) interactions were then tested with the consumers through a sophisticated statistical method of incorporating these pairs into the survey to confirm / reject the hypothesis.
Market researches tried to tackle the issue for many years (e.g., Green, 1973). Yet, many years later, if the expert was right (or lucky?) in foretelling the potential interactions, the results could lead to improved ideas. If not – too bad: some great ideas might have been discarded unnoticed or bad ideas went into production undetected.
Extending RDE to discover all and any interactions
In the previous article Improving the "Stickiness" of Your Website, we discussed Multivariate Landing Page Optimization (MVLPO) approach which helps to identify a winning combination of the elements of a webpage. Rule Developing Experimentation (RDE)
paradigm introduced in the article mixes and matches the elements of the page according to an experimental design and presents synthesized web pages to consumers for evaluation. Collected data then used to estimate individual contribution of every element to the liking of the web pages (conditional probability of people buying from this site, for example). This in turn allowed us to construct the most appealing webpage from the set of elements tested.
In most cases, the results of this approach help you to create optimized web pages. In a number of occasions although, some latent interactions exist between the elements of the page. Using a highly trained expert opinion to guess these interactions is not a very viable option in the fast moving world of web site design not taking into account the price implications. RDE easily overcomes the limitations of the old methods by automatically testing all and every combination of the elements of the page multiple times according to a built-in unique permuted experimental designs. Because the complexity of the statistical foundation are usually incorporated inside the tool, no special knowledge on the user side is needed (if you are still interested, you can find the details in Gofman, 2006; Moskowitz, Gofman, 2004).
Now let's explore how to make sure that the winning individual parts of the pages, when combined, do not fail. Furthermore, let's see how to find a combination of Web page elements that together produces more impact than just the sum of individual impacts. Putting to use the basic math formulas:
We do not want: 1+1 < 2
We want: 1+1 > 2
Golf Gear Case Study: deeper data mining
Note: All the data in this and previous articles are from the actual project, although the visuals and other marketing materials are representative equivalents and not related to any specific website.
In the previous article, we followed the operator of an online golf store who wanted to optimize the landing page to increase the conversion rate and revenue per visit. As it catered to affluent golf players, the general traffic was not very heavy. However, the revenue per customer (RPV) and the customer lifetime value (CLV) were high because the site sold luxury and premium equipment and strived to retain their patrons. The combination of these conditions precluded the operator from experimenting on live website to avoid possible less than optimal experience for their valuable customers.
The operator chose to use MVLPO in a simulated environment using an RDE tool. She had several options for the banner, feature picture, and different promotions and at the end of the project discovered the best combination of these components (Figure 1). She found out that by choosing 'wrong' elements (the lowest scoring vs. the highest) she would loose half of her potential clients. Or, in reverse, by selecting the best possible elements, she could double the number of happy visitors willing to buy from her site.
In virtually any MVLPO case based on traditional methods, this would be the end of the research stage. RDE on the other hand allows for mining the data even deeper.
Figure 1. Optimized webpage for the golf site without taking into account any possible interactions. The conditional probability of visitors being interested in buying from this site was 48%.
In some cases, there are potential interactions between the elements of the page (both positive and negative). Because of the unique permutation algorithm of experimental design, RDE allows for all and every combination of the elements to appear on the test screens multiple time. This means that it is possible to include them as independent variables into regression model. In our case, we have 90 possible combinations.
If this sounds for some readers a bit like a less than pleasurable lecture in statistics, don't quit reading. The good news – this is all incorporated inside RDE approach and available at a virtually 'point-and-click' level. One does not need to know how bits and bytes are moving inside a processor to use a PC for browsing. The same thing is true about discovering possible interaction using RDE – you do not to be a professor of statistics to find it out – RDE does it for you.
Not every case produces meaningful interactions. In many occasions, interactions are not very strong and could be ignored (considered not significant). If the utility (conditional probability of customers being interesting in buying from this site) of the combination is below the empirical threshold of (+/- 5), it could be discarded. In that case, the results of MVLPO would look like Table 1 in the previous article.
It also should be noted that the effect of the interactions changes the regression model and affects somewhat the rest of the utilities. In a model without interactions, the values of hidden synergies and suppressions are distributed among the individual elements. In a more detailed regression model that includes interactions, the values are extracted and assigned to the cross-terms (pairs of elements).
Comparing Standard and Interactions Models
Table 1 contains the utilities of the individual elements of the web page with several discovered meaningful interactions (right column) compared with the Standard model (middle column) from the previous article. This case does not have very high interactions values (in some cases, an interaction along could add 20 or more points to the liking score) but it does demonstrate the approach.
Table 1. Performance of the elements with interactions. Notice, that the values are somewhat different for the model with interactions compared to the standard model.
Standard Model Interactions Model Base Size 125 Constant 10 9 Banners A3 Banner 3 0 -1 A1 Banner 1 -1 0 A2 Banner 2 -1 -1 Promo 1 B2 Free shipping 7 7 B3 $5.99 shipping 3 2 B1 Free $50 card 3 3 Visuals C2 Golfer playing 16 15 C3 High-tech club 8 8 C1 Golf shoes 8 7 Promo 2 D2 Final clearance-up to 65% off 12 13 D1 Save up to $100 8 8 D3 Free personalization 4 4 Promo 3 E1 St. Andrews Sweepstakes 3 3 E2 115% price guarantee 3 3 E3 Golf vacation entry 0 0 INTERACTIONS A2*C2 N/A 6 D2*E3 N/A 7 C1*E2 N/A -9
The data suggest that the winning web page from the previous article was not the one that generates the highest interest in customers to buy from the site.
The optimal webpage (from the previous article) based on the standard model was:
(Conditional Probability of visitors buying from the site) =
= Const + A3 + B2 + C2 + D2 + E1 = 48%.
We can get a higher purchase intent score if we use a slightly different set of elements:
(Conditional Probability of visitors buying from the site) =
= Const + A2 + B2 + C2 + D2 + E3 + D2*E3 + A2*C2 =
= 9 + (-1) + 7 + 15 + 13 + 0 + 7 + 6 = 56%,
producing the optimal concept presented on Figure 2.
We have replaced two marginally higher scoring elements in two categories with lower scoring ones: in Banners, we switched from A3 (0) to A2 (-1); and in Promo 3, from E1(+3) to E3(0). Although with these subtle changes we have lost 4% in the individual values, the identified interactions in the case study compensated the shortfall and added additional 8% to the purchase intent (note, that the utilities for the interaction model are slightly different from the standard regression model and the elements in the case study are representative).
Figure 2. The highest scoring webpage created using Interactions Model. Although the differences are very subtle, the page has 8% higher conditional probability of customers buying from it compared to Standard Model optimization (Fig. 1).
Conclusions
This case study does not have the most impressive interactions I've seen in my experience. Sometimes, the synergy between the elements reaches 15-20 points or even more. In some cases, there are no significant interactions at all. Yet in some others, a negative interaction (suppression) is so strong that it negates the high positive contribution of individual elements (if any).
For many years, the researchers knew about the existence of possible interactions and tried to identify them by incorporating several handpicked pairs into surveys, usually by guessing. With the introduction of RDE to MVLPO, the permuted individual designs afforded for testing all and any possible combinations of the elements multiple times allowing for more precise models and more targeted optimized pages.
The bottom line, it is difficult not to agree that improving the conversion rate by 10-20% would make a very big difference for virtually any website operator. It is possible to achieve that by just recombining your existing materials with a tad deeper data-mining available in some tools as a simple push of a button.
References
Bhatnagar, Parija (06/20/2003). Blue food goes down the drain. CNN/Money. Retrieved on 11/21/2007.
Gofman, A. (2006). Emergent Scenarios, Synergies, And Suppressions Uncovered Within Conjoint Analysis. Journal of Sensory Studies, 2006, 21(4): 373-414.
Gofman, A. Improving the 'Stickeness' of Your Website. Financial Times Press (09/21-2007). Retrieved on 11/21/2007.
Green, Paul E. (1973). On the Analysis of Interactions in Marketing Research Data.
Journal of Marketing Research, Vol. 10, No. 4 (Nov., 1973), pp. 410-420
Moskowitz, H.R. and Gofman, A. (2004). A System and Method for Performing Conjoint Analysis. U.S. Provisional Application No. 60/538,787, Patent Pending.
Moskowitz, Howard R. and A. Gofman (2007). Selling Blue Elephants: How to make great products that people want BEFORE they even know they want them. Wharton School Publishing, 2007.
No comments:
Post a Comment