CRO and Personalization Trends From Opticon19

“Disappoint your customers and they’re just 1 click away from your competition.” That’s what echoed through the room of eager marketers as Jay Larson, CEO of Optimizely, kicked off Opticon19 in San Francisco.

As we looked around the room, we could see others were sharing our awkwardly-mixed sentiment of acceptance, fear and opportunity. Jay is right. In today’s digital landscape, consumers have options, and competitors are proactively positioning themselves to pick up where we left off, or better said – fell short.

Consumer expectations are growing with every digital interaction – faster websites, relevant content, personalized suggestions, cross-device continuity and free beer (oh wait that’s just me). As our tactics and capabilities have expanded, we’ve fostered a culture of “wow me again,” where the bar is seemingly never quite high enough for our modern digital consumer.

Our audience expects individualized customer experiences: “I am a segment of one. Recognize my preferences, eliminate any reference to things that don’t matter to me, and help me get what I want quickly and easily.”

Our mandate is to give customers what they want. Personalized messaging that delights is the new baseline. As expressed by Claire Vo, SVP of Product Management at Optimizely, “The future of digital is massively personalized. The way to get there is experimentation.” This quote would serve as an inspirational lighthouse and “true north” to guide the next several days of collaboration and learning.

Like Kids in a Candy Store – Opticon is a Playground For A/B Testing

In the world of conversion rate optimization, Optimizely is an industry leader, and Opticon brings together the best experimentation professionals to discuss CRO and web personalization trends.

With the theme established, Opticon19 was officially underway and we were ready to collaborate with the most progressive digital experimentation professionals in the industry.

With keynotes and breakout sessions led by experimentation professionals at IBM, Hewlett Packard, Salesforce.com, Netflix, Nike, StubHub, and more, the progressive knowledge being exchanged was truly inspiring. We were pumped to be there and hit the agenda with the enthusiasm of a kid on Christmas morning.

Here is our shortlist of must-know trends in the world of experience-based optimization.

1. Democratizing The Board Room – Experimentation is The Next Big Thing in Business Management

It’s hard to argue with data, but experimentation helps democratize decision making. We are seeing more and more businesses use experimentation as the foundation for high-level, organizational decisions.

While CRO and experimentation, in general, have historically been tools of hands-on, in-the-trenches teams of analysts, developers, product managers, and marketers, the value of experimentation has made its way to the board room. Executives are abandoning the “loudest voice in the room” model, and turning to experimentation data to inform decisions at the highest level of the organization.

Experimentation is no longer purely about optimizing conversion rate; it’s about evaluating risk and determining the most efficient use of resources. If we validate ideas before going all-in, we can essentially de-risk resource investment. That’s good stuff.

2. A Culture of Experimentation – The Company That A/B Tests Together Wins Together

“There’s no such thing as a bad idea” – Not really true as a broad principle (like when my childhood friend talked me into tying a garden hose across the street so we could stop traffic and charge a toll to pass), but definitely true when it comes to exceptional experimentation teams.

Great experimentation teams are driving top-line revenue. Recent cross-industry metrics report an increase of 14% or more for teams with 20 or more experiments per month. So what does it take to create a world-class team? It’s about more than a/b tests and personalized experiences – it’s about a culture of experimentation that permeates the entire organization.

IBM has grown its testing team from 0 to 6000 over the last 4 years. Here’s what Ari Sheinkin, VP of Marketing Analytics and Performance Media, said about the keys to their success:

1. Experimentation is meant to be collaborative. Collaboration is essential.
2. It needs to be democratic. Test prioritization needs objective models – it’s not about job titles.
3. Incentivize the team to get collaborative, especially at first.
4. Iterate on wins and always learn from your non-winners (aka losers – but that’s a bad word in experimentation – see point 5).
5. There are no losers in experimentation; they simply don’t exist. As long as you learn, you win, especially when you consider that only 10% of tests are winners, according to Optimizely.

3. Quantifying The Value of Conversion Rate Optimization – ROI Models Decrypted And Demystified

Without a doubt, the executive team will eventually ask to quantify the value of the conversion rate optimization and personalization program. It’s a legitimate request, but extremely difficult to deliver. How do we assess the synergistic effect of hundreds of tests on varying digital elements, for differing audiences, at different stages of the funnel, and estimate the combined impact on top-level organizational revenue?

It has been a struggle for experimentation teams since CRO and personalization became a thing. For smaller teams, this ask could potentially tap enough resources from actual experimentation to deflate the entire program and bring momentum to a snail’s pace. For larger teams with ample resources, it’s a simple problem of complexity – how on earth do we do it. As expressed by Ian Tucker, Optimization Manager at The Wall Street Journal, “You can’t just tack the ROI of each test on top of the next, because the website evolves and test impact aggregates and overlaps.”

Luckily, our friends at Optimizely have decrypted and demystified the process, creating a rational, executable framework based on the work of some of the most progressive experimentation teams.

By acknowledging and mitigating the impact of common pitfalls (valuation of tests, quantifiable value of proxy metrics, projection of long term impact, overlap and cross-contamination of overlapping tests), the Optimizely model focuses on metric and call to action valuation, goal alignment, and various impact modifiers that account for diminishing returns over time. With the launch of this model on September 12th, 2019, estimating ROI will be a realistic goal for almost any experimentation team.

4. The Big Personalization Trend – The Movement From Personas to 1:1 Personalization

By far, the most excitement at Opticon was centered around personalization, and specifically the opportunities created by leveraging machine learning to deliver real-time, dynamic, personalized digital experiences.

We are moving to a point where two different site visitors might not even experience recognizably-similar versions of a webpage, as sites are leveraging machine learning to instantaneously predict content preferences. We’re amidst an evolution from persona-based personalization to true, individual, 1-to-1 personalization, where each individual receives a truly unique and personal digital experience.

The key to effective personalization has historically been audience creation: we are looking for segments that are both big and actionable. “Actionable” ensures that we can differentiate the experience in a meaningful way that enhances the user experience, increases conversions, and helps align consumer and business goals; “big” ensures that the effort required to create personalized content will be worth it, as differentiation has typically been dev heavy and expensive.

The transition we are experiencing is essentially affecting the necessity to identify “big” audiences, aka “personas.” We create personas because it isn’t realistic to consider the wants/needs of each individual; we simply don’t have the bandwidth to consider the individual nor would the benefit offset the cost. Machine learning is changing all of that.

Machine learning is about prediction. Models/algorithms that predict future behavior based on historical behavior are developed and refined, essentially trying to predict the future by looking at historical relationships and trends in massive amounts of data. These models allow marketers to efficiently optimize, automate and personalize various marketing efforts based on organizational goals.

Machine learning is making true 1-to-1 personalization a reality. As a step in that direction, Optimizely recently launched “adaptive audiences,” a machine-learning-based feature that allows marketers to tag website content, create dynamic customer profiles, and personalize based on adaptive recommendations. 1-to-1 personalization is just a step behind.

5. Parting Thought: How Netflix Won Your Heart, Attention And Recurring payment

We had the privilege of spending some 1-on-1 time with Gibson Biddle, former VP of Product at Netflix. He shared the secret to his success with experimentation at Netflix. It’s simple, yet profound: DHM.

D: Delight the customer in
H: Hard to Copy
M: Marginally enhancing ways

We loved this model. “Delighting” the customer requires us to put them first. Is shoving your contact form in front of them on every page customer-centric? Is anyone delighted by a contact form pop-ups? “Hard to copy” experiences require creativity and progressive thinking – that’s always a good thing. And hearing Gibson use “marginally” was such a relief. We always feel a need to swing for the fences, and big wins are awesome! But an aggregation of marginal improvements can help move us to the top of the food chain, just like it did for Netflix.

Are you ready to build your culture of experimentation and take your A/B testing and personalization to the next level? Learn how our digital marketing services can help your organization grow.