The Value of Conversion Rate Optimization – A Q&A with Siddhartha Kathpalia of VWO

As a creative digital performance marketing agency, Noble Studios is always looking for optimization opportunities for our clients to strengthen the overall performance of their websites and digital marketing campaigns. It’s easier said than done, but through conversion rate optimization (CRO), we’re able to continually test different strategies and design elements, and measure and adjust them in real time.

We sat down with Siddhartha Kathpalia, a product marketer from VWO, one of Noble’s optimization platform partners, to talk all things optimization, and understand the various elements that go into developing an optimization program for businesses across industries to consider.

If you’d like to start or bring your existing optimization strategy to the next level, contact us and we’ll be in touch.

The terms CRO, Personalization, and A/B Testing are often used interchangeably. Can you please explain what each one of these is and how they relate to each other?

CRO, Personalization, and A/B Testing are very different terms. Personalization is presenting to website visitors and audience what they like based on their personal preferences identified. A/B Testing is showing different versions of just about anything to know what resonates better. CRO (Conversion Rate Optimization) is the end-to-end process followed to improve website performance. Personalization is more of a set of hypotheses that A/B Testing can help validate and A/B Testing is one of the crucial components of CRO.

Organizations engaging in optimization programs want to quantify the value of their investment. What are some of the best examples of CRO program validation you have seen from your clients? What are some of the most creative valuation methods?

Testing is an essential component of CRO and helps in gaining the confidence of changes being made permanent. It is in the testing phase that the performance of variations is known; the better performing one is typically deployed. Therefore, testing tells you how much better the winning variation performs over others, which is the uplift and is measured in percentage change of the main metric of the test.

By placing a pixel or code snippet that gets fired when a certain parameter or condition(s) is met, attribution becomes very easy. It is possible to pass a “revenue” value into this pixel or code snippet which can be measured, thus eliminating a few steps in arriving at the impact. [VWO allows you to track revenue goals easily]

One can also integrate their CRO and Analytics tools to view results in their preferred analytics tool.

The scope of CRO program management is often underestimated by organizations. What do organizations need to consider before executing a CRO program?

CRO indeed requires program management! People from different teams and skill sets come together to achieve the desired CRO goals and objectives. CRO can be owned by marketers, product managers, UI/UX designers and/or website developers. We have seen organizations where the CRO team comprises members from each of these teams. Naturally, when members from multiple functions formulate a team, it becomes critical to manage expectations and bandwidth for each of them. This is oftentimes neglected by organizations who are new on their CRO journey (even up to several years). Needless to say, CRO is a culture and therefore requires buy-in from organization leaders and demands a significant amount of time to build. A few things organizations must consider before embarking on their CRO journey:

1. It’s not a project, it’s a way of life
2. It requires dedicated effort from people possessing different skill sets
3. It demands owners who drive it dedicatedly who need to be hired or transitioned into
4. It is a very scientific process and must be treated as one; think of it as the laboratory that changes your organization into a rocket ship – this laboratory needs to be built, sanitized, processified, trusted and budgeted

Mature CRO programs often make use of both A/B testing and personalization. How do you see each playing a role in conversion rate optimization? What is the interaction between the two?

Personalization happens when exclusive, one-on-one custom experiences are delivered to visitors when they land on a website. This means that personalization is very specific to each individual and hence what works for one visitor may not work for another. A/B Testing can help an organization narrow down on the aspect of personalization that works best for both the organization and the visitors. Personalization can help build hypotheses that get validated by A/B Testing.

It takes hundreds of parameters to define an individual. Think of each of these parameters as variables. There is only one way to know whether a variable has any impact on conversion rates (the goal of CRO) – A/B Testing. It’s important to not just know whether a variable affects conversion rates, but also by how much, since there is a setup cost involved in personalization. Multivariate Testing allows isolating and zeroing in on the impact of each variable.

Personalization opportunities are endless. How do you identify key opportunities and prioritize between various experiences?

This is very important – to know which personalization opportunity should be used. The first step is to know what opportunities (variables) are available readily, and which would require infrastructure or effort to deliver. The next step is to heuristically determine whether a variable may have an impact on conversion rates.

Variables that heuristically stand out as having a chance of having an impact on the conversion rates and are readily or more easily available become the first to be tested. If from testing, a variable does come out as impactful, the necessary infrastructure should then be set up for it to be delivered consistently and without glitches. The VWO Plan comes with an in-built Impact-Chance-Ease scoring framework which can help in prioritizing and actioning the testing pipeline.

A/B Testing or Personalization… if you could only do one, which would it be and why?

For most businesses, A/B Testing is the way to go! Personalization is infra-structurally intensive and does not warrant any returns or uplifts (unless tested). A/B Testing, on the other hand, tells you which infrastructure to invest in because it always tells you the path of highest or guaranteed return.

How do you see the impact of GDPR and the California Consumer Privacy Act affecting CRO and personalization in the coming years?

Almost all laws around the world are formulated looking at events from the past. Recent events and their possible recurrence have led to GDPR and CCPA. Businesses have to be more conscious of what data they collect, how they process it, and how it is disposed of. VWO has always given privacy and data security it’s due importance and is ready to abide by these vital laws.

Overall, though, these are positive changes that will bear fruit in the form of more free interactions between prospects and brands – without the underlying fear of data misuse. Any dips in conversion rates in the present will be compensated as conscious brands take compliance in their stride and invest in developing practices that focus on making the process of consent acquisition as painless (and even delightful) as possible.

What advice can you give to an organization looking to start a conversion rate optimization program?

Organizations starting CRO programs are like people keeping new year resolutions – people often decide to create resolutions on their own, and then abandon them soon after the new year. My advice would be to learn from new year resolutions:

1. Be mindful of the fact that resolutions get dropped; take countermeasures
2. Start small, make a habit through consistency
3. Take inspiration from others and slowly create your own process
4. Don’t get disheartened by experiments that don’t give you expected results, there’s always something to leverage

For more mature CRO programs, what are the best ways to continue to find and develop new test ideas?

There are three sources of identifying new test ideas:

1. Looking outward – Take inspiration from what competition is doing and even other innovations in the same or different domain.
2. Looking inward – Keep looking at your digital properties as a user and determine if there are hiccups in the user experience. Also, analyze your digital properties using different analytics tools.
3. Retrospect – Look at your past experiments to see whether any can be run again, given context changes with time. Dive deep into your past experiments to find hidden pieces of knowledge or anything that stands out.
   While the first 2 are generic pieces of advice, applicable to CRO programs of any maturity, the third is only for mature CRO practitioners who have a past to look into.

What do you believe sets VWO apart from other CRO platforms? Are the key differentiating features equally important to all potential users? Or are there a specific “type” of organizations that benefits most from the platform?

Some of the key VWO features that differentiate from other CRO platforms, agnostic of who uses it:

• It’s the only end-to-end Conversion Rate Optimization platform that comes with features to gather user engagement data, prioritize testing pipeline, conduct experiments, get statistically determined results, and perform post-test data analyses.
• VWO is the only CRO platform that runs on Bayesian statistics and not frequentist statistics. VWO SmartStats helps reduce testing time by half. Most other competitors only test for statistical significance, which is a small fraction of what goes into robust decision making.
• VWO SmartCode is one of it’s kind – it’s lightweight, fast, customizable, and comes in all shapes and sizes (to suit the make of the website on which it has to go). Setting it up is super easy through various integrations for top CMSes.
• CRO can always do with some aid, which is why VWO offers world-class support to always be there to clear any roadblocks. Support is offered through all communication channels (email, phone, and chat) and is available 24×7.

Can you share any insights into new feature enhancements planned for 2020? what exciting upgrades and enhancements can your users expect in the next 12 months?

VWO is already ready and compliant with many important internet laws. Our focus has been consistent over the years and will continue to remain so. We are also always looking for ways to optimize the VWO SmartCode to minimize its impact on any website it’s placed on.
Besides our regular areas of focus, we are also focusing on:

1. Deeper Integrations
2. Advanced metrics and segments
3. AI-based text suggestion (already in early access)
4. Enhanced test creation and reporting
5. Improved reporting and world-class server-side testing capabilities
6. Personalization

---

Siddhartha Kathpalia is a Product Marketer at VWO. He has almost 10 years of experience and has worked with various organizations (both B2B and B2C) across different marketing functions. He is also a drummer and plays for a progressive rock band called Time Throttle.