Tuesday, December 1, 2015

Do A/B test results fade over time? Surprising results from three years of data.

TL;DR: WiderFunnel has run a three-year demonstration of the most rigorous way to prove the long-term value of a conversion optimization program. Read the full conversion optimization case study here.

Visiting the DMV is no one’s favorite chore.

Just understanding how it works can be confusing, complicated and frustrating.

DMV.org aims to change that by providing information to simplify the DMV experience. More than 200 million site visitors a year look up information on how to register a car, get a driver’s license, learn more about speeding tickets and so on.

WiderFunnel’s CRO Team Builds Strategy and Gets Results For DMV.org
DMV.org’s homepage.

The thousands of pages of content customized for drivers in each state are fertile ground for a rapid testing optimization program. With WiderFunnel’s support, the company has built a world-class optimization program that has consistently lifted its revenue.

In fact, for three years running, the program has doubled the company’s revenue on their highest revenue products. Yes, that means a compounded doubling each year! At the same time, we’ve been able to help them build a framework of customer insights that continue to deliver top results.

A new personalization and optimization case study

WiderFunnel has just released a written case study that looks at the challenges DMV.org faced, the solutions we established for them, some surprising experiment insights, and the business results they’ve seen. Some of the highlights include:

The lack of conversion strategy

A “hunt and peck” approach and a poorly integrated tech stack generated mismatched data and unreliable results. WiderFunnel worked with DMV.org to develop an integrated long-term optimization program that consistently delivers both revenue lift and customer insights.

Geographic personalizations

A/B testing on geographic personalizations yielded some surprising results that went directly against commonly held personalization “best practices.” In addition, testing revealed one word that makes all the difference to people in *some* geographic areas.

Geo under field winner
Did geographic personalization work in this case?

The minibanner test
One test found that the shape of a minibanner increased revenue per visitor by almost 90% while photos and GIFs moved it less than 10%.

Car-shaped minibanner
How did this minibanner perform against the competition?

Flashy pages
Other auto insurance pages often have lots of bling: bullet points, a flashy click button, lots of logos. Testing on the DMV.org site found that that approach isn’t always the best, and that a simplified transactional page beat a noisy page by 40 percent.

DMV Noisy
We pared down this flashy page…
We tested a less noisy results page.
…and got big results with a cleaner version.

It wasn’t just the tests themselves that made the partnership a success. The integrated team shared its knowledge so DMV.org could build its own expertise and improve its internal processes for future testing, making these outstanding results sustainable.

Now, the integrated team of DMV.org and WiderFunnel in partnership are on pace to run over 500 A/B tests this year and more than double their revenue yet again.

Read the full case study

Learn more about how WiderFunnel integrated with DMV.org to create a testing powerhouse that got results. Read the optimization and personalization case study here.

The post Do A/B test results fade over time? Surprising results from three years of data. appeared first on WiderFunnel.

No comments:

Post a Comment