30
Apr

Everything Changes—Including Your Test Results

April 30th, 2007 by Penny Hagerman


Consumers really are a fickle bunch. Just when you think youve got them figured out, you find out you were wrong—and now you have to start testing the waters all over again to find a way to reach them. (Uh-hum…pardon me while I step up onto my soap box.) soapbox.jpg

What am I ranting about this time? Well, remember my post back in February about the importance of A/B testing as far as the success of particular ad campaigns is concerned? Well, I was pondering that issue further last Friday as coworker Cassie and I were examining recent A/B test results. Here at InsureMe, our PPC team often finds itself frustrated when something we tried last year—or even just a few months ago—worked well then but suddenly isnt working well now. This led me to do some research on what might cause previous test results to become invalid, so I thought Id share. (Maybe this will help you avoid some frustration, too!)


I often find Dr. Ralph Wilsons Web Marketing Today a great resource in Web trends and analysis. His latest newsletter lists these causes for invalidation of test results over time, which totally make sense in my book:

  • The constantly evolving market. What causes your product to sell today probably will not work tomorrow. New technologies, changing business models and introductory services appear every day that change consumer preferences.
  • The improving intelligence of Internet users. As consumers get better and smarter at navigating the Web daily, this can have a huge effect on usability items such as buttons, forms and complex systems.
  • The changing competitive landscape. As you change, so do your competitors—and not working constantly to keep up means losing money!
  • The reality of seasonal effects. This point is felt heavily in the insurance industry, as we all experience every year when third quarter rolls around. It takes a lot of work to push past times of seasonally low traffic and find a unique message that works just right at those times. And when everyone is doing it at once, it can be more than a simple challenge.

This resource suggests testing and re-testing as often as every month or two to get valid test results. As you probably know, I recommend even more frequent testing. But by knowing what might cause your test results to vary so much in such a short timespan, you may be able to relax a little and avoid some of the frustration that goes along with A/B testing.

Whatever you do, keep testing and keep on digging…were right in there with you! :)

Related posts:

  1. A/B Testing Made Easy A/B testing is a great way to refine your PPC...
  2. A/B Testing In the early days of paid search, people churned...
  3. Landing Page Testing - Your Key to Higher Conversion If you aren’t testing landing page variations for your PPC...
  4. Dayparting Features Coming to AdWords Next Month! Jennifer Slegg at Search Engine Watch reports that new dayparting...
  5. Affiliate Marketing Getting You Down? Clean Up Your Act With Some Quick & Easy...

Related posts brought to you by Yet Another Related Posts Plugin.

Share & Enjoy:
  • Digg!
  • Del.icio.us
  • StumbleUpon!
  • Thumb This!

Leave a Reply