Prior to 2006, testing wasn’t discussed much as a means to optimization. Today, there is so much emphasis on testing, to the extent many experts believe that without it, all optimization efforts will fail. When did this change happen, and why? And is conversion optimization possible with testing or not?

Testing has always been a staple ingredient in our optimization efforts. It allows us to continuously measure improvements, vary elements and combinations, and address the constantly changing needs of the customers quickly and effectively. We have had instances where testing could not be used with clients because the testing software was not compatible with their platform, etc.

Additionally, prior to 2006, testing was a last touch ingredient within our projects (i.e. it was not integrated with every optimization recommendation as it is now), which made it less important to the project, the client, and many times, the results.

This issue comes up when a client comes to us and says, “I can’t deploy testing.” Whatever reason it may be, as experts we need to work around all obstacles – or do we? Should we abandon a project that does not allow for testing, or should we optimize as we did before 2006?

The reality is that the online space is constantly changing, which means companies need to adapt accordingly, testing or not. However, this does make our jobs, as optimization experts, a lot more difficult. Testing eases the optimization process tremendously because within one session you can test many different elements at one time. You can run the new design or new elements against each other and measure the impact of the changes. Without testing, you end up having to do a lot of complex math to decide if the new design is better than the old design.

How should you proceed with optimization when you simply CANNOT test?

1. Collecting Market Information:

It’s our preference that you actually develop personas and conduct usability tests based on the personas and customer profiles created. If you do not have the resources or experience to do that, then we suggest that you conduct customer satisfaction surveys, demographical collection surveys, and untargeted usability testing. The more information you gather about your market, the easier it is to consider areas that require optimization. Moreover, you begin to understand what works and what doesn’t work with your customer base.

2. Analytics:

The wealth of information and analysis that can be done on your website through analytics is limitless. Many paid packages such as Omniture and Coremetrics provide detailed analysis of the website’s page by page performance, highlighting areas of possible improvement. Google Analytics, a free tool, provides useful information that can be customized and segmented to reveal poorly performing areas throughout the website.

Doing this type of deep analysis is worthwhile. If you are not going to spend money on getting the site optimized by professionals, our advice would be to get analytics experts to give you an in-depth analysis on your website’s performance. That way you have the information and can act upon it accordingly.

There are some interesting theories that we’ve developed such as the bounce around effect, and the product page effectiveness index (PPEI), but we’ll explain those in a later a post. These theories can help pinpoint areas that require immediate improvement in order to see a greater ROI. So although testing could help optimize those areas faster and more effectively, improving upon them without testing is very possible if the analysis is done correctly.

3. Incremental Changes:

If you cannot test, then simply do not make too many changes at once. There is no way to measure the effectiveness of each specific change without the tools necessary. So list the areas that need improvement based on market research and analytics. Prioritize areas based on (and now this is where your optimization experience comes in handy) what you feel will have the greatest impact to your ROI.

For example: I may have listed the following areas to improve upon:

• The design of the checkout page
• Product page layout
• Continuity between PPC ad and top landing page (by $ index)

Each of these poses a level of importance, but which area do you start with? Starting all together will not let you trace where the spike or drop in conversion rate resulted from. Based on the analysis you do, always assign each page a dollar index, that way you can sort which offers the highest $ index. Also incorporate consider the LOE each change will take because allocating resources to a project is time and money. 

Possibilities

Optimizing without testing is possible. It will take a longer time and more patience, but leaving your sign stagnant is detrimental to your growth and success as an ecommerce company. Have you had to deal directly with optimizing without testing? What types of challenges have you faced?