5 Pitfalls You Want to Avoid When Testing on Your Site

Posted on Nov 25, 2019 - By Leesa Love

No business is perfect, which is why it’s important to stay on your toes and always be on the lookout for ways you can make improvements. When it comes to online business, this is where A/B testing comes into play. A/B testing involves changing an element on your website and comparing the data from before and after, in order to see if the changes had any effects on your conversion rates.

When employed correctly, this can become a powerful tool for optimizing both your website and your business. However, there are several opportunities for mistakes along the way, which can lead to skewed or completely inaccurate and unhelpful results. Aside from not testing at all, factors like testing during the wrong time or changing too many variables at once can put a wrench in your data. Once you know about the potential mistakes you may encounter, you can begin to form an effective testing plan.

1. Testing the Wrong Elements

The most vital step when creating your testing plan is deciding exactly which elements to test. This requires knowing which metrics you want to improve, and how the elements on your page affect those metrics. Otherwise, you risk testing insignificant elements or changing aspects that do not affect the particular metric you want to focus on. Some of the best pages to test include your:

  • Product page.
  • Home page.
  • Contact page.

These pages will likely see the most traffic, which means the elements on them are vital to your conversion rates. Ultimately, you need to decide on your goals for testing. If you want to get more users to buy your product, make sure to test the call to action on your product page rather than the images on your home page.

2. Testing Multiple Variables Together

When choosing which elements of your site to test, it’s also important to remember that rule we all learned in science class: only test one variable at a time. While you might think a complete website overhaul will save time, in reality, making several changes at once will muddy your results.

This makes it impossible to identify which changes worked, which stayed the same and which actually had negative effects. Then even if positive results did occur, they may be diminished or negated altogether.

To ensure you get clear and worthwhile results, choose one aspect to test against your control – the site as it was originally – and go from there. To stay organized, you can create a plan that divides your testing into phases. That way, you can ensure that all of your testing efforts culminate in a fully-optimized website.

3. Testing During the Wrong Time

A major factor to consider when testing is how the traffic from your website changes from one season, or even one day, to the next. Every business is likely to have times when traffic is at its lowest, and it’s worth noting when yours are in order to choose the best time to test.

For the most accurate results, you want to make sure you are comparing data from similar time periods. In other words, if your control version is from a period of high traffic, make sure you test during a high traffic period as well. Similarly, if you pull data for your control starting on a Monday, it’s a good idea to start your test on a Monday as well.

4. Closing the Test Too Early

Along with testing during the right time, you also need to be sure that you test for the right length of time. This means setting a goal for the amount of conversions you want to reach before ending each test.

You may be tempted to cut your test short if you see that the changes you made are already outperforming what you had before. However, in doing so you risk getting skewed, underdeveloped results, or even a false positive. False positives are changes that seem to perform very well in the initial stages but taper off in the long run.

The base rule for testing is that you want to achieve a minimum of 1,000 conversions before forming a conclusion. Another good rule of thumb to follow is running your test for a minimum of 2 weeks, along with reaching 1,000 conversions.

5. Forgetting Your Brand

While testing is an invaluable tool for your website, it’s important to know when to draw the line. Otherwise, you can fall into the trap of making changes just to test them, even if the changes don’t necessarily align with your brand.

Your brand is the most important element your online business has to offer. Without it, your work would be lost in a sea of generic websites and products. Not only that, changing and testing too many elements can also detract from the overall cohesion of your website, leaving a cold, disjointed series of webpages behind.

To combat this mistake, it’s important to keep your branding in mind when planning out your tests. If your brand is associated with certain colors, try to stick with those same colors, especially on a page-by-page basis. If there is a specific tone you want to convey, avoid using language that contradicts that tone.

Following your own brand guidelines can help you maintain consistency throughout your site. Ultimately, this will ensure that any changes you do make have a worthwhile and positive impact on your business.