Over a century of combined SEO experience, and yet, we didn’t get it right!
We run SEO tests with our tool SplitSignal to determine which changes are worth keeping and which were just a shot in the dark that didn’t pan out.
But, to make it more fun and maybe more useful for our readers, we also let our communities on LinkedIn, Twitter, and Facebook participate by taking a guess as to how a certain change would perform.
Most of the time, we have to admit that the experts truly know what they are talking about. However, for these 10 tests, those who got it right were in the very small minority. In fact, for a few of these, literally 0 out of over 100 SEO experts could accurately predict the result.
X Times Everyone Was Wrong
Now, if you are new to SEO, or if you are a client that is insecure about hiring SEOs after reading this, you should bear in mind that these tests are extremely difficult. To accurately predict what will happen, one has to expand their focus over the entirety of what the testing website represents.
Two separate websites are not very likely to react exactly the same to a change. An SEO trying to guess one of these has a very tall order, and they usually don’t dedicate hours to investigating the website to give us an accurate guess. Instead, they see our polls as what they truly are—a game.
Bear in mind—This list starts with the highest guess rate, and at the bottom of the list, you will find the test results nobody could predict
To open the list, we want to start with a peculiar example. Sure, it has fewer correct guesses than #9, but it stumped our SEO experts and followers in a way that most people didn’t even want to try and guess.
In this poll, 39.4% of people voted that they just wanted to see the results. Out of the remaining crowd, another 40% expected the change to make either a positive or a negative change.
The observed increase to the pages was just over 4%, which is nothing to scoff at, but then again, now you can probably see why this is the first entry on the list.
However, to confirm any change and attribute it to our test, we are always looking for a confidence level of at least 95%. Coupled with a relatively low increase of just 4%, we can deduce that this test brought no significant change. Read here on how to read the SplitSignal test results.
What was it that stumped the experts? It’s simple, most of us don’t play with emojis. Some people view it as childish, and some aren’t ready to experiment with them. After all, we all tend to stick to safe strategies.
That doesn’t mean that there isn’t something we can take away from this test. So-called “no change” results are a good learning opportunity for SEOs.
What we can say from this is that you should probably run a test or two and see whether or not it works for your website. You might be surprised!
9. Moving the Description of Category Pages
Over the years, we’ve seen a lot of SEO experts claim that category content should either be on the top or on the bottom of the page. So, we tested it.
In this case, only 31.6% of people figured out that these changes could have a negative effect. Over 40%, however, believed in the hype.
Is this a final “debunk” of the theory? Of course not. It’s SEO, but it shows that we need a lot more testing before we make big claims. If the result was significantly positive, we could say, “Myth kinda confirmed.” But, as it is, we just want to do more tests.
This entry is very different compared to other ones for a very simple reason—the majority of people didn’t vote for the opposite result. Instead, 66.1% of people voted that there is essentially no difference between the two.
At the same time, only 24.8% of SEO experts predicted that you should use dashes instead of pipes for SEO.
In essence, for this test, we used a website that bought into the theory that pipes are superior because they take fewer pixels in terms of width. As we know, Google only gives your titles a certain number of pixels, so separating your titles with pipes, in theory, makes sense.
However, we can see that users simply prefer dashes. Dashes make sense. They are used in “real life”, and pipes just aren’t on the same level in terms of public perception.
By delving deeper into the analysis of this test result, we can see that the number of impressions didn’t change at all. So the 9% increase we got comes solely from the fact that people prefer clicking on titles written with dashes as opposed to those that used pipes.
Ok, when we name it like that, it feels kind of obvious. But that could just be the use of the word “just.” However, when we presented this test to everyone, almost 55% of people agreed that this version of the Meta Description is superior to the original one.
However, as our test shows, it didn’t really work out all too well. The website experienced a drop in organic clicks of 4.7%. A fact that was only predicted by 23.8% of SEO Experts who “played the game.”
While the test results weren’t positive, they definitely show how important meta descriptions are. And yes, a lot of SEOs still swear that they are basically irrelevant. Turns out—your Meta Description is one of the first things searchers see. And, if they don’t like it, they just might decide not to click on your page.
6. Adding the Review Ratings and the Star Emoji to the Titles of Travel Pages
With this test, we are slowly moving into the territory of tests that are super hard to predict. In this case, only one in 8 SEO experts guessed that the results would be negative.
To be clear, emojis are a controversial topic in SEO. Some experts say that they seem unprofessional, while others swear by their effectiveness. Apparently, it all depends (who would’ve guessed).
In this case, it depends on who your audience is. You shouldn’t expect to lose ratings for adding emojis to your titles. But, you have to be capable of predicting the behavior of your target audience.
Also, since we want to be fair with our “players,”—the results could be brought into question. Click on this if you want to read more about adding emojis to your title.
5. Title Case vs Sentence Case for Titles
Ok, while we feel like we have to use this test in the article, we have to admit that the low success rate of the guesses is more than partially our fault. When we were making the poll on Twitter, we could have done it in a clearer way. We have to assume that at least some of our followers didn’t quite understand the “assignment.”
But, even with that in mind, we’ve had quite a number of experts comment and send their opinions to our “DMs.” And, yeah, most people thought that the sentence case made more sense. Seems like SEOs aren’t always fans of the infamous “Title Case.”
Unfortunately, changing the title from Title Case to sentence case gave us a slight drop in organic traffic. So, the old rule of “if it’s easier to scan, it’s probably better” seems to hold true.
Ok, by now, we can all agree that a well-written Meta Description is very important. And one of the elements that you should probably include in your meta descriptions is a call to action.
However, we ran this test to see if we could make a general change across the website in the same way and add a call to action that way. While it won’t give us customized descriptions for each page, it will provide us with a CTA in hopes of getting the searchers to click more.
Unfortunately, it turned out it’s just not that simple. We expected a positive result ourselves, which puts it in the 95% of people who got this answer wrong.
Not only did the results dropdown. But this way of introducing CTAs in the meta description tanked the organic clicks by almost 20%.
Going Down to the 0%-ers
The top three polls for this list are truly something special. Not only did everyone (ourselves included) guess wrong, but we were all so confident about what the results would show. Unfortunately, the following tests have shown that SEO is much harder than anyone thinks.
For the first 0% test, we have one that had us scratching our heads. We all predicted that the results would either be positive (92.9% of people) or that there would be no significant change (7.1%).
Unfortunately, our test has concluded that this change led to a drop in organic clicks of 4.6%. Now, a drop that’s under 5% is nothing to panic about.
Those things happen and are easily fixed or survived. But still, why didn’t we all get it right this time?
Well, while we could expect that these titles would get more people to click on the page, we forgot about the SERP rankings. Namely, over the 3 weeks, this test was running, there was some decline in ranking for many of these pages.
Also, overoptimizing online stores for location-based searches could lead to you appearing in searches for the wrong audiences. Not what you want.
Some tests bring about unexpected results simply because the hypothesis is unnecessarily complicated. In this case, we have tested using adding CSS Font-weight treatment to the main H1 for pages of an ecommerce website.
And, naturally, most people simply didn’t think it mattered. Namely, 71.8% of our voters expected no significant change in either direction. We, on the other hand, were in the bunch of 28.2% that expected the performance to improve.
However, what nobody expected was a 7% drop in just a few weeks. And unfortunately, that is what actually happened.
Why? Well, we don’t really know, we noticed that the rankings dropped, but this could have just been bad timing with a competitor taking over for a few weeks.
This client is one of our favorite ones. After all, who doesn’t like pizza? And, for their test, we decided to re-run a test we had done earlier, just a bit differently. We simply added a call to action to the title tags of their pages.
And, it was a very simple call to action—right before the title, we added the word “Order”.
We can tell you. We definitely expected a decently positive result for this. And we weren’t the only ones 85.7% of people who voted agreed with us. The rest just thought that the change wouldn’t be significant.
As you probably caught on to the trend of these list entries, exactly 0% of people guessed that it would plummet the results. In fact, over the next three weeks, the organic clicks for the test group of pages went down by as much as 14.9%. That is a significant potential for a loss right there. Thankfully, we organized an easily reversible test, and the client didn’t have to suffer a loss.
As to why it happened? We will let you try to figure this one out. We have our analysis in the original article that contains the test, and you are more than welcome to read it. But, we are also looking forward to hearing your opinion on the matter.
The Importance of This Article
This article wasn’t made to make fun of SEO experts or to make anyone look bad. Instead, it is here because it shows how important it is to test your theories.
As you can see, we’ve had people very confidently predicting incorrect results time and time again. To be fair, in most of our tests, SEO experts get it right.
But, the existence of these tests is proof that you can’t take anything for granted.
If you want to make large-scale changes to your client’s website, you should figure out a way to test them out first. And for that, we would recommend trying out our new tool SplitSignal.
Contact our SplitSignal team and get started with your own SEO split tests.