Update: Yoast is still doing this in their latest version released after this article was published. Update: January 9th: Updated with Yoast comments Yoast SEO is one of the most popular WordPress Plugins with over 5 million sites using their software. While Yoast provides a template to fill in the meta description field, it also generates a sitemap. Sitemaps are extremely important for Google to be able to see your site and be able to decide what it should index and what it shouldn’t index.

Yoast is essentially telling Google to not look at your sitemap. This makes it much harder for Google to find links on your site. This started around July 5th, 2018 and cost one of our clients tourslosangeles.com over 100,000 dollars. We made a free fix with our plugin Airtight Security so you can continue to use Yoast and Google can see your sitemap. This means Google can find your site a lot easier. All you have to do to get our plugin is go to your WordPress site and log in. Then go to your plugins and click add new. After clicking add new search for airtight security and download our plugin and activate it. It will automatically fix the issue without any intervention by you.

Does A sitemap guarantee Google Will Index My Links?

No, it does not and Moz pointed this out in an extremely honest and detailed piece on xml sitemaps they wrote. What a sitemap does is let Google see you have content, rank the content using their algorithm and decide if the content should be included on Google and how valuable it is. The reason this is so important is that serious companies will be writing content that is of value every single day and expect their sitemap to help Google find it quickly instead of having to manually go to Google Webmasters and enter each link manually.

What Happens When Google Sees my Sitemap With Yoast?

Here are screenshots of what Google sees when a site is using Yoast.

Is that not enough proof for you that Yoast is blocking your sitemap from being indexed? This is what happens when you try to index a Yoast sitemap without going to the live test view, which is what we displayed above.

Prove it is Yoast Causing this!

Our team initially thought the site was infected with malware, but we found no malware. We finally found the code in yoast making it not possible to index the sitemap.

When we turn on our program airtight security the no-index header created by Yoasts sitemaps are removed.

When we turn off airtight security and use a chrome extension that let’s you see http headers, you can visually see the no-index header on the sitemap.

YOAST SEO Premium Fixes This, Right?

Since we started noticing Yoast pushing their premium version what seemed to be more than usual, we thought Yoast  had  patched  the  issue in their free version if you upgraded to their paid version. We were surprised when we bought the program for analysis to learn that the premium version also suffers from the same no-index issue. The only difference is their premium support can help you with anything SEO related. So, if you use Yoast SEO Premium, you are still affected by this issue.

So, how can you fix Yoast from harming your site? Simple use our plugin Airtight Security. How can you check to see if any other software is harming your site? If you are a premium airtight security user, we will take a look at a plugin you’re concerned about for no additional cost. You also get access to a powerful WordPress security scanner that detects exploits. We only scan 50 plugins and themes combined at a time as no site designed properly should have more than that. As we expected when we released our product people wanted to use it to scan all 54 thousand WordPress plugins at once. Our software is not currently intended for that purpose, which is why we limit it to a combined amount of 50 plugins and themes, since sites shouldn’t have more than that. 

Leafly is a website that suffered a massive depletion of users, yet magically jumped back up and are doing great. So how is this possible if they use Yoast? Because they abandoned WordPress when customers started dropping off as the chart shows in September and October. We know they used Yoast thanks to this site that tracks users of Yoast.

Think about that for a moment, companies are having to spend money to switch platforms because of Yoast, which is in the popular plugins section, which in most users minds is a recommendation from WordPress on what to use.

Let’s look at some sites still using WordPress and Yoast.

Cheatsheet.com has had a major decrease at the same time of all the other sites, though it is worth noting they currently use Yoast SEO Premium. It is of no surprise to us that cheatsheet.com has had a massive decrease of visitors to their site since Yoast put a no-index on the sitemap. While we could compare millions of  sites this helps give you a visual of the issue that Yoast caused. It is also important to note that around the time this issue started, someone filed a github complaint that they noticed the rss feeds were not being indexed.

Does Yoast Know About the No-index issue?

Yoast considers this a feature, not a bug or an issue. Their employee jono-alderson addressed the feature. Jono said on August 26th, 2018 when this started about the RSS Feed issue that ”
From an SEO perspective, it’s generally worthwhile preventing Google from indexing RSS feeds via the x-robots HTTP header. Note for reference, that when this has a value of noindex, that doesn’t prevent Google from accessing or consuming the information – just from indexing it.
That aside, we should definitely add the ability to filter this value, so that we can be podcast-friendly. Easy fix! “

Let’s break that down into easily, consumable pieces. First they claim no-index does not stop Google from “accessing or consuming the information”.

Google and any other search engine goes to a link, checks the headers and if the header says no-index, they go away, since that is what no-index means. So, from the statement by the Yoast employee since they can access the site and be told to leave, that is fine. What we haven’t mentioned is that Yoast uses noindex, follow which is very misleading and we clear up how this messes up your site in the words of Google’s Webmaster Round Table John Mueller who is in Charge of Webmaster Trends Analyst at Google.

Let’s be very clear, they do not consume the information on your sitemap, meaning they can not use it they ignore it per Yoast’s instructions. Google explains why they ignore it in the next paragraph. Also, one person pointed out they are violating Google’s rules on RSS feeds for podcasts. 

Google who is an industry leader in SEO says the exact opposite about Yoast’s noindex, follow technique in their Google SEO round table. John Mueller who is the Webmaster Trends Analyst at Google essentially said that if you put noindex, follow they won’t index that page or follow any of the links.

John Mueller explained how Google handles the exact type of code that Yoast is using in a 2017 Google webmaster round table.
“It’s tricky with noindex, which I think is something of a misconception in general within the SEO community. With a noindex and follow it’s still the case that we see the noindex. In the first step we say ‘okay you don’t want this page shown in the search results’. We’ll still keep it in our index, we just won’t show it and then we can follow those links.”

That part seems to support Yoast’s claim, but the next paragraph debunks Yoasts claim.

“If we see the noindex there for longer than we think this page really doesn’t want to be used in search so we will remove it completely. And then we won’t follow the links anyway. So noindex and follow is essentially the same as a noindex, nofollow. There’s no really big difference there in the long run. “

So, what John Mueller is saying is that if you put noindex, follow on a page for a few days they would still follow the links and add the content into Google, like Yoast claims. However, if the noindex, follow stays on the same page for say a few weeks they will ignore that page and all the links on it. So, in short Google is addressing the exact code Yoast is using months before Yoast released it. Since the sitemap made by Yoast never removes the no-index header Google now ignores the sitemap and all of it’s links.

This disproves everything that Yoast claims and is why your site is having so much trouble. When it comes to how search engine optimization works for Google, I listen to Google, not Yoast.

Since Yoast has not been able to implement a fix in their plugin you can use our fix by logging into your WordPress site going to plugins -> add new and search for Airtight Security. Activate the plugin and the issue is fixed.

Joost De Valk from Yoast commented and doubled down, almost as if he did not review what Google said or the charts from his current and previous customers. Here is his quote “

I’m sorry but this just isn’t true AT ALL. XML sitemaps aren’t indexed like normal webpages. Or at least: they shouldn’t be. Google reads them differently and doesn’t obey the indexing directives when it ingests them like that. Sometimes they get linked to on the web as well. At that point, Google *does* index them normally, and follows indexing directives. So we set the noindex header on the XML sitemaps so as to make it impossible for XML sitemaps to start showing up in search results. They do *not* prevent Google from using them for what they’re important for: getting URLs into the index.

We talk regularly to Google and are in fact looking at making XML sitemaps better for everyone together with them, so I’m 100% certain of this.”

None of what he said is supported by information from Google, charts showing damage as we showed above or even from Yoast’s customers.