The noindex Nobody Meant to Ship
An accidental noindex is the most expensive one-line bug in this business, and it hides in two places, one of which never appears in view-source.
A noindex directive removes a page from search entirely, and the overwhelming majority of the ones in the wild were not chosen by anybody. They arrive during a staging build, get applied by a plugin to a whole content type, or come along with a template somebody copied. Then they sit there.
It hides in two places
The obvious one is the meta tag in the document head:
<meta name="robots" content="noindex, follow">The one that gets missed is the HTTP response header:
X-Robots-Tag: noindexThat second one never appears in view-source. It never appears in your CMS. It is set by the server, or a proxy, or a CDN rule, and the only way to see it is to look at the response headers:
curl -sI https://example.com/your-page/ | grep -i x-robotsQuickRankAI checks both, which is why it sometimes reports a noindex on a page whose markup looks perfectly clean.
The staging launch, which is how most of them happen
The build goes up on a staging domain. Somebody adds a site-wide noindex, correctly, because a half-finished site should not be indexed. Launch day arrives, the domain changes, the content gets a final review, and the noindex is on a settings screen nobody opened.
Two months later somebody asks why the new site gets no search traffic.
The reason this survives so long is that everything looks fine. The pages load. The content is good. Analytics shows visitors, because direct and referral traffic still works. The only symptom is an absence, and absences are hard to notice.
The plugin version
Common on WordPress. An SEO plugin has settings for whether each content type should be indexed, and the defaults are not always what you would choose. Custom post types in particular tend to arrive set to noindex, so an entire section of a site quietly stays out.
Check the archive and taxonomy settings too. A site where the posts are indexed but every category page is not has removed its own internal structure from consideration.
Do not use robots.txt to keep a page out
This is the mistake that follows the first one. Somebody wants a page gone, so they add a Disallow line to robots.txt. The crawler now cannot fetch the page, which means it cannot see the noindex tag on it, which means the URL can still appear in results based on links pointing at it.
Blocking prevents fetching. It does not prevent indexing. If you want a page out, let the crawler in and let it read the noindex.
How to confirm the repair landed
Check both locations again after the change, because the two are set in different systems and fixing one does not touch the other:
curl -sI https://example.com/your-page/ | grep -i x-robots
curl -s https://example.com/your-page/ | grep -i 'name="robots"'Both should come back empty. Then re-run the inspection, which names which of the two locations it found the directive in.
Run this against your own page. The inspection reports whether the repair described here is actually needed, and what else is sitting next to it.