Hi all - I’m having a weird issue. Google Search Console is showing me that the majority of pages at my website are set to “noindex”. In WeWeb, none of the public pages have the noindex toggle switched on. I even added an explicit meta tag on each page telling crawlers to index the site, but that didn’t help. I looked at the source of the home page which contains an iframe with a noindex tag, but the home page seems to be indexing fine. Pages such as the contact page don’t have any noindex tags, but two tags saying it should be indexed: Contact Us - Dak - Yet google still says this page shows as noindex. Is this a bug or am I doing something wrong?
There is no meta tag telling crawlers to index a page, indexing is the default behaviour. The robots meta tags only ask to not index, and you trust the crawler to respect your wish.
In the meta tag in your homepage
<meta content="index, follow" name="robots"> the attribute content it’s not a valid value. See more details here Robots Meta Tags Specifications | Google Search Central | Documentation | Google Developers.
Other things can get in your way when handling crawlers:
- Check your sitemap, I think you need to have one address on each line, see here Build and Submit a Sitemap | Google Search Central | Documentation | Google Developers.
- Check that your robots.txt is valid (https://www.google.com/webmasters/tools/robots-testing-tool)
My home page is not one of the pages that I’m having an issue with.
WeWeb creates a sitemap automatically as well as the robots.txt
I didn’t see any noindex tag on the contact us page, nor does any of my SEO tools.
Maybe Google Search Console is telling you that it won’t index the iframe, as an iframe is indeed another page embedded into yours.
Anyway, you shouldn’t have any issues as all your website pages are set to doindex, dofollow.
I’ve just been told by our CTO that the pages on *.weweb-preview.io have a noindex tags, but it’s removed when you’re on your own domain.
I appreciate you taking time to help me with this. I would agree, I shouldn’t have any issues, but Google Search Console says every single page except the root domain is set to noindex.
As of tonight, the following pages show as indexed, but the majority still show as set to noindex.