How to fix pages that can’t be crawled without a specific reason.
How to fix pages that can’t be crawled without a specific reason. This can be an important issue. Very often it’s something that you don’t need to worry about, and I will show you why.
So if you click through to the summary page, you’ll notice that in this first column here, there are only four here, four URLs, three of them.
I will highlight the three that we’re talking about, have the word tag in the URL. If you’re using WordPress and you see these URLs with the word tag or with the word category in them, those are okay.
You actually don’t want those pages to be crawled. In effect, when we’re setting up SEO on a WordPress site, the default settings, we will set these to be not indexed.
So they will actually, that will cause them to appear here, in this, era as pages that can’t be crawled.
When I see this both for tag pages and category pages, what I do is I hide these issues because this is actually something that this is the effect that we want to have happened.
So you can hide an issue by clicking a check box next to the page or pages that you want to hide.
And then click hide selected. You can do the same thing. One URL, a time over here on the right, by clicking this little eyeball with a, with a line through it.
But if you hide these pages, what that will do wills is remove them from this, the SEMrush audit. And if you were to rerun this scan in the future, it will ignore those, those pages that do leave.
Occasionally. Sometimes there are some other pages that can’t be crawled. Like there is one here. And if we go ahead and try to click to visit this page, to see what the issue might be about, we get a 404.
So most likely this link will be detailed in one of the other earlier issues, internal links that are broken and pages that return a 4xx error.
So in this case, you can ignore this. Let me find that page again. You can ignore this, don’t hide it because you want to make sure that this is removed when you resolve those other areas.
So just leave this as is, and you can move on to the next issue.