Unexpected crawl results
Sometimes a page is flagged in a way that looks wrong, a missing title you know is there, or a redirect you did not expect. Almost always, the crawler is faithfully reporting what it actually fetched.
Start with the evidence
Open the page's detail view. It shows the exact title, status code, tags, and content the crawler received for that URL. Comparing that against what you expect usually explains the result in a few seconds, because it shows what the crawler saw, not what you assume is there.
Common explanations
| What you see | Likely cause | What to do |
|---|---|---|
| An old title or content | The crawler fetched a cached version | Re-crawl, and check your caching |
| An unexpected redirect | The URL redirects to another | Follow the redirect; link to the final URL |
| Missing content or links | Built with JavaScript, not in the HTML | Turn on rendering and re-crawl |
| Page marked not indexable | A noindex, robots rule, or canonical | See indexability |
| Your recent edit is not reflected | The page was crawled before you changed it | Re-crawl to pick up the change |
Re-check a single page
After a change, use quick check to re-fetch one URL immediately and confirm the page now looks the way you intended, without re-crawling the whole site.
Compare crawls if something changed
If a result changed unexpectedly between crawls, crawl history shows which crawl it changed in, which narrows down what caused it, often a deploy or content edit.
Trust the evidence
The crawler reports what it fetched from the URL, the same way a search engine would. If a result surprises you, the detail view almost always reveals a real cause: a cache, a redirect, a robots rule, or a JavaScript-built page. That is useful information, because whatever confuses the crawler will usually confuse a search engine too.
Frequently asked questions
Why is my page flagged with something I know I fixed?
The page was probably crawled before your change, or a cached version was served. Re-crawl, or run a quick check on that URL.
The crawler shows an old version of my page. Why?
A cache served an older copy. Re-crawl, and review your caching so search engines are not served stale pages either.
A page redirects unexpectedly. What now?
Open the detail view to see the redirect target, then point your internal links at the final URL.
The result still looks wrong after re-crawling. What should I check?
Whether the content is built with JavaScript (turn on rendering), and whether a robots rule or canonical is affecting the page.