Crawled a live WordPress/Elementor chiropractic practice site (14...

Nav-and-footer link-graph harvesting from a single homepage fetch as the primary URL inventory method, cross-checked with a site: query, then targeted fetches of only the template-divergent pages. Treated crawl artifacts as diagnostic signal rather than noise: map iframe query strings, canonical vs resolved destination URL, and meta modified_time were each used to surface data conflicts and content staleness the client had not flagged. Explicitly bounded the crawl's blind spots in the deliverable (orphaned pages, noindexed pages, backlinked PDFs) with instructions to reconcile against sitemap_index.xml, GA4 landing pages, Search Console pages, and a backlink export before finalizing the redirect map.

Outcome: success. Nav-and-footer link-graph harvesting from a single homepage fetch as the primary URL inventory method, cross-checked with a site: query, then targeted fetches of only the template-divergent pages. Tre

Applies when: Crawled a live WordPress/Elementor chiropractic practice site (14 indexable pages, no blog) and produced a same-domain redesign migration checklist. Method: fetched the homepage first to harvest the f ## Known Failure Direct fetch of /sitemap_index.xml and /robots.txt was blocked because the fetch tool only permits URLs already present in prior search or fetch results, and neither is linked from the rendered homepage. Worked around it with a site: search, but that returns only indexed pages — orphaned, noindexed, and paginated URLs stay invisible. Also, /contact-us/ returned a JavaScript-required interstitial instead of page content, so the contact form's fields and handler could not be inspected; this was itself recorded as a finding (the page has no crawlable NAP content independent of the form script). Lesson: for migration crawls, a headless crawler or direct sitemap access is required for a complete inventory — a fetch-tool crawl produces a confident-looking but incomplete URL list, which is the exact failure mode that causes missed 301s at launch.