Most traffic losses I trace back to a technical fault turn out to be small. A noindex tag that rode along with a release, a robots.txt rule nobody remembers adding, a canonical pointing at the wrong path. Google never sends an alert for these. Pages just slip out of the index, and the first sign is usually a dip on a revenue report weeks later.
A good audit works like a post-mortem done before the damage. You find the fault, work out which pages and how much money it touches, and fix it at the template level so it stays fixed. The checklist below follows the order I use on real sites, from access problems to polish.
Three free or low-cost tools carry most of the work: Google Search Console, Screaming Frog SEO Spider, and PageSpeed Insights. They show how Google fetches and renders your pages, and that is the only view of your site that affects rankings.
1. Check Whether Google Can Crawl and Index Your Important Pages
Every other SEO effort depends on indexation. A page Google cannot fetch or keep will never rank, however strong the copy is.
I begin in the Page Indexing report and go straight to the Not Indexed table. Two statuses tell you the most. Discovered – currently not indexed means Google found the URL but has not crawled it, which often points to a slow server or low perceived value. Crawled – currently not indexed means Google visited the page and decided against keeping it. That one usually comes down to thin content, near-duplicates, or rendering trouble.
Next, open robots.txt at the root of the domain. Look for a stray Disallow: / or blocked folders holding JavaScript and CSS. If Googlebot cannot load those files, it renders a broken version of the page and may index it that way. Also remember that robots.txt only controls crawling. Google’s robots.txt documentation is clear that a blocked URL can still appear in results if other pages link to it.
Then look for noindex in two places: the HTML source and the HTTP headers. A robots meta tag set to noindex inside a production template can empty a whole page group within days. The usual cause is a staging setting copied into a live release. The same directive can hide in an X-Robots-Tag response header, where nobody sees it unless they inspect the request. Google covers both methods in its guide to blocking indexing.
For any high-value URL, run a live test in the URL Inspection tool. It returns the rendered DOM, the status code and a list of resources that failed to load, so you stop guessing what Google saw.
Keep perspective, though. Carts, staging environments, internal search results and admin panels belong outside the index, and seeing them listed as excluded means your setup is working.
Two terms are worth keeping apart. Crawlability asks whether Google can fetch the page. Indexability asks whether the fetched page clears Google’s bar on quality, canonicals, and policy.
2. Audit XML Sitemaps and Canonical URLs
A sitemap says which URLs you want crawled. A canonical tag says which version of a page counts as the main one. When the two agree, Google follows your lead. When they disagree, Google picks for itself.
Your sitemap should list only canonical URLs that return 200 OK. Remove redirects, error pages, parameter variants, and old HTTP addresses. Each file can hold 50,000 URLs or 50 MB uncompressed, according to Google’s sitemap guidelines, so bigger sites need a sitemap index file.
Check the canonical tag on every major template. Most pages should point to themselves. Google’s page on consolidating duplicate URLs covers the odd cases, such as cross-domain canonicals.
The mismatch I see most often is small. The sitemap lists a path without a trailing slash, while the canonical on that page includes one. Google now has two candidates and no instruction, and it may choose the version you did not want ranking.
Run through these normalization checks to confirm each page has one address:
- Send every HTTP request to HTTPS with a permanent 301 redirect.
- Choose www or non-www and redirect the other version.
- Use one trailing slash convention in all internal links.
- Handle UTM and tracking parameters with canonical tags, since Google retired its URL Parameters tool in April 2022.
3. Find Broken URLs, Redirect Problems, and Server Errors
Broken URLs cost you in two ways. Crawlers waste requests on them, and visitors who hit them rarely come back. A full crawl catches what manual clicking misses.
Run a complete crawl in Screaming Frog and filter for internal 404 responses. Fix the link that points to the dead page before anything else. Redirecting the dead URL treats the symptom while the broken link keeps sending people there.
Redirect chains deserve a look as well. Each hop adds delay, and Googlebot follows at most 10 hops before it gives up. Aim for exactly one hop between the old URL and its final destination. Chains tend to build up after migrations, when each redesign adds another layer on top of the last.
Server errors matter more than most teams assume. Occasional 5xx responses, especially 500 and 503, point to overloaded hosting or slow database calls. When they persist, Google reduces how fast it crawls the site, a behavior described in its crawl budget documentation.
Check Search Console for Soft 404 flags too. These are pages that return a 200 status while showing an empty template or a message saying the item is gone. Google’s soft 404 help page explains why they get treated as errors. The fix is to return a real 404 or 410.
Not every dead URL deserves a redirect. Redirect when the page has backlinks, a record of search traffic, or a genuinely equivalent replacement. Otherwise, let it return a 404, or a 410 Gone if the removal is deliberate. Sending dead product pages to the homepage is a shortcut that usually backfires, because Google reads it as a soft 404 anyway.
4. Review Site Architecture and Internal Linking
Architecture decides where link equity and crawl attention go. Pages that sit deep in the structure or have few internal links get less of both.
Start with depth. Landing pages that need four or five clicks from the homepage get crawled less often and carry less internal authority. Small sites should keep important pages within about three clicks. Large ecommerce sites cannot do that for every product, so they lean on strong category hierarchies and related-product modules.
Next, find orphan pages. These exist on the server and sometimes in the sitemap, yet no indexable page links to them. Compare a fresh crawl against your sitemap and analytics landing pages, and the orphans show up as the URLs missing from the crawl.
Body links beat menu links. A link inside a relevant paragraph tells Google what the target page is about, while a footer link repeated on every page says very little. It also helps readers, which is the better reason to do it.
A few features need extra scrutiny:
- Pagination should use standard anchor links, since script-only click events are invisible to crawlers.
- Faceted filters that stack query parameters need canonical tags or crawl controls to stop URL bloat.
- Mega menus should expose real anchor tags in the rendered HTML.
On crawl budget, many teams worry too early. Google says it matters mainly for sites with roughly one million unique pages that change weekly or 10,000 or more pages that change daily. A site below those numbers gets more from better structure and content than from budget tuning.
5. Test JavaScript Rendering and Mobile Experience
React, Angular, and Vue sites work fine in search when built carefully. The risk comes from how Google handles them. It crawls the raw HTML first and renders the page later, and anything that needs scripts to appear can be delayed or missed. Google’s JavaScript SEO basics describe the process.
Compare the raw response with the rendered HTML. Body text, navigation links, canonical tags, and structured data should be present in the first response or render without errors. One trap catches teams often. If the original HTML carries a noindex directive, Google may skip rendering entirely, so a script that tries to remove it later never gets the chance.
Mobile deserves equal attention because Google finished moving every site to mobile-first indexing in July 2024, per its mobile-first indexing documentation. The mobile version is the one that gets evaluated. Crawl with a mobile user agent and confirm that copy, links, schema, and media match the desktop page.
While you are there, check for usability problems on small screens:
- Overlapping or clipped text.
- Tap targets sitting too close together.
- Popups that cover the content on load.
Test before you assume. A JavaScript site that renders cleanly in URL Inspection and shows its content in the rendered HTML is fine. Also keep file size in mind. Googlebot reads only the first 15 MB of an HTML file, so pages stuffed with inline scripts can lose content past that mark.
6. Check Core Web Vitals and Page Speed
Speed problems rarely show up in a single metric. They show up as visitors who leave early and pages that sit just below the pass line. Google’s Core Web Vitals measure what real visitors experience, which makes them more honest than any lab score.
Three metrics decide whether a page passes:
- Largest Contentful Paint (LCP) tracks loading. The target is 2.5 seconds or less for the main content block.
- Interaction to Next Paint (INP) tracks responsiveness. Stay at 200 milliseconds or less for taps, clicks and key presses.
- Cumulative Layout Shift (CLS) tracks visual stability. Keep it at 0.1 or lower.
INP is the newest of the three. It replaced First Input Delay on March 12, 2024, as Google confirmed. Plenty of audit templates still list FID, so update yours before it goes to a client.
Slow LCP usually comes from four places: a sluggish server, an oversized hero image, render-blocking scripts and late-loading fonts. The cheapest win is often the fetchpriority attribute on the hero image, which tells the browser to grab it early. It is one line of code, and the gain is often visible.
INP trouble almost always traces back to JavaScript. Long tasks hold the browser’s main thread, so a tap waits until the work finishes. Split long tasks into smaller pieces and defer third-party scripts that do not need to run on load. Google’s INP guide shows how to find the slow interactions.
CLS tends to be the quickest fix. Add width and height attributes to images and video embeds, or use the CSS aspect-ratio property, so space is reserved before the file arrives. Late banners and ad slots cause many shifts, so give them fixed containers.
Use lab and field data together, since they answer different questions. PageSpeed Insights runs Lighthouse in controlled conditions, which makes bugs repeatable. Field data comes from the Chrome User Experience Report, covers a rolling 28-day window, and is judged at the 75th percentile. That means three out of four visits must meet the threshold for the page to pass.
When the two disagree, believe the field data. A page can look perfect in the lab and still struggle on a mid-range phone with a weak connection.
7. Review HTTPS, Security, and International SEO Settings
Security checks protect visitors and keep the site available. International checks make sure each audience lands on the right regional page.
Start with TLS certificates. Confirm each one is valid, not close to expiry, and installed on every hostname, including www, non-www, and subdomains. An expired certificate throws a full-page browser warning, and most visitors leave instead of clicking through. An automated expiry alert costs almost nothing and prevents an embarrassing outage.
Every HTTP request should redirect to HTTPS with a permanent 301. Then look for mixed content, where a secure page loads images, scripts or styles over HTTP. Browsers block or flag these files, and a blocked script can quietly break rendering or tracking. The mixed content guide on web.dev explains how to find them.
Three security headers are worth checking on any production site:
- Strict-Transport-Security (HSTS) tells browsers to use HTTPS on later visits. Sites that want preloading can follow the rules at hstspreload.org.
- Content-Security-Policy (CSP) limits which sources can load scripts and styles, which reduces the damage from injected code.
- X-Frame-Options controls whether other sites can frame your pages, which helps block clickjacking.
Roll out a strict CSP carefully. A policy that is too tight can block your own analytics or checkout scripts, so run it in report-only mode first.
Hreflang applies only to sites serving more than one language or region. Language codes should follow ISO 639-1 and region codes ISO 3166-1 Alpha-2. A classic slip is writing UK where the correct code is GB.
Every version must also link back to the others. Google’s localized versions documentation says annotations without return links may be ignored. An x-default entry gives visitors from unmatched regions a sensible fallback.
If you serve one market in one language, skip hreflang. It adds maintenance work and new ways to break things with no upside.
8. Validate Structured Data and Search Appearance
Structured data turns page content into code machines can read. It helps Google understand a page and can make eligible pages appear with richer listings.
Test your markup with the Rich Results Test and the Schema Markup Validator. The first checks eligibility for Google features and the second checks general Schema.org syntax. Run both, because passing one does not guarantee the other.
Look for missing required properties and for markup that does not match visible content. Google’s structured data policies require that markup describe what users can actually see. Marking up hidden or misleading content can lead to a manual action.
Keep expectations current as well. In August 2023, Google limited FAQ rich results to well-known government and health sites and reduced HowTo rich results. Many sites still carry FAQ markup that no longer shows anything. Check whether yours earns its place before adding more.
Beyond schema, audit the basics that shape your listing:
- Missing or duplicate title tags across templates.
- Meta descriptions that are blank or repeated.
- Heading hierarchies that skip levels or use several H1 tags without reason.
Google may rewrite titles and descriptions when another version fits the query better, so write them for clarity first. The title link documentation explains what drives those rewrites.
Then open the Enhancements reports in Search Console. They list the rich result types Google detected and flag errors by page group, so you can fix a template once instead of chasing single URLs.
Valid markup does not promise a rich result. Google treats it as one signal among many and decides case by case.
9. Prioritize the Technical SEO Issues You Find
An audit only pays off when it ends in a plan engineers can act on. Sorting by severity keeps critical blockers ahead of cosmetic work.
Critical issues stop crawlers from reaching or indexing pages that earn revenue. Fix these first:
- A noindex tag on a primary page template.
- Robots.txt rules blocking major directories.
- Canonical tags pointing to wrong, missing, or looping URLs.
- Persistent 5xx errors on important pages.
High priority issues waste crawl effort or hide key content. Schedule them right after:
- Broken internal links returning 404 on secondary templates.
- JavaScript rendering failures that hide body text.
- Redirect chains and loops.
- Duplicate URLs from unmanaged parameters.
Lower priority issues bring gradual gains without blocking indexation. Batch them into routine maintenance:
- Missing alt text on older blog posts.
- Core Web Vitals scores slightly outside targets.
- Optional schema warnings that leave the main markup valid.
Rank by business impact, not raw error counts. One canonical error on your top revenue page outweighs five hundred alt text warnings on archived posts. When you present findings, tie each issue to the traffic or revenue it puts at risk. Fixes get approved much faster that way than from a list of error codes.
10. Verify Fixes and Record the Audit Results
A closed ticket does not prove a fix works. Deployments fail, caches serve old files, and later releases bring old bugs back. Verification is what turns an audit into lasting improvement.
Re-crawl the affected page groups in Screaming Frog and compare against your baseline. Then spot-check single URLs in URL Inspection and confirm the live page returns 200 OK, the right canonical, and no blocking directives. For bigger issues, use the validation option inside the relevant Search Console report, which asks Google to recheck the affected URLs.
Keep a shared log. For each issue, record these details:
- The affected URL pattern or template.
- Severity and the date found.
- The owner responsible.
- The exact change deployed.
- Verification status and date.
Store baseline numbers beside each entry, such as indexed page counts, crawl stats and impressions. That record shows which fixes moved the needle, and it gives the next auditor a history to work from.
Technical SEO Audit Tools
The right toolkit depends on site size and platform. These are the core tools:
- Google Search Console is the free starting point for index coverage, live inspection, sitemap submission, and error reports straight from Google.
- Screaming Frog SEO Spider is a desktop crawler that checks status codes, metadata, redirects, canonicals, and rendered JavaScript. The free version crawls up to 500 URLs.
- PageSpeed Insights combines Chrome UX Report field data with Lighthouse lab diagnostics.
- Rich Results Test checks whether markup qualifies for Google’s enhanced features.
- Schema Markup Validator checks structured data syntax against Schema.org standards.
- Server access logs show how often bots visit, which URLs they request, and which status codes they get. No external crawler can show you that.
Match the stack to the scale. A small brochure site can run a solid audit with Search Console, PageSpeed Insights, and the free Screaming Frog version. Platforms with millions of URLs need an enterprise crawler and regular log analysis.
Conclusion
A technical audit tells you what stands between your pages and the index. The order of the fixes matters as much as the fixes. Handle crawling and indexing first, then duplicate signals and broken paths, and leave speed and polish for last.
Treat the audit as a habit and not a one-off project. A quarterly review suits most sites, with a quick check after every major release or migration. Catching a stray noindex tag in a day costs almost nothing, and catching it a month later can cost a quarter of your traffic.
Regular technical audits help identify issues that affect crawling, indexing, and overall search performance. To strengthen your SEO workflow further, explore these related resources on technical SEO tools and on-page content optimization: