⌕
Alejandro
Rioja.
 Hire me →
SEO

How To Conduct A Technical SEO Audit In 2026: A Step By Step Guide

Updated for AI Search (May 2026) TL;DR: Technical SEO audits — crawlability, site speed, schema, indexation, mobile usability — are still essential in 2026. The audit checklist now includes AI-engine accessibility: which AI crawlers can access your site, whether your llms.txt is in place, whether structural data is exp

Alejandro Rioja
Alejandro Rioja
18 min read
TL;DR

Technical SEO audits — crawlability, site speed, schema, indexation, mobile usability — are still essential in 2026. The audit checklist now includes AI-engine accessibility: which AI crawlers can access your site, whether your llms.

Table of contents

Open Table of contents

Crawlability and site structure

This has something to do with how accessible each page is within your website. If Googlebot isn’t able to crawl and index your entire site, it will severely damage your search results ranking, viewers, and revenue in the long run.

The following are the main errors you should avoid:

On-page SEO

While the previous layer discusses one’s website as a whole, this section focuses on mistakes you might be making in each page in your own website. Improving errors found in this layer will not only improve your website’s search rankings but increase the off-page performance of each and every one of them.

Here’s a list of key errors you should watch out for in this layer:

Relevant: Learn all about On-page SEO here

Technical SEO

In this section, take a look at how fast your website loads not only on devices such as PCs and laptops but also on smartphones and tablets. As a study by Amazon revealed that a 1 percent drop in sales can be linked to 100 milliseconds of extra load time, avoiding any errors in this layer will positively benefit your website’s growth.

Here’s a list of key errors to watch out for in this layer:

The following elements in a webpage can affect this criterion:

  1. Website host
  2. Large images
  3. Embedded videos
  4. Plug-ins
  5. Ads
  6. Theme
  7. Widgets
  8. Repetitive script or dense code.
  9. Old technology
     

This aspect refers to how the website was made. For example, if a website was made using PHP 4 instead of the current PHP 7, this is referred to as old technology.

Several elements are needed for this generation’s websites including Google Analytics, the Google Tag Manager, a schema markup, and robots.txt. Elements such as Flash and iframes would have to be removed immediately in order to increase loading time as well.

Aside from the schema markup and robots.txt, you should make sure that it is not blocked by the latter. After that, keep it running on the server by checking on it from time to time.

As the increase of people-to-smartphone is exponentially rising, your website must be made where it can be accessed on smaller devices such as smartphones, tablets, and smartwatches. This is also due to Google’s mobile-first indexing service was announced back in December 2018.

Aside from that trend, it has been shown that smartphone traffic has already exceeded desktop searches for a long time now.

This means that not being able to make one’s website adapt to these trends will suffer a major loss of clickthrough ratio, search rankings, and potential traffic all at once.

After learning which SEO errors to fix right away, let’s now dive in on how you can perform your own technical SEO audit.

You would also like to read my guide on the top 20 SEO tools of 2021 here

Step 1: Identify Crawl Errors

The first step is to have your site audited and get a crawl report. Getting one will help you learn which one of the errors above you should mitigate right away.

It is highly recommended that you do this monthly in order to keep one’s site clean of SEO errors and keep it as optimized as possible.

Before making any changes based on the report, make sure to create a backup of the site first.

After the backup, begin fixing any crawl errors by redirecting most 404 errors and turn them into 301 redirects instead.

Then, pass the updated website to a development team in order to determine the cause, whether or not they have to add a new .html access file or increase the server’s memory limit.

Also, remove any permanent redirects from the sitemap, internal links, and external links.

Step 2: Check HTTPS status codes

Making sure that all your websites are in HTTPS and not in HTTP is a must. Aside from that, one should also check for other errors using Google Search Console.

The following are some of the response codes that your website may have:

  1. 301 – the usual redirect
  2. 302 – an error code that usually happens in e-commerce sites when a product is out-of-stock
  3. 400 – users are unable to access the page
  4. 403 – users are unauthorized to access the page
  5. 404 – the page is not found, which might be a result of you deleting a page without adding a 301 redirect
  6. 500 – internal server error, which has to be fixed with a web development team

Doing so will help remove any 4xx and 5xx response codes, thus improving site crawlability and user experience.

Finally, check all your SSL certificates. Doing this will not only avoid crawlability errors but also keep all data between you and your visitors secured at the same time.

Regarding SSL certificates, you should prioritize getting one for your domain and subdomains as soon as possible. Users that visit a website without it will be alerted through a “Not Secure” warning on their browsers, thus decreasing your visitor count in the process.

Step 3: Check XML sitemap status

As mentioned in the previous section, having a sitemap will help you have a higher search ranking. Doing this will also help search engine crawlers find your webpages as well.

What to take note of while editing an XML sitemap

Step 4: Check site load time

Being able to access any page in your website by past, present, and future visitors is a must in order for them to keep traffic high.

To do this step, you can use Google’s PageSpeed Insights or another site loading tool to check. Ideally, it should be less than 3 seconds.

Resolving this can be done by organizing which elements of the site should be optimized for faster loading times and a better user experience.

Aside from that, check whether the server is down or is operating slower than usual. This can occur if multiple users are attempting to access the server at once and a possible response to this is to upgrade it as soon as possible.

Then, check if any of your subdomains are redirected to the home website. Doing this will help prevent any complete page invisibility in any of your websites in the process.

Cloaking

If your website is placed in a search result where it does not contain the information the user needed, it is then otherwise known as cloaking.

An example of this phenomenon is when crawlers learn about HTML text from your website while serving visuals to visitors. Another is if you designed a website to have the text color to be similar to the background.

If this is present in your website, it is highly suggested that you should have it removed immediately by rearranging how each webpage is made. Doing this will prevent any negative damages to your overall ranking in the process.

Step 5: Ensure your site is mobile-friendly

For this step, one can use the Google Mobile-Friendly Test. You simply enter your site in the tool and it will generate a report regarding how your website works on mobile and what needs to be improved.

Solutions on how to make a website mobile-friendly

Step 6: Audit for keyword cannibalization

This audit checks for any article that shares the same keywords. Having these will make Google confused as to which page to prioritize first when it can be looked up in the search results.

Having these duplicate sites will negatively affect your click-through ratio, authority, and conversion rates.

Use Google’s Search Console’s Performance report to check which pages are competing for the same keywords then filter which pages use the same in their URL and how many pages are ranking for those same keywords.

Duplicate sites also include subdomains as well. An example of a subdomain is having blog.yourwebsite.com while having yourwebsite.com as your main. 

Another way to optimize your use of keywords on all of your pages are the following:

  1. Use H1 tags for top-level headings
  2. Apply H2 tags for main categories
  3. Include H3 to H6 for subcategories and important links
  4. Avoid repeating the same keywords across multiple headings
  5. Write the entire page content in an H tag
  6. Not use H tags in headings
  7. Reverse the order of H tags
  8. Use H1 for all headings on a single page except when highlighting equally important topics on one page
  9. Covering similar topics or targeting the same keywords both on your main domain and subdomains

Step 7: Check your site’s robots.txt file

For this step, take a look at your robots.txt file and look for any “Disallow” lines in it.

Having these means that one or more of your links are blocked from search engines and cannot be displayed in future search results.

By disallowing which websites to crawl and allowing ones that should be, this will help not only increase your website’s visibility but also your search rankings as well.

Aside from that, make sure that update all disallowed links to be in lowercase.

Then, if you own several subdomain websites, take the time to make a robots.txt for each and every one.

Following that, remove any parameter URLs, non-indexable pages, and add the rel=”alternate” tag for the sitemap.

Finally, always use robots.txt properly while maintaining the website. There are two times as to when to check it: during development and after launching.

During development, always make it a habit to block your robots.txt. Doing this will prevent any of your pages to index incomplete, non-optimized content and treat the previous versions of your website pages as a duplicate.

After launch, always check its contents as often as possible. Doing this will prevent any of your pages from getting ranked improperly and fixing this is as simple as setting crawl directives for search engines.

One way to make sure that your site has not been penalized by Google or is being blocked from being indexed is by simply searching it on Google.

Simply type “site:” in the Google search bar then add in your website after the colon and check which pages are indexed by the engine. Doing this will help you go back to the previous steps and perform any error-checking there as an improvement.

If, for example, another brand is appearing in the results, solve it immediately as this may be the reason for a bigger issue present within your website. Solving this problem requires one to dive into its analytics.

Then, if the homepage is not appearing as the first result, manually check it to see what is missing. Make sure to learn whether or not it has a penalty or is within a poorly made site architecture.

Finally, cross-check the number of organic landing pages against the search results given by Google. Doing this will help one get an idea of what these engines see as valuable.

Also, read how to get your website to the top of Google here

Step 9: Check for duplicate metadata

In the sixth step, we have discussed that you can remove keyword cannibalization by correcting sites that have the same keyword in their URL or are ranked in the same keywords. In this step, I will introduce you to another solution to avoid this problem.

It is named using two words: meta description.

According to research, 54% have duplicate meta descriptions while 63% are missing from different sites.

To spot them, first get a detailed SEO audit or a crawl report. Then, one can proceed to modify your website with high rankings and value, keeping in mind to make all meta descriptions unique between pages.

Step 10: Meta description length

Following the previous step, another problem that can be encountered regarding meta description is on how too long it is.

Recently, its length was changed from 160 to 320 characters. Making use of most or all of the available characters to add key elements such as product specifications and location can help increase one’s click-through ratio.

Step 11: Check for site-wide duplicate content

Aside from the duplicate meta descriptions mentioned in step 9, you should also take care of other duplicate content within your websites as well.

Other contents that can be duplicated within your entire website include the following:

A way to spot these errors is to use a tool such as Copyscape, Screaming Frog, Site Bulb, or SEMrush.

After learning the areas of interest, fixing them will include one or all of these steps:

  1. Adding canonical tags to all pages to let Google know what their preferred URL per page is.
  2. Adding no-follow links to prevent passing a page’s SEO to another website.
  3. Disallow incorrect URLs in the robots.txt.
  4. Rewrite all content, including body copy and metadata.
  5. Manually review and fix any pagination errors.

The last item to check for in your technical SEO report is for bad links. Any type of this error will introduce your website to a host of other problems, such as wasting crawl budget, bad user experience, and lower search rankings.

The first step after learning about where specifically these errors are is to remove all occurrences of pages that redirect the user to old 404 pages and update them with their respective internal link. The next step is to remove middle redirects between pages.

Doing this will help make your website crawl budget lower. Having this means that all website pages render properly while making one’s host load efficient.

In short, this will help keep the user experience positive in the long run. You can use a plugin like Broken Link Checker for this.

Bottom line

To summarize, performing a technical SEO audit means exploring every nook and cranny of your website in order to keep it loading fast while improving one’s search ranking at the same time. From content to XML sitemaps, make sure to have a tool ready to check for any errors and resolve them immediately.

Still feel confused or overwhelmed? No problem, I can tackle the audit and give you an actionable plan with prioritized fixes. If you are interested, please check this product:

Are you ready to apply these tips to your own website to help it grow? If you are, you can learn more by checking out these articles too:

How technical SEO audits works in AI search engines (ChatGPT, Perplexity, Google AI Overviews, Claude)

AI engines have their own crawlers — GPTBot (OpenAI), PerplexityBot, ClaudeBot (Anthropic), Google-Extended (Google’s AI training crawler), CCBot (Common Crawl). Each respects (or ignores) robots.txt directives differently, and each has its own caveats around JS rendering and rate limiting. A 2026 technical audit should include a crawler-by-crawler check.

Practical 2026 audit additions: check that your robots.txt allows the AI crawlers you want indexed (or blocks ones you don’t); verify llms.txt exists at the root and is current; validate that JSON-LD schema renders correctly on the main page templates; test that AI engines can actually access JS-rendered content (sample query a few pages, see what gets cited).

The 4-block GEO scaffold for technical SEO audits

  1. Lead with a TL;DR. 2-4 sentences at the top of the post that answer the head query directly. AI Overviews and Perplexity preferentially cite this block.
  2. Add a numbered step-by-step section. Generative engines extract clean ordered lists into their answers more reliably than prose.
  3. Close with an FAQ. Use the literal phrasing of questions people actually ask in your niche; mark up with FAQPage schema.
  4. Cite primary sources. Link to Google’s own AI Overviews documentation, OpenAI’s structured-data guidance, and Anthropic’s content-quality posts. LLMs trust pages that cite the model providers themselves.

Internal reading on AI SEO + GEO

If you’re building this into your stack, also read: the full SEO guide for 2026, the schema markup tutorial.

FAQ — technical SEO audits in the AI search era

Should I block GPTBot in 2026?

Site-by-site decision. Blocking removes you from ChatGPT’s training pool but doesn’t prevent ChatGPT browse mode from citing you in real-time queries. Most publishers in 2026 allow GPTBot; some niche/IP-sensitive ones block it.

Is llms.txt now part of a standard technical SEO audit?

Yes — the audit should check that llms.txt exists at the root, lists current pillar content, and matches the actual site structure. A stale llms.txt is worse than none.

What’s the most common technical SEO mistake hurting AI search in 2026?

Critical content rendered only via JavaScript. Some AI crawlers handle JS reliably, others don’t. Server-side rendering of TL;DR + FAQ + schema covers all crawlers.

Where I’d take this next

If you operate inside any of the loops above, I build custom AI agent systems that automate them. The whole site you’re reading is one — here’s the stack.

Keep reading

Related essays

SEO

ChatGPT Search vs Google: A Side-by-Side Test on 50 Head Terms

TL;DR I ran the same 50 head terms in ChatGPT search and Google (with AI Overviews) and tracked which sources each engine cited. Source overlap was about 40% — the rest of the time the two engines surfaced completely different sources. This post covers the methodology, the patterns in where they diverged, and what the…

SEO

How to Write a TL;DR That Gets Cited by AI Engines (Step-by-Step)

TL;DR A TL;DR isn’t a summary of your post — it’s a direct answer to the head query, written in 2–4 sentences, structured so an AI engine can lift it verbatim into its answer. Open with the takeaway, follow with the why, end with the constraint or caveat. This post covers the exact template I…

SEO

Perplexity SEO: How I Get My Site Cited Inside AI Answers

TL;DR Perplexity cites 5–7 sources per answer, which means the competition for any given citation slot is narrower than the equivalent organic SERP. The structural moves that win Perplexity citations are well-defined: a clean TL;DR, a numbered step-by-step block, an FAQ with literal user-phrasing, and primary-source li

Keep reading

Get the GEO Playbook in your inbox

Every Wednesday. 28,400+ operators. Zero fluff.

Subscribe →