Tame the Bots Blog Posts

A fairly common belief is that Google stop rendering pages after 5 seconds, I did some testing to show that's not true, but why your measurements might be lying to you to make you think so, because time in the web rendering service is artificial.

Now both Google and Bing offer some metrics, how closely does a site perform in both?

I spoke at the excellent Tech SEO North event on the 18th June 2026, you can find my deck here

A look at how LLM crawlers handle next.js's out of order html streaming.

Googlebot only gets the first 2 MB of your HTML, but that's probably not an issue

It seems there's been a change in how Google's rendering works when a page is set to noindex, with JavaScript now being executed.

Core Web Vitals can be affected by where your users are, and what language they speak.

View the original, un-rewritten version of a wayback machine page, get the html they saved without the changes to links etc.

A time-series visualisation of where Google crawled tamethebots.com from.

Some of the best advice I've heard about HCU and EEAT is literally just "Don't Dream it, Be it"

Using the new Soft Navigation features to gather accurate Core Web Vitals data on SPA sites, and seeing if the improvements for 2nd plus navigations are worth it.

Sometimes it's useful to look past the single 75th percentile figure to see where you're heading.

A quick round-up of tools that handle hreflang testing and checking

A quick test to find out how the new (ish) robots directive indexifembedded works with iframes containing html and pdfs.

I was really happy to be a guest on Dan Sure's Expert on the Wire podcast, talking about JavaScript SEO and more!

Priority Hints help lower LCP if used in the right way.

A quick experiment to find out if content produced by a web worker can be rendered and indexed by Google, and if there are any limitations.

A script using nodejs to access BigQuery and write a log file from logflare.app data to NGINX's default access log format, perfect if you have tools that work with that.

A quick write up of my experience facilitating a session at the Google Search Central Unconference, held on 21st June 2021

What does 28 days mean?

My first time speaking at BrightonSEO!

Potential cumulative layout shift metric changes, feeback wanted!

A halloween themed talk on core web vitals I did for Dan Taylor, Video & Slides

The new Web Vitals performance metrics bring the focus beyond that of speed.

A non-techy guide to what https status codes are, and what they mean for your website.

Chrome's javascript bookmarklets can be a great additon to an SEO's toolbox, here's a few examples.

A quick study to find out if googlebot uses Etag response headers, and sends If-None-Matched request headers to get 304 responses.

404, 301, it's still important for your JavaScript site, find out about your options!

Does Google use third party review sites to rank yours?

Using JavaScript to tell if someone has clicked on an iframe embedded in your page