Launch day has a way of hiding the small stuff.
You've been staring at the product for weeks, and the last thing you're thinking about is whether staging's robots.txt came along for the ride. I wrote about that feeling in Ship It Scared, and this check is my way of taking a few of those worries off the list.
What It Checks
It fetches your page from this site's server, plus the handful of files every site is expected to have, like robots.txt, the sitemap, and the favicon. It also tries the http:// version of the page, and one made-up address to see whether missing pages return a real 404. It never goes looking for files you didn't publish.
Each result is marked as fine, worth a look, or to fix. The ones to fix will hurt you on day one, like a page that tells search engines to stay away. The ones worth a look can wait until the first week, and anything in grey is optional.
The Ones That Hurt Most
A leftover noindex. Staging sites tell search engines to stay away, and that tag has a habit of surviving the move to production. Nothing looks wrong until you realize the site has been live for a month and never showed up in search.
A robots.txt that blocks everything. It's the same story, where Disallow: / from staging gets copied over to production.
A canonical URL pointing at staging. Search engines will treat the staging address as the real page and ignore yours.
A soft 404. This is when a made-up address returns a 200 and your home page. Broken links never show up as errors, and search engines can end up indexing junk addresses.
Frequently Asked Questions
Why is the response time different from what I see?
It's measured from this site's server, from the start of the request to the last byte of the HTML. Your visitors are somewhere else, and their browsers load a lot more than the HTML, so I'd treat it as a rough signal that the backend is slow.
Can I check a page that isn't public yet?
Only if it's reachable from the internet. Pages on localhost, a private network, or behind a login can't be checked from here.