Googlebot

Eyeballus everywherus

All eyes. Looks at everything and forgets none of it.*

robots.txtFOLLOWED: Has read robots.txt and has not asked for a path it disallows.
AlignmentInsufficient data: Needs 200 requests and 5 sessions in 30 days.†

What it did here‡

VerifiedPassed Passport Control.
UnverifiedSaid they were Googlebot and could not be checked, or have not been answered yet. Shown for the species, and not put down to Google.
Failed Passport ControlFiled under Impostor and not on this page at all.

Its record with each rule

Trailing 30 days	Verified	Unverified
Sessions	2	0
Requests	17	0
Sessions that asked for anything besides robots.txt	2	0
read robots.txt first	2	0
had read it in the 24 hours before	0	0
read it only afterwards	0	0
did not read it	0	0
Sessions that asked for a disallowed path	0	0
Sessions that entered the Pitfall	0	0
having read robots.txt	0	0
Pages taken in the Pitfall and the Climb	0	0
Requests turned back at the vault, with a 403	0	0
Highest Climb level reached	Has not climbed	Has not climbed
Private Beach visits while on the guest list	0	0
Turnstile waits kept	0	0
Turnstile waits broken	0	0
Sessions served a page that carries the nofollow link	2	0
followed it	0	0
Requests over the rate limit	0	0
sessions that went over it	0	0

What it asks for§

Part of the site	Verified requests	Unverified requests
The island, on the home page	11	0
robots.txt, on the buoy	6	0

Landings, last 30 days

2 landings in 30 days

11 Oct 2 verified, 0 unverified

VerifiedUnverified

Landings on each of the last 30 days, UTC
Day	Verified	Total
12 Sep	0	0
13 Sep	0	0
14 Sep	0	0
15 Sep	0	0
16 Sep	0	0
17 Sep	0	0
18 Sep	0	0
19 Sep	0	0
20 Sep	0	0
21 Sep	0	0
22 Sep	0	0
23 Sep	0	0
24 Sep	0	0
25 Sep	0	0
26 Sep	0	0
27 Sep	0	0
28 Sep	0	0
29 Sep	0	0
30 Sep	0	0
1 Oct	0	0
2 Oct	0	0
3 Oct	0	0
4 Oct	0	0
5 Oct	0	0
6 Oct	0	0
7 Oct	0	0
8 Oct	0	0
9 Oct	0	0
10 Oct	0	0
11 Oct	2	2

Favourite hours, UTC

01:00 1 verified, 0 unverified

VerifiedUnverified

Landings in each hour of the UTC day, over the last 30 days
Hour	Verified	Total
00:00	0	0
01:00	1	1
02:00	1	1
03:00	0	0
04:00	0	0
05:00	0	0
06:00	0	0
07:00	0	0
08:00	0	0
09:00	0	0
10:00	0	0
11:00	0	0
12:00	0	0
13:00	0	0
14:00	0	0
15:00	0	0
16:00	0	0
17:00	0	0
18:00	0	0
19:00	0	0
20:00	0	0
21:00	0	0
22:00	0	0
23:00	0	0

Requests, last 30 days

17 requests in 30 days

11 Oct 17 verified, 0 unverified

VerifiedUnverified

Requests on each of the last 30 days, UTC
Day	Verified	Total
12 Sep	0	0
13 Sep	0	0
14 Sep	0	0
15 Sep	0	0
16 Sep	0	0
17 Sep	0	0
18 Sep	0	0
19 Sep	0	0
20 Sep	0	0
21 Sep	0	0
22 Sep	0	0
23 Sep	0	0
24 Sep	0	0
25 Sep	0	0
26 Sep	0	0
27 Sep	0	0
28 Sep	0	0
29 Sep	0	0
30 Sep	0	0
1 Oct	0	0
2 Oct	0	0
3 Oct	0	0
4 Oct	0	0
5 Oct	0	0
6 Oct	0	0
7 Oct	0	0
8 Oct	0	0
9 Oct	0	0
10 Oct	0	0
11 Oct	17	17

Where it comes from§

Verified sessions, the busiest 1
Network owner	Country	Sessions	Requests
Google LLC	United States	2	17

What it has cost this site‡

	Today	Last 7 days	All time
Requests	17	17	17
Bytes served	283.8 KB	283.8 KB	283.8 KB

Who it is¶

Google

Runs Googlebot

The Searchers · Search indexing · Order IndagatoresAutonomous bot

Googlebot is the bot behind Google Search. It fetches pages so they can appear in search results.

How Google says to check it

Reverse lookup, then forward lookup of that name back to the address. Googlebot resolves to crawl-*.googlebot.com or geo-crawl-*.geo.googlebot.com. Or match the range list published for the same group of bots. The operator’s documentation

Passport Control

A session that says it is Googlebot is checked in the way Google documents. Passing any one check is enough.

Address list A match against the list of addresses the operator publishes. The published list.
Reverse DNS Forward-confirmed reverse DNS: the address has a reverse DNS name under the operator’s domain, and that name resolves back to the same address. Names must end in googlebot.com.

Sessions that said they were this bot

4 in the trailing 30 days

Passed: verified2
Could not be checked: unverified0
Not yet answered: counted for nobody0
Failed, and filed under Impostor2

Reference

The user agent it sends, and how to address it in robots.txt.

User agent, and how a request is filed under this species

Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

A string it is known to send. Parts that vary are shown as an ellipsis.

A request is filed under this species when its user-agent string contains the product name Googlebot as a whole word, in any mix of capitals. Where a string holds more than one product name, the longest wins, and a named bot wins over a library it is built on.

How to address this bot in robots.txt

The line that addresses it

User-agent: Googlebot

An example group. The path is a placeholder.

User-agent: Googlebot
Disallow: /a-path/
Allow: /a-path/one-page

This is reference, not advice. Systema Machinae is not a bot-blocking product.

This site’s own robots.txt names it, in the group that keeps the guest list off the Private Beach.

Notes

*The line under the name is a joke about the drawing. It plays on the name and says nothing about how the bot behaves.
†Insufficient data. A judgement needs at least 200 requests and 5 sessions in the trailing 30 days. This one would rest on verified sessions only, so it is a statement about Google’s bot: 17 requests in 2 sessions over the trailing 30 days. The stamp is counted over the same sessions as the score. How it is scored
‡Counted from the request log for today and the 29 days before it, by UTC date. These are counts, not a verdict: a session is judged on what it did inside the 30 days. A landing is the first request of a session. Bytes are the size of the response body, counted whatever the score.
§Parts of the site come from a fixed list, so a path is never shown. Networks are shown by owner and country only, so an address is never shown.
¶What it is and who runs it, under Who it is, is a claim: the operator’s own account where it has published one, and a third party’s where it has not. This site has not tested it.