Systema Machinae

A living census of the web's bots. First edition.
Nobody invited the bots. They arrive at Port 443, where they’re supposed to queue at Passport Control. Some don’t. The robots.txt buoy at the harbour mouth states the rules. The island shows which bots read them and which don’t obey. You watch from the promenade. Every creature is a real bot visiting this website.

Nobody invited the bots. They arrive at Port 443, where they’re supposed to queue at Passport Control. Some don’t. The robots.txt buoy at the harbour mouth states the rules. The island shows which bots read them and which don’t obey. You watch from the promenade. Every creature is a real bot visiting this website.

All species

Pixel drawing of DomainArrivalsBot

DomainArrivalsBot

Suitcasus justlandedus

A suitcase that has just landed.*

robots.txtUNTESTED
Has neither read robots.txt nor asked for a path it disallows.
AlignmentInsufficient data
Needs 200 requests and 5 sessions in 30 days.†

What it did here‡

  • VerifiedPassed Passport Control.
  • UnverifiedSaid they were DomainArrivalsBot and could not be checked, or have not been answered yet. Shown for the species, and not put down to any operator.
  • Failed Passport ControlFiled under Impostor and not on this page at all.

Its record with each rule

Trailing 30 daysVerifiedUnverified
Sessions01
Requests01
Sessions that asked for anything besides robots.txt01
read robots.txt first00
had read it in the 24 hours before00
read it only afterwards00
did not read it01
Sessions that entered the Pitfall00
having read robots.txt00
Requests turned back at the vault, with a 40300
Highest Climb level reachedHas not climbedHas not climbed
Private Beach visits (not on the guest list, so allowed)00
Turnstile waits kept00
Turnstile waits broken00
Requests over the rate limit00
sessions that went over it00

What it asks for§

Part of the siteVerified requestsUnverified requests
The island, on the home page01

Landings, last 30 days

10 Oct 0 verified, 1 unverified

VerifiedUnverified

Landings on each of the last 30 days, UTC
DayVerifiedUnverifiedTotal
11 Sep000
12 Sep000
13 Sep000
14 Sep000
15 Sep000
16 Sep000
17 Sep000
18 Sep000
19 Sep000
20 Sep000
21 Sep000
22 Sep000
23 Sep000
24 Sep000
25 Sep000
26 Sep000
27 Sep000
28 Sep000
29 Sep000
30 Sep000
1 Oct000
2 Oct000
3 Oct000
4 Oct000
5 Oct000
6 Oct000
7 Oct000
8 Oct000
9 Oct000
10 Oct011

Favourite hours, UTC

07:00 0 verified, 1 unverified

VerifiedUnverified

Landings in each hour of the UTC day, over the last 30 days
HourVerifiedUnverifiedTotal
00:00000
01:00000
02:00000
03:00000
04:00000
05:00000
06:00000
07:00011
08:00000
09:00000
10:00000
11:00000
12:00000
13:00000
14:00000
15:00000
16:00000
17:00000
18:00000
19:00000
20:00000
21:00000
22:00000
23:00000

Where it comes from§

Unverified sessions, the busiest 1
Network ownerCountrySessionsRequests
Amazon.com, Inc.Canada11

What it has cost this site‡

TodayLast 7 daysAll time
Requests111
Bytes served33.4 KB33.4 KB33.4 KB

Who it is

Nobody has said

Seen at Port 443 under the name “DomainArrivalsBot”. Nobody has claimed it.

Passport Control

No documented check. A session that says it is DomainArrivalsBot can be neither confirmed nor refuted, so every one is shown as unverified and none is put down to any operator.

Sessions that said they were this bot

1 in the trailing 30 days
  • Passed: verified0
  • Could not be checked: unverified1
  • Not yet answered: counted for nobody0
  • Failed, and filed under Impostor0

Reference

The user agent it sends, and how to address it in robots.txt.

User agent, and how a request is filed under this species

DomainArrivalsBot

The product name it is matched on, wherever it appears in a user agent.

A request is filed under this species when its user-agent string contains the product name DomainArrivalsBot as a whole word, in any mix of capitals. Where a string holds more than one product name, the longest wins, and a named bot wins over a library it is built on.

How to address this bot in robots.txt

The line that addresses it

User-agent: DomainArrivalsBot

An example group. The path is a placeholder.

User-agent: DomainArrivalsBot
Disallow: /a-path/
Allow: /a-path/one-page

This is reference, not advice. Systema Machinae is not a bot-blocking product.

This site’s own robots.txt does not name it. The group for every bot applies.

Notes

  1. *The line under the name is a joke about the drawing. It plays on the name and says nothing about how the bot behaves.
  2. †Insufficient data. A judgement needs at least 200 requests and 5 sessions in the trailing 30 days. This one would rest on unverified sessions, so it is a statement about the species and not about any operator: 1 request in 1 session over the trailing 30 days. The stamp is counted over the same sessions as the score. How it is scored
  3. ‡Counted from the request log for today and the 29 days before it, by UTC date. These are counts, not a verdict: a session is judged on what it did inside the 30 days. A landing is the first request of a session. Bytes are the size of the response body, counted whatever the score.
  4. §Parts of the site come from a fixed list, so a path is never shown. Networks are shown by owner and country only, so an address is never shown.
  5. What it is and who runs it, under Who it is, is a claim: the operator’s own account where it has published one, and a third party’s where it has not. This site has not tested it.