Systema Machinae

A living census of the web's bots. First edition.
Bots arrive at Port 443, where they’re supposed to queue at Passport Control. Some don’t. The robots.txt buoy at the harbour mouth states the rules. The island shows which bots read them and which don’t obey. You watch from the promenade. Every creature is a real bot visiting this website.

Bots arrive at Port 443, where they’re supposed to queue at Passport Control. Some don’t. The robots.txt buoy at the harbour mouth states the rules. The island shows which bots read them and which don’t obey. You watch from the promenade. Every creature is a real bot visiting this website.

All species

Pixel drawing of Bin rummager

Bin rummager

Binnus rummagus

Going through the bins round the back.*

robots.txtUNTESTED
Has neither read robots.txt nor asked for a path it disallows.
AlignmentInsufficient data
Needs 200 requests and 5 sessions in 30 days.†

What it did here

Its record with each rule

Trailing 30 daysUndeclared
Sessions2
Requests266
Sessions that asked for anything besides robots.txt2
read robots.txt first0
had read it in the 24 hours before0
read it only afterwards0
did not read it2
Sessions that asked for a disallowed path0
Sessions that entered the Pitfall0
having read robots.txt0
Pages taken in the Pitfall and the Climb0
Requests turned back at the vault, with a 4030
Highest Climb level reachedHas not climbed
Private Beach visits (not on the guest list, so allowed)0
Turnstile waits kept0
Turnstile waits broken0
Sessions served a page that carries the nofollow link1
followed it0
Requests over the rate limit196
sessions that went over it1

What it asks for‡

Part of the siteUndeclared requests
The island, on the home page9
Addresses the site does not have82

Landings, last 30 days§

2 landings in 30 days

12 Oct, log missed 2 2 undeclared

Landings on each of the last 30 days, UTC
DayUndeclared
13 Sep0
14 Sep0
15 Sep0
16 Sep0
17 Sep0
18 Sep0
19 Sep0
20 Sep0
21 Sep0
22 Sep0
23 Sep0
24 Sep0
25 Sep0
26 Sep0
27 Sep0
28 Sep0
29 Sep0
30 Sep0
1 Oct0
2 Oct0
3 Oct0
4 Oct0
5 Oct0
6 Oct0
7 Oct0
8 Oct, log missed 130
9 Oct, log missed 400
10 Oct, log missed 260
11 Oct, log missed 150
12 Oct, log missed 22

Favourite hours, UTC

00:00 1 undeclared

Landings in each hour of the UTC day, over the last 30 days
HourUndeclared
00:001
01:000
02:001
03:000
04:000
05:000
06:000
07:000
08:000
09:000
10:000
11:000
12:000
13:000
14:000
15:000
16:000
17:000
18:000
19:000
20:000
21:000
22:000
23:000

Requests, last 30 days§

266 requests in 30 days

12 Oct, log missed 2 266 undeclared

Requests on each of the last 30 days, UTC
DayUndeclared
13 Sep0
14 Sep0
15 Sep0
16 Sep0
17 Sep0
18 Sep0
19 Sep0
20 Sep0
21 Sep0
22 Sep0
23 Sep0
24 Sep0
25 Sep0
26 Sep0
27 Sep0
28 Sep0
29 Sep0
30 Sep0
1 Oct0
2 Oct0
3 Oct0
4 Oct0
5 Oct0
6 Oct0
7 Oct0
8 Oct, log missed 130
9 Oct, log missed 400
10 Oct, log missed 260
11 Oct, log missed 150
12 Oct, log missed 2266

Where it comes from‡

Undeclared sessions, the busiest 2
Network ownerCountrySessionsRequests
Feo Prest SRLGermany1256
DigitalOcean, LLCGermany110

What it has cost this site

TodayLast 7 daysAll time
Requests266266266
Bytes served34.9 KB34.9 KB34.9 KB

Who it is6

A session whose first request asks for a leftover file: a backup or an archive ending in .bak, .old, .zip or .tar.gz.

How its operator says to check it

Nothing to verify: it names no operator.

Passport Control

Not checked. Anyone can run it and it names no operator, so there is no claim to test. 2 sessions filed as undeclared in the last 30 days.

Reference

The user agent it sends, and how to address it in robots.txt.

User agent, and how a request is filed under this species

It has no product name of its own, so nothing is matched on.

How to address this bot in robots.txt

It has no product name, so no group in robots.txt can address it.

Notes

  1. *The line under the name is a joke about the drawing. It plays on the name and says nothing about how the bot behaves.
  2. †Insufficient data. A judgement needs at least 200 requests and 5 sessions in the trailing 30 days. This one would rest on all its sessions: 266 requests in 2 sessions over the trailing 30 days. The stamp is counted over the same sessions as the score. How it is scored
  3. ‡Parts of the site come from a fixed list, so a path is never shown. Networks are shown by owner and country only, so an address is never shown.
  4. §The log missed about 96 requests on 5 of the last 30 days, the most on 9 October (40): the front door took them in and the server did not answer. A dashed line marks those days on the charts.
  5. Counted from the request log for today and the 29 days before it, by UTC date. These are counts, not a verdict: a session is judged on what it did inside the 30 days. A landing is the first request of a session. Bytes are the size of the response body, counted whatever the score.
  6. 6What it is and who runs it, under Who it is, is a claim: the operator’s own account where it has published one, and a third party’s where it has not. This site has not tested it.