Lab notes · 2026-08-28

AS218833: from ASN assignment to two-site anycast in one afternoon

We build catchment observability for anycast operators. To do that honestly, we need our own anycast network to measure, break and fix. Today it came alive, and our own tooling watched every minute of it.

Timeline (UTC)

• 14:37 RIPE NCC assigns AS218833 (ANYCAST-LAB) with 2a03:5840:161::/48. The sponsoring LIR had ROA and route6 object registered before we even logged in.
• 16:24 BIRD comes up on the Frankfurt node, two MD5 sessions to the upstream's route servers.
• 16:27 The first announcement hits the default-free zone. Our RIS Live consumer, subscribed to the prefix since the morning, writes 520 announce events into Postgres as collectors around the world pick up paths like … 1299 35661 218833.
• 16:29 First ping to 2a03:5840:161::1 from a machine in New Jersey succeeds. Global reachability in about two minutes.
• 17:19 The second site (New Jersey, different upstream: AS20473) reaches Established. Same prefix, two origins. That is anycast.

The part we actually care about

A prefix in the DFZ is table stakes. The point is that the network is observable from birth:

• The RIS Live consumer recorded both announcement waves per collector and peer, so we can replay how visibility spread.
• Each node answers with its own NSID (fra1-lab.anycast.dev and ewr1.anycast.dev), so any vantage point can tell which site BGP routed it to. From Europe you land in Frankfurt, from the US in New Jersey.
• The prefix is not idle: it now carries the authoritative nameserver ns.portalix.org (2a03:5840:161::53) for our own domains, over IPv6 anycast. dnstap on both nodes feeds per-minute aggregates into the same pipeline we sell, so real resolver traffic shows up in our catchment dashboard within minutes of the AAAA going live.

Field notes, so you don't repeat ours

• Both upstreams require TCP-MD5. With the password set, a session that idles in Connect is usually the far side still provisioning; without it you get an instant reset.
• Vultr enables BGP per account, but the instance must be restarted from their panel before the route servers talk to you. A reboot from inside the VM is not enough.
• Loopback service addresses do not survive reboots on cloud images that manage networking outside ifupdown. A tiny systemd oneshot unit (Before=bird.service knot.service) is the boring, portable fix.
• Knot answers EDNS NSID with the hostname unless you set server.nsid explicitly. identity alone only covers id.server.

Day two: IPv4

The leased IPv4 /24 (94.249.165.0/24) went live the next day, from the same two sites. The instructive part was what had to be true first: the LIR's automation had created the IRR route object for our ASN but the RPKI ROA still pointed at theirs, so the prefix validated as invalid for us. Announcing at that point would have been dropped silently by every strict validator while looking fine in our own router. We checked before announcing, asked the LIR to fix the ROA, and only then turned it on. A query from a third-party network in Bern now reaches Frankfurt in 8 ms and identifies itself as fra1-lab.anycast.dev; from New Jersey the same address answers as ewr1.anycast.dev.

What happens next

Per-continent catchment baselines from ~1,000 RIPE Atlas vantage points, now dual-stack and over both UDP and TCP, and traffic-engineering experiments with disposable nodes, each one measured as a before/after diff. The lab's etiquette, prefixes and opt-out live at anycast.org.

If you run anycast DNS and wonder where your traffic actually lands: that measurement is what we sell.