On August 28, 2026 Cloudflare opened BotBase to the people who run the bots. BotBase for Operators lets a crawler operator submit an entry to the bot directory Cloudflare launched on July 1, watch it move from "Waiting for review" to "Accepted" or "Rejected", and, in the words of Julian Laxman's launch post, have the identity claim checked by a machine rather than by hand: Cloudflare will "fetch your IP list, confirm your reverse DNS, or validate your Web Bot Auth signature automatically". Submissions have grown about seven times since 2023, which is why the manual queue had to go.
One of those three proofs is a feature of the DNS almost as old as the DNS itself, sketched as an IN-ADDR domain in RFC 883 in 1983 and given its lasting form in RFC 1035 in November 1987. Reverse DNS, rDNS for short, is the mapping from an IP address back to a name. It lives in PTR records, published under in-addr.arpa for IPv4 and ip6.arpa for IPv6, in a zone controlled by whoever holds the address block. On its own it is a convenience for log files. With a forward lookup bolted on, it becomes a proof of address ownership that mail receivers have demanded for thirty years and that bot directories now accept.
The reverse tree hangs off in-addr.arpa
A forward lookup is the DNS working as designed. You hold example.com, you publish an A record in your zone, and a resolver walks from the root to com to your servers to fetch it. Delegation runs right to left through the name, and every label is somebody's responsibility.
An IP address has no zone. The fix in section 3.5 of RFC 1035 was to turn the address into a name: write the four octets in reverse order and append in-addr.arpa. The Googlebot address 66.249.66.1 becomes 1.66.249.66.in-addr.arpa, and the record stored there is a PTR pointing to crawl-66-249-66-1.googlebot.com. The reversal is the whole trick. DNS delegates from the right, addresses are allocated from the left, so flipping the octets lines the two hierarchies up: 66.in-addr.arpa goes to whoever administers 66.0.0.0/8, and the /24 holder gets 66.249.66.in-addr.arpa underneath it.
IPv6 reverses one nibble at a time
RFC 3596 does the same for IPv6, one hexadecimal digit per label, low-order nibble first, under ip6.arpa. Google's resolver at 2001:4860:4860::8888 expands to 2001:4860:4860:0000:0000:0000:0000:8888, so its reverse name is thirty-two labels long:
8.8.8.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.6.8.4.0.6.8.4.1.0.0.2.ip6.arpa.Scale is the real problem. RFC 8501 works it out for a single /48: at a thousand PTR records written per second, "the zone would still not be complete after 38 trillion years". So consumer IPv6 space has almost no real reverse data. ISPs return NXDOMAIN, serve a wildcard (which fails the forward check below, because one name cannot resolve back to every address in the block), or synthesize a PTR on the fly when queried. A missing PTR on an IPv6 address is not evidence of anything, and a present one may have been invented the moment you asked.
The zone belongs to whoever holds the block
IANA manages the apex of both reverse zones. Both are served by dedicated nameservers, a through f under in-addr-servers.arpa and ip6-servers.arpa, whose naming scheme RFC 5855 defines and which the five RIRs and ICANN operate. Before that, in-addr.arpa was authored at ARIN and served from a subset of the root servers. ip6.arpa moved to the new servers in December 2010, and in-addr.arpa followed in February 2011.
From there the chain follows allocation. IANA delegates each allocated /8 to the RIR that administers it. The RIR delegates on octet boundaries to its members: in the RIPE region a /17 to /24 gets one or more /24 zones, a /9 to /16 gets /16 zones, and each is a domain object with nserver attributes in the RIPE Database. IPv6 works the same way on nibble boundaries, typically a /32 to the LIR and a /48 to a customer who asks.
This is the part that surprises people who have only ever edited forward zones. Hold a /24 or more and you get a reverse zone of your own. Hold a /29 from a business ISP and there is no label boundary at /29, so the ISP either edits the /24 zone on your behalf or delegates you a synthetic child zone through the CNAME trick in RFC 2317. On a cloud provider, the provider holds the zone and gives you a form. When there is no form, there is no PTR.
A PTR is a claim, forward confirmation is the check
Anyone who controls a reverse zone can publish any name in it. If we set the PTR for an address in our own space to crawl-66-249-66-1.googlebot.com, nothing in the DNS refuses. What we cannot do is touch googlebot.com, Google's forward zone, and make that name resolve to our address.
Forward-confirmed reverse DNS, FCrDNS, closes the loop: resolve the PTR for the IP, resolve the resulting name forward, and check that the original IP is among the A or AAAA answers. RFC 8601 formalizes this as the iprev method and names the outcomes: pass, fail when the two lookups disagree, permerror when no PTR is published at all, temperror for a DNS failure. Google's Googlebot verification page spells out the same steps with the host command:
dig -x 66.249.66.1 +short
crawl-66-249-66-1.googlebot.com.
dig crawl-66-249-66-1.googlebot.com +short
66.249.66.1A pass proves only that the address holder and the domain holder agree. It says nothing about what the client intends to do.
Three ways a bot proves its addresses
Cloudflare's IP validation documentation covers two of the three methods BotBase now checks, and the Web Bot Auth page the third. They differ less in what they prove than in who has to hold what.
| Method | What the operator publishes | What a pass proves | Where it fails |
|---|---|---|---|
| Published IP list | "A fixed and limited set of IP addresses" in plain text, JSON or CSV, "used solely by the bot owner", plus a user-agent pattern | The request came from an address the operator claims | Lists go stale, and shared cloud egress cannot be listed without listing everyone else on it |
| Reverse DNS | "A list of domain suffixes to validate DNS records", PTR records "set correctly", plus a user-agent pattern | The reverse zone holder for that address named the operator's domain | No PTR on rented cloud space; wildcard or synthetic IPv6 PTRs; a forged PTR if the check skips the forward step |
| Web Bot Auth | An Ed25519 public key in a JSON Web Key Set at /.well-known/http-message-signatures-directory; each request carries Signature, Signature-Input and Signature-Agent headers per RFC 9421 | The request was signed by whoever controls that key directory, independent of the source IP | Every fetcher must sign every request; a leaked key signs for the attacker until it is rotated |
The reverse DNS row deserves a closer look, because Cloudflare's description of it is thinner than the standard. The documentation says Cloudflare collects the addresses it observes with the bot's user agent, runs a reverse lookup, and "if the reverse DNS of an IP resolves to one of the provided domains, Cloudflare considers it valid and stores it". It does not mention the forward step. We would assume Cloudflare confirms forward and left it out of the public page, but anyone building their own verifier should not copy the shorter description.
The order in which these methods appeared tells a story. IP lists and reverse DNS both worked because crawlers ran on the operator's own address space: Google can write googlebot.com into the PTRs of 66.249.64.0/19 because Google holds the zone. Then crawlers moved to rented cloud capacity where nobody gets to set a PTR and the egress range is shared with a thousand other tenants. Cloudflare's July 2025 post adding message signatures to its verified bots program says as much: the old checks rely on "IP address range (which may be shared by other services, or change over time)". Web Bot Auth exists because the reverse tree cannot describe a bot that has no address of its own. Our walkthrough of verifying Googlebot by IP covers the range files.
Mail receivers demanded this first
None of this is new to anyone who has run an outbound mail server. RFC 1912, from February 1996, already said "for every IP address, there should be a matching PTR record in the in-addr.arpa domain". Thirty years later Google's sender guidelines require that a sending SMTP server's public IP "must have a corresponding PTR record that resolves to a hostname" and that the same hostname has an A or AAAA record "that resolves to the same public IP address". That is FCrDNS, stated as a condition of delivery to Gmail.
The mail world also shows what rDNS tells you when it is present but generic. Hoang and colleagues, in a 2020 study built on the Rapid7 reverse DNS dataset for the whole public IPv4 space (Assessing the Privacy Benefits of Domain Name Encryption), counted 1.27 billion IPv4 addresses with a PTR record. Only about 172 million of them, 14 percent, pointed at a meaningful hostname. The rest were the dash-separated address encodings hosting providers generate in bulk, their example being 54.69.253.182 resolving to ec2-54-69-253-182.us-west-2.compute.amazonaws.com. More than 3 million PTRs carried a mail prefix, put there because receivers refuse messages without one.
A generic PTR like that one is still information. It names the provider and often the region, and the ranges that carry such names are datacenter space rather than households, one of the inputs to a hosting classification alongside the announcing ASN. What it cannot do is tell two tenants of the same provider apart. And its absence says nothing at all.
Checking rDNS from a terminal and from an API
The terminal version is the two commands shown above: dig -x builds the in-addr.arpa or ip6.arpa name for you and asks for the PTR. Do the forward lookup yourself, compare, and treat a missing PTR as "unknown" rather than "bad". To see what a single address says about itself, our reverse DNS lookup tool runs the live query for you.
At scale you do not want a DNS round trip inside every request. The Ipregistry API returns the PTR hostname alongside the network and location data when you add hostname=true to a lookup. It is a fresh resolution, so the call is a little slower than a plain query:
https://api.ipregistry.co/66.249.66.1?key=YOUR_API_KEY&hostname=true
{
"ip": "66.249.66.1",
"hostname": "crawl-66-249-66-1.googlebot.com",
"connection": { "asn": 15169, "organization": "Google LLC", "type": "hosting" }
}The PTR, the ASN and the connection type answer three different questions about the same address, and a verifier that keys on all three is far harder to fool than one that trusts any of them alone. Ipregistry serves them in a single response, and a free account comes with 20,000 lookups to get started.
Keep users in the loop Ship release notes that get read. Try Noticeable