|
|
||
|---|---|---|
| apiserving/discoverydata | ||
| java/com/google | ||
| javascript | ||
| javatests/com/google/domain/registry | ||
| scripts | ||
| third_party | ||
| .gitignore | ||
| CONTRIBUTING.md | ||
| CONTRIBUTORS | ||
| LICENSE.txt | ||
| README.md | ||
| WORKSPACE | ||
Domain Registry
Domain Registry is a production service for managing registrations on top-level domains in a shared namespace. Domain Registry runs on Google App Engine and is written primarily in Java. It achieves in a hundred thousand lines of code what Jon Postel used to do on index cards.
This is the software that Google Registry uses to operate TLDs such as .GOOGLE, .HOW, .SOY, and .みんな.
What is a Registry?
When it comes to internet land, ownership flows down the following hierarchy:
- ICANN
- Registries (e.g. Google Registry)
- Registrars (e.g. Google Domains)
- Registrants (e.g. you)
A registry is any organization that operates an entire top-level domain. For example, Verisign controls all the .COM domains and Affilias controls all the .ORG domains.
How Scalable is Domain Registry?
We successfully verified that Domain Registry is able to perform 1,000 EPP "domain creates" per second, with 99th percentile latency at ~3 seconds, and 95th percentile latency at ~1 second. Please note that 1,000 was the highest QPS our load tester allowed.
In theory, Domain Registry is infinitely scalable. The only limitation is that each individual EPP resource can only support one write per second, which in practice, is more like ten. However reads to a single resource are free and unlimited.
How Reliable is Domain Registry?
Domain Registry achieves its scalability without sacrificing the level of correctness an engineer would expect from an ACID SQL database.
Domain Registry is built on top of Google Cloud Datastore. This is a global NoSQL database that provides an unlimited number of Paxos entity groups, each of which being able to scale to an unlimited size while supporting a single local transaction per second. Datastore also supports distributed transactions that span up to twenty-five entity groups. Transactions are limited to four minutes and ten megabytes in size. Furthermore, queries and indexes that span entity groups are always eventually consistent, which means they could take seconds, and very rarely, days to update. While most online services find eventual consistency useful, it is not appropriate for a service conducting financial exchanges. Therefore Domain Registry has been engineered to employ performance and complexity tradeoffs that allow strong consistency to be applied throughout the codebase.
Domain Registry has a commit log system. Commit logs are retained in datastore for thirty days. They are also streamed to Cloud Storage for backup purposes. Commit logs are written across one thousand entity group shards, each with a local timestamp. The commit log system is able to reconstruct a global partial ordering of transactions, based on these local timestamps. This is necessary in order to do restores. Each EPP resource entity also stores a map of its past mutations with 24-hour granularity. This makes it possible to have point-in-time projection queries with effectively no overhead.
The Registry Data Escrow (RDE) system is also built with reliability in mind. It executes on top of App Engine task queues, which can be double-executed and therefore require operations to be idempotent. RDE isn't idempotent. To work around this, RDE uses datastore transactions to achieve mutual exclusion and serialization. We call this the "Locking Rolling Cursor Pattern." One benefit of this pattern, is that if the escrow service should fall into a failure state for a few days, or weeks, it will automatically catch up on its work once the problem is resolved. RDE is also able to perform strongly consistent queries with snapshot isolation across the entire datastore. It does this by sharding global indexes into entity groups buckets (which can be queried with strong consistency) and then rewinding the entities to the desired point in time.
The Domain Registry codebase is also well tested. The core packages in the codebase (model, flows, rde, whois, etc.) have 95% test coverage.
Setup
All you have to do is install Bazel and clone this repository locally. Once that's done, here are some example commands to get you started:
# Run all tests
bazel test //java{,tests}/com/google/domain/registry/...
# Run the registry_tool command
bazel run //java/com/google/domain/registry/tool:registry_tool -- --help
# Run the Registrar Console on a local development server
bazel run //javatests/com/google/domain/registry/server -- --help
bazel run //javatests/com/google/domain/registry/server
google-chrome http://localhost:8080/registrar
Services
Domain Registry provides the following IETF standard services.
Extensible Provisioning Protocol (EPP)
EPP is the core service of the registry. It's an XML protocol that's used
by registrars to register domains from the registry on behalf of registrants.
Domain Registry implements this service as an App Engine HTTP servlet listening
on the /_dr/epp path. Requests are forwarded to this path by a public-facing
proxy listening on port 700.
To supplement EPP, Domain Registry also provides a public API for performing
domain availability checks. This service listens on the /check path.
RFCs
- RFC 5730: EPP
- RFC 5731: EPP Domain Mapping
- RFC 5732: EPP Host Mapping
- RFC 5733: EPP Contact Mapping
- RFC 3915: EPP Grace Period Mapping
- RFC 5734: EPP Transport over TCP
- RFC 5910: EPP DNSSEC Mapping
- Draft: EPP Launch Phase Mapping (Proposed)
Registry Data Escrow (RDE)
RDE and BRDA are implemented as a cron mapreduce that takes a strongly consistent point-in-time snapshot of the registration database, turns it into a gigantic XML file, and uploads it to an SFTP server run by a third party escrow provider. This happens nightly with RDE and weekly with BRDA.
This service exists for ICANN regulatory purposes. ICANN needs to know that, should a registry business ever implode, that they can quickly migrate their TLDs to a different company so that they'll continue to operate.
RFCs
- Draft: Registry Data Escrow Specification
- Draft: Domain Name Registration Data (DNRD) Objects Mapping
- Draft: ICANN Registry Interfaces
Trademark Clearing House (TMCH)
Domain Registry integrates with ICANN and IBM's MarksDB in order to protect trademark holders, when new TLDs are being launched.
RFCs
WHOIS
WHOIS is a simple text-based protocol that allows anyone to look up
information about a domain registrant. Domain Registry implements this as an
internal HTTP endpoint running on /_dr/whois. A separate proxy running on port
43 forwards requests to that path. Domain Registry also implements a public HTTP
endpoint that listens on the /whois path.
RFCs
Registration Data Access Protocol (RDAP)
RDAP is the new standard for WHOIS. It provides much richer functionality, such
as the ability to perform wildcard searches. Domain Registry makes this HTTP
service available under the /rdap/... path.