mirror of
https://github.com/google/nomulus.git
synced 2025-04-30 03:57:51 +02:00
Expand list of system capabilities in README and add known issues
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=131074174
This commit is contained in:
mcilwain
Ben McIlwain
parent
87314c4469
commit
5dc1f716bd
3 changed files with 163 additions and 52 deletions
188
README.md
188
README.md
|
|
@ -8,6 +8,9 @@ code what [Jon Postel][postel] used to do on index cards.
|
|||
This is the software that [Google Registry][google-registry] uses to operate
|
||||
TLDs such as .GOOGLE, .HOW, .SOY, and .みんな.
|
||||
|
||||
For more in-depth documentation, including install and setup instructions, see
|
||||
the Markdown documents in the `docs` directory.
|
||||
|
||||
### What is a Registry?
|
||||
|
||||
When it comes to internet land, ownership flows down the following hierarchy:
|
||||
|
|
@ -75,29 +78,10 @@ consistency) and then rewinding the entities to the desired point in time.
|
|||
The Domain Registry codebase is also well tested. The core packages in the
|
||||
codebase (model, flows, rde, whois, etc.) have 95% test coverage.
|
||||
|
||||
## Capabilities
|
||||
|
||||
## Setup
|
||||
|
||||
All you have to do is [install Bazel][bazel-install] and clone this repository
|
||||
locally. Once that's done, here are some example commands to get you started:
|
||||
|
||||
```sh
|
||||
# Run all tests
|
||||
bazel test //java{,tests}/google/registry/...
|
||||
|
||||
# Run the registry_tool command
|
||||
bazel run //java/google/registry/tool:registry_tool -- --help
|
||||
|
||||
# Run the Registrar Console on a local development server
|
||||
bazel run //javatests/google/registry/server -- --help
|
||||
bazel run //javatests/google/registry/server
|
||||
google-chrome http://localhost:8080/registrar
|
||||
```
|
||||
|
||||
|
||||
## Services
|
||||
|
||||
Domain Registry provides the following IETF standard services.
|
||||
Domain Registry has the following capabilities, many of which are standard IETF
|
||||
services.
|
||||
|
||||
### Extensible Provisioning Protocol (EPP)
|
||||
|
||||
|
|
@ -105,21 +89,19 @@ Domain Registry provides the following IETF standard services.
|
|||
by registrars to register domains from the registry on behalf of registrants.
|
||||
Domain Registry implements this service as an App Engine HTTP servlet listening
|
||||
on the `/_dr/epp` path. Requests are forwarded to this path by a public-facing
|
||||
proxy listening on port 700.
|
||||
proxy listening on port 700. Poll message support is also included.
|
||||
|
||||
To supplement EPP, Domain Registry also provides a public API for performing
|
||||
domain availability checks. This service listens on the `/check` path.
|
||||
|
||||
#### RFCs
|
||||
|
||||
- [RFC 5730: EPP](http://tools.ietf.org/html/rfc5730)
|
||||
- [RFC 5731: EPP Domain Mapping](http://tools.ietf.org/html/rfc5731)
|
||||
- [RFC 5732: EPP Host Mapping](http://tools.ietf.org/html/rfc5732)
|
||||
- [RFC 5733: EPP Contact Mapping](http://tools.ietf.org/html/rfc5733)
|
||||
- [RFC 3915: EPP Grace Period Mapping](http://tools.ietf.org/html/rfc3915)
|
||||
- [RFC 5734: EPP Transport over TCP](http://tools.ietf.org/html/rfc5734)
|
||||
- [RFC 5910: EPP DNSSEC Mapping](http://tools.ietf.org/html/rfc5910)
|
||||
- [Draft: EPP Launch Phase Mapping (Proposed)](http://tools.ietf.org/html/draft-tan-epp-launchphase-11)
|
||||
* [RFC 5730: EPP](http://tools.ietf.org/html/rfc5730)
|
||||
* [RFC 5731: EPP Domain Mapping](http://tools.ietf.org/html/rfc5731)
|
||||
* [RFC 5732: EPP Host Mapping](http://tools.ietf.org/html/rfc5732)
|
||||
* [RFC 5733: EPP Contact Mapping](http://tools.ietf.org/html/rfc5733)
|
||||
* [RFC 3915: EPP Grace Period Mapping](http://tools.ietf.org/html/rfc3915)
|
||||
* [RFC 5734: EPP Transport over TCP](http://tools.ietf.org/html/rfc5734)
|
||||
* [RFC 5910: EPP DNSSEC Mapping](http://tools.ietf.org/html/rfc5910)
|
||||
* [Draft: EPP Launch Phase Mapping (Proposed)](http://tools.ietf.org/html/draft-tan-epp-launchphase-11)
|
||||
|
||||
### Registry Data Escrow (RDE)
|
||||
|
||||
|
|
@ -132,21 +114,17 @@ This service exists for ICANN regulatory purposes. ICANN needs to know that,
|
|||
should a registry business ever implode, that they can quickly migrate their
|
||||
TLDs to a different company so that they'll continue to operate.
|
||||
|
||||
#### RFCs
|
||||
|
||||
- [Draft: Registry Data Escrow Specification](http://tools.ietf.org/html/draft-arias-noguchi-registry-data-escrow-06)
|
||||
- [Draft: Domain Name Registration Data (DNRD) Objects Mapping](http://tools.ietf.org/html/draft-arias-noguchi-dnrd-objects-mapping-05)
|
||||
- [Draft: ICANN Registry Interfaces](http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05)
|
||||
* [Draft: Registry Data Escrow Specification](http://tools.ietf.org/html/draft-arias-noguchi-registry-data-escrow-06)
|
||||
* [Draft: Domain Name Registration Data (DNRD) Objects Mapping](http://tools.ietf.org/html/draft-arias-noguchi-dnrd-objects-mapping-05)
|
||||
* [Draft: ICANN Registry Interfaces](http://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-05)
|
||||
|
||||
### Trademark Clearing House (TMCH)
|
||||
|
||||
Domain Registry integrates with ICANN and IBM's MarksDB in order to protect
|
||||
trademark holders, when new TLDs are being launched.
|
||||
|
||||
#### RFCs
|
||||
|
||||
- [Draft: TMCH Functional Spec](http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08)
|
||||
- [Draft: Mark and Signed Mark Objects Mapping](https://tools.ietf.org/html/draft-lozano-tmch-smd-02)
|
||||
* [Draft: TMCH Functional Spec](http://tools.ietf.org/html/draft-lozano-tmch-func-spec-08)
|
||||
* [Draft: Mark and Signed Mark Objects Mapping](https://tools.ietf.org/html/draft-lozano-tmch-smd-02)
|
||||
|
||||
### WHOIS
|
||||
|
||||
|
|
@ -156,10 +134,8 @@ internal HTTP endpoint running on `/_dr/whois`. A separate proxy running on port
|
|||
43 forwards requests to that path. Domain Registry also implements a public HTTP
|
||||
endpoint that listens on the `/whois` path.
|
||||
|
||||
#### RFCs
|
||||
|
||||
- [RFC 3912: WHOIS Protocol Specification](https://tools.ietf.org/html/rfc3912)
|
||||
- [RFC 7485: Inventory and Analysis of Registration Objects](http://tools.ietf.org/html/rfc7485)
|
||||
* [RFC 3912: WHOIS Protocol Specification](https://tools.ietf.org/html/rfc3912)
|
||||
* [RFC 7485: Inventory and Analysis of Registration Objects](http://tools.ietf.org/html/rfc7485)
|
||||
|
||||
### Registration Data Access Protocol (RDAP)
|
||||
|
||||
|
|
@ -167,19 +143,127 @@ RDAP is the new standard for WHOIS. It provides much richer functionality, such
|
|||
as the ability to perform wildcard searches. Domain Registry makes this HTTP
|
||||
service available under the `/rdap/...` path.
|
||||
|
||||
#### RFCs
|
||||
* [RFC 7480: RDAP HTTP Usage](http://tools.ietf.org/html/rfc7480)
|
||||
* [RFC 7481: RDAP Security Services](http://tools.ietf.org/html/rfc7481)
|
||||
* [RFC 7482: RDAP Query Format](http://tools.ietf.org/html/rfc7482)
|
||||
* [RFC 7483: RDAP JSON Responses](http://tools.ietf.org/html/rfc7483)
|
||||
* [RFC 7484: RDAP Finding the Authoritative Registration Data](http://tools.ietf.org/html/rfc7484)
|
||||
|
||||
- [RFC 7480: RDAP HTTP Usage](http://tools.ietf.org/html/rfc7480)
|
||||
- [RFC 7481: RDAP Security Services](http://tools.ietf.org/html/rfc7481)
|
||||
- [RFC 7482: RDAP Query Format](http://tools.ietf.org/html/rfc7482)
|
||||
- [RFC 7483: RDAP JSON Responses](http://tools.ietf.org/html/rfc7483)
|
||||
- [RFC 7484: RDAP Finding the Authoritative Registration Data](http://tools.ietf.org/html/rfc7484)
|
||||
### Backups
|
||||
|
||||
The registry provides a system for generating and restoring from backups with
|
||||
strong point-in-time consistency. Datastore backups are written out once daily
|
||||
to Cloud Storage using the built-in Datastore snapshot export functionality.
|
||||
Separately, entities called commit logs are continuously exported to track
|
||||
changes that occur in between the regularly scheduled backups.
|
||||
|
||||
A restore involves wiping out all entities in Datastore, importing the most
|
||||
recent complete daily backup snapshot, then replaying all of the commit logs
|
||||
since that snapshot. This yields a system state that is guaranteed
|
||||
transactionally consistent.
|
||||
|
||||
### Billing
|
||||
|
||||
The registry performs a regular daily export of `BillingEvent` entities from
|
||||
Cloud Datastore, where they are stored and updated by the running system, to
|
||||
[BigQuery][bigquery], where they can be analyzed using SQL scripts to generate
|
||||
monthly invoices per registrar.
|
||||
|
||||
### High availablity with horizontal scaling
|
||||
|
||||
Because the registry runs on the Google Cloud Platform stack, it benefits from
|
||||
high availability, automatic fail-over, and horizontal auto-scaling of compute
|
||||
and database resources. This makes it quite flexible for running TLDs of any
|
||||
size.
|
||||
|
||||
### Automated tests
|
||||
|
||||
The registry codebase includes ~400 test classes with ~4,000 total unit and
|
||||
integration tests. This limits regressions, ensures correct system
|
||||
functionality, and allows for easy continued future development and refactoring.
|
||||
|
||||
### DNS
|
||||
|
||||
An interface for DNS operations is provided, along with a sample implementation
|
||||
that uses the [Google Cloud DNS](https://cloud.google.com/dns/) API. A bulk
|
||||
export tool is also provided to export a zone file for an entire TLD in BIND
|
||||
format.
|
||||
|
||||
* [RFC 1034: Domain Names - Concepts and Facilities](https://www.ietf.org/rfc/rfc1034.txt)
|
||||
* [RFC 1035: Domain Names - Implementation and Specification](https://www.ietf.org/rfc/rfc1034.txt)
|
||||
|
||||
### Exports
|
||||
|
||||
The registry uses background batch processes to periodically export information
|
||||
from the running system, including billing information, all EPP entities,
|
||||
backups, lists of all registered domain names, registrar contact emails,
|
||||
ICANN-mandated reports, database snapshots, and reserved terms.
|
||||
|
||||
### Metrics and reporting
|
||||
|
||||
The registry records metrics and regularly exports them to BigQuery so that
|
||||
analyses can be run on them using full SQL queries. Metrics include which EPP
|
||||
commands were run and when and by whom, information on failed commands, activity
|
||||
per registrar, and length of each request.
|
||||
|
||||
[BigQuery][bigquery] reporting scripts are provided to generate the required
|
||||
per-TLD monthly
|
||||
[registry reports](https://www.icann.org/resources/pages/registry-reports) for
|
||||
ICANN.
|
||||
|
||||
### Registrar console
|
||||
|
||||
The registry includes a web-based registrar console that registrars can access
|
||||
in a browser. It provides the ability for registrars to view their billing
|
||||
invoices in Google Drive, contact the registry provider, and modify WHOIS,
|
||||
security (including SSL certificates), and registrar contact settings. Main
|
||||
registry commands such as creating domains, hosts, and contacts must go through
|
||||
EPP and are not provided in the console.
|
||||
|
||||
### Admin tooling
|
||||
|
||||
The registry comes with a fully featured `registry_tool` command-line tool (see
|
||||
`docs/` for full documentation) that allows developers and support personnel of
|
||||
the registry to run a full range of commands, including creating new registrars,
|
||||
running arbitrary EPP commands, inspecting the state of important things in the
|
||||
system, and creating new TLDs.
|
||||
|
||||
### Plug-and-play pricing engines
|
||||
|
||||
The registry has the ability to configure per-TLD pricing engines to
|
||||
programmatically determine the price of domain names on the fly. An
|
||||
implementation is provided that uses the contents of a static list of prices
|
||||
(this being by far the most common type of premium pricing used for TLDs).
|
||||
|
||||
## Known issues
|
||||
|
||||
There are a few things that the registry cannot currently do, and a few things
|
||||
that are out of scope that it will never do.
|
||||
|
||||
* You will need a DNS system in order to run a fully-fledged registry. If you
|
||||
are planning on using anything other than Google Cloud DNS you will need to
|
||||
provide an implementation.
|
||||
* You will need an invoicing system to convert the internal registry billing
|
||||
events into registrar invoices using whatever accounts receivable setup you
|
||||
already have. A partial implementation is provided that generates generic CSV
|
||||
invoices (see `MakeBillingTablesCommand`), but you will need to integrate it
|
||||
with your payments system.
|
||||
* You will likely need monitoring to continuously monitor the status of the
|
||||
system. Any of a large variety of tools can be used for this, or you can
|
||||
write your own.
|
||||
* You will need a proxy to forward traffic on EPP and WHOIS ports to the HTTPS
|
||||
endpoint on App Engine, as App Engine only allows incoming traffic on
|
||||
HTTP/HTTPS ports. Similarly, App Engine does not yet support IPv6, so your
|
||||
proxy would have to support that as well if you need IPv6 support. Future
|
||||
versions of [App Engine Flexible][flex] should provide these out of the box,
|
||||
but they aren't ready yet.
|
||||
|
||||
[bigquery]: https://cloud.google.com/bigquery/
|
||||
[datastore]: https://cloud.google.com/datastore/docs/concepts/overview
|
||||
[gae]: https://cloud.google.com/appengine/docs/about-the-standard-environment
|
||||
[bazel-install]: http://bazel.io/docs/install.html
|
||||
[epp]: https://en.wikipedia.org/wiki/Extensible_Provisioning_Protocol
|
||||
[flex]: https://cloud.google.com/appengine/docs/flexible/
|
||||
[google-registry]: https://www.registry.google/
|
||||
[gtld]: https://en.wikipedia.org/wiki/Generic_top-level_domain
|
||||
[icann]: https://en.wikipedia.org/wiki/ICANN
|
||||
|
|
|
|||
|
|
@ -64,6 +64,24 @@ computer too badly.
|
|||
--local_resources=1000,3,1.0
|
||||
Executed 360 out of 360 tests: 360 tests pass.
|
||||
|
||||
## Running a development instance locally
|
||||
|
||||
`RegistryTestServer` is a lightweight test server for the registry that is
|
||||
suitable for running locally for development. It uses local versions of all
|
||||
Google Cloud Platform dependencies, when available. Correspondingly, its
|
||||
functionality is limited compared to a Domain Registry instance running on an
|
||||
actual App Engine instance. To see its command-line parameters, run:
|
||||
|
||||
$ bazel run //javatests/google/registry/server -- --help
|
||||
|
||||
Then to fire up an instance of the server, run:
|
||||
|
||||
$ bazel run //javatests/google/registry/server {your params}
|
||||
|
||||
Once it is running, you can interact with it via normal `registry_tool`
|
||||
commands, or view the registrar console in a web browser by navigating to
|
||||
http://localhost:8080/registrar .
|
||||
|
||||
## Deploying the code
|
||||
|
||||
You are going to need to configure a variety of things before a working
|
||||
|
|
|
|||
|
|
@ -9,6 +9,15 @@ if the most recent export failed. Its code lives inside the tools package
|
|||
(`java/google/registry/tools`), and is compiled by building the `registry_tool`
|
||||
target in the Bazel BUILD file in that package.
|
||||
|
||||
To build the tool and display its command-line help, execute this command:
|
||||
|
||||
$ bazel run //java/google/registry/tool:registry_tool -- --help
|
||||
|
||||
For future invocations you should alias the compiled binary in the
|
||||
`bazel-genfiles/java/google/registry` directory or add it to your path so that
|
||||
you can run it more easily. The rest of this guide assumes that it has been
|
||||
aliased to `registry_tool`.
|
||||
|
||||
The registry tool is always called with a specific environment to run in using
|
||||
the -e parameter. This looks like:
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue