google-nomulus/docs/configuration.md

# Configuration

There are multiple different kinds of configuration that go into getting a
working registry system up and running. Broadly speaking, configuration works in
two ways -- globally, for the entire sytem, and per-TLD. Global configuration is
managed by editing code and deploying a new version, whereas per-TLD
configuration is data that lives in Datastore in `Registry` entities, and is
updated by running `nomulus` commands without having to deploy a new version.

## Initial configuration

Here's a checklist of things that need to be configured upon initial
installation of the project:

*   Create Google Cloud Storage buckets (see the [Architecture
    documentation](./architecture.md) for more information).
*   Modify configuration files ("nomulus-config-*.yaml") for all environments
    you wish to deploy.

## Environments

Before getting into the details of configuration, it's important to note that a
lot of configuration is environment-dependent. It is common to see `switch`
statements that operate on the current `RegistryEnvironment`, and return
different values for different environments. This is especially pronounced in
the `UNITTEST` and `LOCAL` environments, which don't run on App Engine at all.
As an example, some timeouts may be long in production and short in unit tests.

See the [Architecture documentation](./architecture.md) for more details on
environments as used by Nomulus.

## App Engine configuration

App Engine configuration isn't covered in depth in this document as it is
thoroughly documented in the [App Engine configuration docs][app-engine-config].
The main files of note that come pre-configured in Nomulus are:

*   `cron.xml` -- Configuration of cronjobs
*   `web.xml` -- Configuration of URL paths on the webserver
*   `appengine-web.xml` -- Overall App Engine settings including number and type
    of instances
*   `datastore-indexes.xml` -- Configuration of entity indexes in Datastore
*   `queue.xml` -- Configuration of App Engine task queues
*   `application.xml` -- Configuration of the application name and its services

Cron, web, and queue are covered in more detail in the "App Engine architecture"
doc, and the rest are covered in the general App Engine documentation.

If you are not writing new code to implement custom features, is unlikely that
you will need to make any modifications beyond simple changes to
`application.xml` and `appengine-web.xml`. If you are writing new features, it's
likely you'll need to add cronjobs, URL paths, Datastore indexes, and task
queues, and thus edit those associated XML files.

The existing codebase is configured for running a full-scale registry with
multiple TLDs.  In order to deploy to App Engine, you will either need to
[increase your
quota](https://cloud.google.com/compute/quotas#requesting_additional_quota) to
allow for at least 100 running instances or reduce `max-instances` in the
backend `appengine-web.xml` files to 25 or less.

## Global configuration

Global configuration is managed through YAML files that are built with and
deployed in the app. The full list of config options and their default values
can be found in the [`default-config.yaml`][default-config] file. If you wish to
change any of these values, do not edit this file. Instead, edit the environment
configuration file named
`google/registry/config/files/nomulus-config-ENVIRONMENT.yaml`, overriding only
the options you wish to change. Nomulus ships with blank placeholders for all
standard environments.

You will not need to change most of the default settings. Here is the subset of
settings that you will need to change for all deployed environments, including
development environments. See [`default-config.yaml`][default-config] for a full
description of each option:

```yaml
appEngine:
  projectId: # Your App Engine project ID
  toolsServiceUrl: https://tools-dot-PROJECT-ID.appspot.com  # Insert your project ID
  isLocal: false  # Causes saved credentials to be used.

gSuite:
  domainName: # Your G Suite domain name
  adminAccountEmailAddress: # An admin login for your G Suite account
```

For fully-featured production environments that need the full range of features
(e.g. RDE, correct contact information on the registrar console, etc.) you will
need to specify more settings. The `nomulus-config-production-sample.yaml` file
contains an exhaustive list of all settings to override.

From a code perspective, all configuration settings ultimately come through the
[`RegistryConfig`][registry-config] class. This includes a Dagger module called
`ConfigModule` that provides injectable configuration options. While most
configuration options can be changed from within the yaml config file,
certain derived options may still need to be overriden by changing the code in
this module.

## OAuth 2 client id configuration

The open source Nomulus release uses OAuth 2 to authenticate and authorize
users. This includes the `nomulus` tool when it connects to the system to
execute commands. OAuth must be configured before you can use the `nomulus` tool
to set up the system.

OAuth defines the concept of a *client id*, which identifies the application
which the user wants to authorize. This is so that, when a user clicks in an
OAuth permission dialog and grants access to data, they are not granting access
to every application on their computer (including potentially malicious ones),
but only to the application which they agree needs access. Each environment of
the Nomulus system should have its own client id. Multiple installations of the
`nomulus` tool application can share the same client id for the same
environment.

There are three steps to configuration.

*   **Create the client id in App Engine:** Go to your project's
    ["Credentials" page](https://console.developers.google.com/apis/credentials)
    in the Developer's Console. Click "Create credentials" and select "OAuth
    client ID" from the dropdown. In the create credentials window, select an
    application type of "Desktop app". After creating the client id,
    copy the client id and client secret which are displayed in the popup
    window.  You may also obtain this information by downloading the json file
    for the client id.

*   **Copy the client secret information to the config file:** The *client
    secret file* contains both the client ID and the client secret. Copy the
    respective values to the config file for the environment that the credential
    is created for (e. g. `nomulus-config-production.yaml`) under the
    `registryTool` section. This will make the `nomulus` tool use this
    credential to authenticate itself to the system.

*   **Add the new client id to the configured list of allowed client ids:** The
    configuration files include an `oAuth` section, which defines a parameter
    called `allowedOauthClientIds`, specifying a list of client ids which are
    permitted to connect. Add the client ID to the list. You will need to
    rebuild and redeploy the project so that the configuration changes take
    effect.

Once these steps are taken, the `nomulus` tool will use a client id which the
server is configured to accept, and authentication should succeed. Note that
many Nomulus commands also require that the user have App Engine admin
privileges, meaning that the user needs to be added as an owner or viewer of the
App Engine project.

## Sensitive global configuration

Some configuration values, such as PGP private keys, are so sensitive that they
should not be written in code as per the configuration methods above, as that
would pose too high a risk of them accidentally being leaked, e.g. in a source
control mishap. We use a secret store to persist these values in a secure
manner, and abstract access to them using the `Keyring` interface.

The `Keyring` interface contains methods for all sensitive configuration values,
which are primarily credentials used to access various ICANN and ICANN-
affiliated services (such as RDE). These values are only needed for real
production registries and PDT environments. If you are just playing around with
the platform at first, it is OK to put off defining these values until
necessary. To that end, a `DummyKeyringModule` is included that simply provides
an `InMemoryKeyring` populated with dummy values for all secret keys. This
allows the codebase to compile and run, but of course any actions that attempt
to connect to external services will fail because none of the keys are real.

To configure a production registry system, you will need to either use the
KmsKeyring or write your own replacement module using `DummyKeyringModule` for
guidance.  Such a module should provide either an instance of
`InMemoryKeyring` or your own custom implementation of `Keyring`.

In either case, configure the `keyring` section of the config file with the
appropriate parameters.  Use an `activeKeyring` of "KMS" with a project id for
KMS to configure accordingly, for example:

    keyring:
      activeKeyring: KMS
      kms:
        projectId: acme-registry-keys

## Per-TLD configuration

`Registry` entities, which are persisted to Datastore, are used for per-TLD
configuration. They contain any kind of configuration that is specific to a TLD,
such as the create/renew price of a domain name, the pricing engine
implementation, the DNS writer implementation, whether escrow exports are
enabled, the default currency, the reserved label lists, and more. The `nomulus
update_tld` command is used to set all of these options. See the [admin tool
documentation](./admin-tool.md) for more information, as well as the
command-line help for the `update_tld` command. Unlike global configuration
above, per-TLD configuration options are stored as data in the running system,
and thus do not require code pushes to update.

[app-engine-config]: https://cloud.google.com/appengine/docs/java/configuration-files
[default-config]: https://github.com/google/nomulus/blob/master/java/google/registry/config/files/default-config.yaml
[registry-config]: https://github.com/google/nomulus/blob/master/java/google/registry/config/RegistryConfig.java

## Cloud SQL Configuration

Nomulus is in the process of being ported to Cloud SQL.  As such, parts of the
system already require access to Cloud SQL and the necessary configuration
must be applied.

### Create Postgres Cloud SQL Instance

You can create a cloud SQL instance using the gcloud command:

    $ gcloud sql instances create nomulus --database-version=POSTGRES_11 \
       --cpu=1 --memory=4G

Note that for a production instance, you will likely want to be far more
generous with both CPU and memory resources.

Now get the connection name for the new database:

    $ gcloud sql instances describe nomulus | grep connectionName
    connectionName: your-project:us-central1:nomulus

Copy the connection name into your configuration (see below).

Now set the password for the default user:

    $ gcloud sql users set-password postgres \
        --instance=nomulus --project=$PROJECT_ID \
        --prompt-for-password

Store this password somewhere secure.

Now create database users for the tool and for the backend.  First, you'll
need to create a password.  This can simply be a sequence of random
characters.  Write it to the file `/tmp/server.pass` (we'll use a single
password for the two user accounts here, you are encouraged to use different
passwords for your production systems).  Make sure that this file does not
contain a newline after the password.  Now create the two user accounts:

    $ gcloud sql users create nomulus --instance=nomulus \
      --project=$PROJECT_ID "--password=`cat /tmp/server.pass`"
    $ gcloud sql users create tool --instance=nomulus \
      --project=$PROJECT_ID "--password=`cat /tmp/server.pass`"

Now enable access to the Cloud SQL admin APIs:

    $ gcloud services enable sqladmin.googleapis.com \
        --project=$PROJECT_ID

### Installing the Schema

Google's Nomulus team makes use of Spinnaker-based continuous integration to
perform weekly pushes of both the Nomulus software and the SQL database
schema.  Organizations wishing to use the Nomulus software will likely want to
do something similar.  However, for purposes of this exercise we will push the
schema from the build system.

First, download the [Cloud SQL
Proxy](https://cloud.google.com/sql/docs/mysql/sql-proxy).  This will allow
you to connect to your database from a local workstation without a lot of
additional configuration.

Create a service account for use with the proxy:

    $ gcloud iam service-accounts create sql-proxy \
        --project=$PROJECT_ID \
        --description="Service account for use with Cloud SQL Proxy" \
        --display-name="Cloud SQL Proxy"

Give the service account admin permissions:

    $ gcloud projects add-iam-policy-binding $PROJECT_ID \
        --member=serviceAccount:sql-proxy@$PROJECT_ID.iam.gserviceaccount.com \
        --role=roles/cloudsql.admin

Create a JSON key for the service account:

    $ gcloud iam service-accounts keys create sql-admin.json \
        --project=$PROJECT_ID \
        --iam-account=sql-proxy@$PROJECT_ID.iam.gserviceaccount.com

Now start the proxy:

    $ PORT=3306   # Use a different value for this if you like.
    $ ./cloud_sql_proxy -credential_file=sql-admin.json \
        -instances=$PROJECT_ID:nomulus=tcp:$PORT
    2020/07/01 12:11:20 current FDs rlimit set to 32768, wanted limit is 8500. Nothing to do here.
    2020/07/01 12:11:20 using credential file for authentication; email=sql-proxy@pproject-id.iam.gserviceaccount.com
    2020/07/01 12:11:20 Listening on 127.0.0.1:3306 for project-id:nomulus
    2020/07/01 12:11:20 Ready for new connections

Finally, upload the new database schema:

    $ ./nom_build :db:flywayMigrate --dbServer=localhost:$PORT \
        --dbName=postgres --dbUser=nomulus --dbPassword=`cat /tmp/server.pass`

Now you'll need to give the "tool" user access to all tables.  You can do this
either with a locally installed version of PostgreSQL or from the Cloud Shell.
From local postgres, first, with your proxy is still running, connect using
psql.

    $ psql -h localhost -p 3306 postgres nomulus                                                                                                                         ~/w/nom.admin-docs
    Password for user nomulus: <enter the password from /tmp/server.pass>
    psql (12.2 (Debian 12.2-1+build2), server 11.6)
    Type "help" for help.

    postgres=>

Enter the following command at the postgres prompt:

    GRANT SELECT, INSERT, UPDATE, DELETE
    ON ALL TABLES IN SCHEMA public
    TO tool;

From the [Google Cloud Console](https://console.developers.google.com), click
the cloud shell icon in the toolbar (the ">_" icon).  You should be able to
connect to your database with gcloud:

    $ gcloud sql connect nomulus --user=nomulus

From this, you should have a postgres prompt and be able to enter the "GRANT"
command specified above.

### Cloud KMS

You'll need to choose a project to host the KMS keyring. Best practice is to
create a separate project with a strict IAM policy. However, you can use your
current project when experimenting.

You need to create a KMS keyring in the chosen project. The default keyring
name is 'nomulus', though you can override it in the config file.

    $ gcloud kms keyrings create "nomulus" --location "global" \
        --project $KEYS_PROJECT_ID

Next, you need to create two keys in the keyring:

    $ gcloud kms keys create "cloud-sql-password-string" \
        --project $KEYS_PROJECT_ID \
        --location "global" \
        --keyring "nomulus" \
        --purpose "encryption"

    $ gcloud kms keys create "tools-cloud-sql-password-string" \
        --project $KEYS_PROJECT_ID \
        --location "global" \
        --keyring "nomulus" \
        --purpose "encryption"

#### Install Cloud SQL Passwords in Nomulus Server

Use the update_kms_keyring command to upload the Cloud SQL passwords to the
Nomulus server.  We'll use the password same set of passwords we specified
above when creating database user accounts.  These should currently be stored
in `/tmp/server.pass`.

Paste the password for the Registry server user to a file, say
/tmp/server.pass. Make sure to avoid any trailing '\n' inserted by the editor.

    $ set ENV=alpha
    $ nomulus -e $ENV update_kms_keyring --keyname CLOUD_SQL_PASSWORD \
        --input /tmp/server.pass

Repeat the steps for the tools sql password:

    $ nomulus -e $ENV update_kms_keyring --keyname TOOLS_CLOUD_SQL_PASSWORD \
        --input /tmp/tools.pass

Use get_keyring_secret command to verify the data you put in:

    $ nomulus -e alpha -e alpha get_keyring_secret --keyname CLOUD_SQL_PASSWORD
    [your password]
    $ nomulus -e alpha -e alpha get_keyring_secret --keyname CLOUD_SQL_PASSWORD
    [your password]

#### The Relevant Parts of the Configuration File

    cloudSql:
      jdbcUrl: jdbc:postgresql://google/postgres
      username: nomulus
      instanceConnectionName: THE_NAME_SHOWN_ON_THE_DB_INFO_PAGE

    keyring:
      activeKeyring: KMS
      kms:
        projectId: KEYS_PROJECT_ID

    registryTool:
      clientId: TOOLS_OAUTH_CLIENT_ID
      clientSecret: TOOLS_OAUTH_SECRET
      username: tool