Add instructions for two-step DB schema updates (#1283)

* Add instructions for two-step DB schema updates

These expanded steps are required by the recent enabling of the SQL integration test suite.
This commit is contained in:
Ben McIlwain 2021-08-13 17:21:38 -04:00 committed by GitHub
parent aa132fa7c7
commit 8916d6fbaf

View file

@ -3,7 +3,7 @@
This project contains Nomulus's Cloud SQL schema and schema-deployment
utilities.
### ER Diagrams
### Entity Relationship (ER) diagrams
The following links are the ER diagrams generated from the current SQL schema:
@ -14,7 +14,7 @@ shows all columns, foreign keys and indexes.
shows only significant columns, such as primary and foreign key columns, and
columns that are part of unique indexes.
### Database Roles and Privileges
### Database roles and privileges
Nomulus uses the 'postgres' database in the 'public' schema. The following
users/roles are defined:
@ -31,14 +31,18 @@ users/roles are defined:
* Reporting job user and individual human readers may be granted this
role.
### Schema DDL Scripts
### How to update the schema
Currently we use Flyway for schema deployment. Versioned incremental update
scripts are organized in the src/main/resources/sql/flyway folder. A Flyway
'migration' task examines the target database instance, and makes sure that only
changes not yet deployed are pushed.
Below are the steps to submit a schema change:
Because we have SQL integration tests enabled to ensure that deployments are
rollback-safe, which prevent Java code from executing against a version of the
schema it is incompatible with, you will need to commit your schema additions in
two separate PRs, with a wait for a deployment in-between, as explained in the
following steps:
1. Make your changes to entity classes, remembering to add new ones to
`core/src/main/resources/META-INF/persistence.xml` so they'll be picked up.
@ -76,6 +80,16 @@ Below are the steps to submit a schema change:
You'll want to have a look at the diffs in the golden schema to verify that
all changes are intentional.
6. Now, split your outstanding changes into two PRs. The first PR should _only_
include your new Flyway version `.sql` file, its addition to the `flyway.txt`
index, changes to the `nomulus.golden.sql` schema file, and changes to the
Entity Relationship diagram `.html` files. The second PR should include
everything else, including _all_ changes to `.java` files and the
`db-schema.sql.generated` changes that derive from them.
7. Submit the first PR and wait until it is successfully deployed to production,
then submit the second PR. Note, if you are removing things from the schema
(rather than adding them), then these PRs should be in the opposite order:
Java changes first, then SQL changes afterwards.
Relevant files (under `db/src/main/resources/sql/schema/`):
@ -91,7 +105,16 @@ example, when adding a new column to a table, we would deploy the change before
adding it to the relevant ORM class. Therefore, for a short time the golden file
will contain the new column while the generated one does not.
### Schema Push
Note that, when making schema changes, you _cannot_ add a new `NOT NULL` column
to an existing table that does not have a default value, or make any other
similar addition of a constraint that will be violated by existing data. If you
wish to rename a column, you must first add a new column with the desired name,
copy over its contents using a `@PostLoad` action in Java, re-save all rows,
update the Java to no longer contain the old column, wait for a deployment, and
then remove the old column. A rename operation requires the most complicated
series of steps to complete, as it is effectively an add followed by a remove.
### Schema push
Currently Cloud SQL schema is released with the Nomulus server, and shares the
server release's tag (e.g., `nomulus-20191101-RC00`). Automatic schema push
@ -126,7 +149,7 @@ following command to deploy the local schema,
./nom_build :db:flywayMigrate --dbServer=[alpha|crash] --environment=[alpha|crash]
```
#### Glass Breaking
#### Glass breaking
If you need to deploy a schema off-cycle, try making a release first, then
deploy that release schema to Cloud SQL.
@ -134,7 +157,7 @@ deploy that release schema to Cloud SQL.
TODO(weiminyu): elaborate on different ways to push schema without a full
release.
#### Notes On Flyway
#### Notes on Flyway
Please note: to run Flyway commands, you need Cloud SDK and need to log in once.
@ -154,7 +177,7 @@ The Flyway-based Cloud Build schema push process is safe in common scenarios:
* Concurrent deployment runs are safe. Flyway locks its own metadata table,
serializing deployment runs without affecting normal accesses.
#### Schema Push to Local Database
#### Schema push to local database
The Flyway tasks may also be used to deploy to local instances, e.g, your own
test instance. E.g.,