zydronium/google-nomulus
1
0
Fork 0
mirror of https://github.com/google/nomulus.git synced 2025-08-14 05:29:38 +02:00
Code Issues Projects Releases Packages Wiki Activity

Explain how to perform Datastore schema migrations in developing doc

Browse source 
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136522383
This commit is contained in:
mcilwain 2016-10-18 14:41:36 -07:00 • committed by Ben McIlwain
parent d355424f72
commit 43821f650e
1 changed files with 55 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

55
docs/developing.md
View file

@ -35,3 +35,58 @@ If you are adding new URL paths, or new directories of web-accessible resources,
you will need to make the corresponding changes in `RegistryTestServer`. This you will need to make the corresponding changes in `RegistryTestServer`. This
class contains all of the routing and static file information used by the local class contains all of the routing and static file information used by the local
development server. development server.
## Performing Datastore schema migrations
At some point in the development of a large product it will most likely become
necessary to perform a schema migration on data persisted to the database.
Because Datastore is a schema-less database, adding new fields is as simple as
writing to them, but changing the names or data formats of existing fields
requires more care, especially on a running production environment. The
Objectify documentation has a good [introductory
guide](https://github.com/objectify/objectify/wiki/SchemaMigration) to schema
migrations that is worth reading. Beyond that, you may want to use some of the
following techniques.
The requirements for a good schema migration are as follows:
* There must be no down-time or interruption of service.
* The upgrade must be rollback-safe, i.e. the new version of the app can be
reverted and everything will continue working.
In order to meet these requirements, a multiple-phase roll-out strategy is used
as follows:
1. Dual-write, reading from old field. Add the new Datastore fields along with
any required Datastore indexes. Use an `@OnSave` method on the entity to
copy over the contents of the old field to the new field every time the
entity is saved.
2. Deploy the new version of the app.
3. Re-save all affected entities in Datastore. For `EppResources` this can be
accomplished by running
[`ResaveAllEppResourcesAction`](https://github.com/google/nomulus/blob/master/java/google/registry/tools/server/ResaveAllEppResourcesAction.java);
for other entities you may need to write something custom. Re-saving all
entities forces the `@OnSave` method to fire for every entity, copying over
the contents of the old fields to the new fields. Any additional entities
that are created after the mapreduce is run will have the right values for
the new field because the `@OnSave` method is writing them.
4. Dual-write, now reading from new field. Switch over all places in the code
that are using the data to read from the new fields rather than from the old
fields. Adjust the `@OnSave` method so that it is copying over the contents
from the new field to the old field. Dual-writing to the old fields ensures
that it is safe to roll back to the prior version if necessary, since the
data it is expecting will still be there.
5. Deploy the new version of the app.
6. Delete the old fields, their indexes, and the `@OnSave` method.
7. Deploy the new version of the app. The schema migration is now complete.
The migration away from using a wrapper class around Keys on `DomainBase`
objects is instructive as an example:
* [Step
1](https://github.com/google/nomulus/commit/861fd60d2cb408ba2f9570d3881d316475b728c2),
which implements dual-writing.
* [Step
4](https://github.com/google/nomulus/commit/361a53a3c985c14539e5ec1a31cf4ad192f67a5d),
which switches over to using the new fields.
* Step 6, which removes the old fields (work pending).