* Clean up Gradle Flyway tasks in :db
Simplified the command line by revising the semantics of some
properties.
Added examples of Flyway task invocations.
This script still uses the GCS file-based credential. We will migrate it
to the Secret Manager soon.
* Minor python changes
Use dataclasses instead of attrs. The former is part of the standard lib
while the latter may need to be installed separately.
Also added python3 to the list of prerequisites.
* Enhance the test for forbidden Schema changes
Current test is git-based. It is difficult to maintain and does not
catch out-of-order version numbers. It is also more aggressive than
necessary, failing on changes to submitted scripts that have not been
deployed yet.
The new test starts a database, deploys the current schema to it,
then deploys the set of Flyway scripts in this repository to the database.
Add a "pseudo-task" in nom_build to do the three step process of generating
the golden schema. In the course of this, add support for pseudo-tasks in
general, improve the database directory readme and make nom_build not call
gradlew if there are no tasks.
* End-to-end Datastore to SQL pipeline
Defined InitSqlPipeline that performs end-to-end migration from
a Datastore backup to a SQL database.
Also fixed/refined multiple tests related to this migration.
* Create a nom_build wrapper script
nom_build is a wrapper around ./gradlew. It's purpose is to help us
deal with properties. The main problem that it is trying to solve is
that when properties are specified using -P, we don't get an error if
the property we specify isn't correct. As a result, a user or a build
agent can launch a build with unintended parameters.
nom_build consolidates all of the properties that we define into a
python script where the properties are translated to flags (actual
gradlew flags are also proxied). It also generates the property file
and warns the user if the current properties file is out of sync with
the script and includes documentation on each of the properties.
