* Use base64-encoded SQL credentials
Encode Cloud SQL credential files on gcs with base64,
to be consistent with our Cloud Build practices.
Also renamed a property that specifies where to publish
the schema jar. New name is schema_publish_repo.
The segregated test targets in core break the --tests filter. Fix this by
defining a "testFilter" property and creating the FilteringTest task type that
applies it to the property set by "--tests".
* Make Flyway schema task work with prod and sandbox
Also renamed the 'superuser' role to 'admin' since
we do not own super user in Cloud SQL.
* Allow pushing schema to all env with Flyway
Desktop schema push to production is needed in the short term.
Long-termly we need to decide if this should be kept for glass
breaking
Schema push to sandbox and production requires interactiveconfirmation.
Also fixed a typo in initialize_roles.sql.
* Add Bloom filters to the Cloud SQL PremiumList schema
They are slightly different from the existing Bloom filters stored in Datastore
in that they now use an ASCII String encoding rather than the more generic
CharSequence, and there is no maximum size (whereas we previously had to live
within the 1 MB max entity size for Datastore).
* Modify Cloud SQL user management scripts
Create readonly and readwrite roles that may be granted to users.
Also configured default privileges for tables created in the future.
Made sure arbitrary users may not create database or tables.
* Modify Cloud SQL user management scripts
Create readonly and readwrite roles that may be granted to users.
Also configured default privileges for tables created in the future.
Made sure arbitrary users may not create database or tables.
* Load persistence.xml classes before adding test entities
* Also use persistence.xml in GenerateSqlSchemaCommand
* Add exception message
* remove duplicate line
* Add initial support for persisting premium lists to Cloud SQL
This adds support to the `nomulus create_premium_list` command only; support for
`nomulus update_premium_list` will be in a subsequent PR.
The design goals for this PR were:
1. Do not change the existing codepaths for premium lists at all, especially not
on the read path.
2. Write premium lists to Cloud SQL only if requested (i.e. not by default), and
write to Datastore first so as to not be blocked by errors with Cloud SQL.
3. Reuse existing codepaths to the maximum possible extent (e.g. don't yet
re-implement premium list parsing; take advantage of the existing logic), but
also ...
4. Some duplication is OK, since the existing Datastore path will be deleted
once this migration is complete, leaving only the codepaths for Cloud SQL.
* Refactor out common logic
* Add DAO test
* Add tests for parsing premium lists
* Use containsExactly
* Code review changes
* Format
* Re-generate schema
* Fix column names
* Make some tests pass
* Add SQL migration scripts
* Fix test errors
* Add an explanation to dummied-out JPA init
Add a more elaborate explanation of why actual JpaTransactionManager
initialization was removed from the factory.
* Add a DAO for RegistryLock objects
* Add an index on verification code and remove old file
* Move to v4
* Use camelCase in index names
* Javadoc fixes
* Allow alteration of RegistryLock objects in-place
* save, load-modify, read in separate transactions
* Change the creation timestamp to be a CreateAutoTimestamp
* Add persistence.xml to the war files
* Always use the DummyJpaTransactionManager
Use the DJTM until we get all of the dependencies set up for all of the
environments.
This shouldn't affect any of the unit tests, these use the
JpaTransactionManagerRule to set up a local database and connection.
This fixes the App Engine build.
* Implement CreateAutoTimestampConverter
Implement a JPA-based converter for CreateAutoTimestamp, allowing us to
persist instances of this class.
Note that converters appear to be required to convert to and from database
types that are generally known to JDBC. For example, conversion to Timestamp
works, conversion to OffsetDateTime does not (even though this works through
the JDBC interface directly).
* Improve error handling for environment property
Improve the error messages that we get for a bad or missing environment
property. Move the property processing into the main build so that we do it
only once and configure all of the appengine deployment tasks to check that a
project has been defined and print a friendly error if it hasn't.
Note that even if the check isn't configured, this change prevents deployment
because the gcpProject will be set to null. It just won't print as useful an
error message.
Tested: ran appengineDeployAll with and without the environment property, ran
"build" to verify that we don't get any complaints for non-deployment targets.
* Changes for review.
* Changes for review
* Changes for review.
* Changes for review.
* Give JpaTransactionManagerRule more parameters
Allow users of the rule to add annotated classes and properties, both useful
for testing.
* Change in response to review.
* Changes for review.
* Move test EntityManagerFactory create method
Move the test create method into the JpaTransactionManagerRuleTest.
* Remove nomulus SQL dialect from G.S.S.Command
Remove NomulusPostgreSQLDialect from GenerateSqlSchemaCommand (it has been
moved to its own top-level class).
* Add maven-publish task for SQL schema jar
Add task to publish SQL schema jar with flyway scripts and
golden schema to a maven repo. This will be used
for pre-release testing in the future.
This task is not part of build and needs to be invoked explicitly.
User needs to provide schema_jar_repo and schema_version
properties.
* Merge branch 'master' of https://github.com/google/nomulus into publish-schema-jar
* Add maven-publish task for SQL schema jar
Add task to publish SQL schema jar with flyway scripts and
golden schema to a maven repo. This will be used
for pre-release testing in the future.
This task is not part of build and needs to be invoked explicitly.
User needs to provide schema_jar_repo and schema_version
properties.
Export of cl/270900150.
To refer to a KMS key or key ring, we should use the stable `.self_link`. Using `.id` instead provides an unstable identifier which may change (and it will change in the upcoming update of the google provider to 2.9.1). A change in the identifier will cause Terraform to destroy and recreate the key. Destroying the key means all data associated with it is lost; the key cannot be recreated.
This CL replaces `.id` with `.self_link`, so all of those problems will not happen. In addition, `prevent_destroy` protects the key against delete-and-recreate in general.
* Upgrade to Truth 1.0
Refactored fail(...) to assertWithMessage().fail().
Upgraded com.google.monitoring-client family of dependencies to 1.0.6
Also fixed bad use of io.StringIO (on binary buffer) recently introduced to
google-java-format-diff.py.
For some reason tools.jar (which we don't need in the runtime in the first
place) gets exploded readonly, causing subsequent builds to fail since they
can't overwrite it. This hack makes it writable at the end of explodeWar.
* Fix dependency-locking config
Reenable dependency locking after a bug errorneouly turned it off.
Removed the guava-related workaround that forcefully resolve to
the -jre distribution.
Enabled locking for buildSrc by updating its property file.
Updated all lock files.
* Output command test output as well as consuming it
CommandTestCase currently consumes stdout & stderr for the command being
tested. Unfortunately, this results in us not being able to see the command
output. Add an output splitter so that output gets written to the original
stream in addition to being captured.
A simpler approach would be to print the captured data after command
completion. However, this won't work for tests that become hung and also
won't display results in real-time.
Tested: Ran a command test with verboseTestOutput=true, verified that standard
output was visible.
* Save and restore original stdout/err in cmd tests
We have to restore the original stdout/stderr print streams otherwise we end
up nesting them across tests which eventually causes the RDE tests to OOM.
* Add RegistryLock schema to Flyway deployment folder
Added creation script of RegistryLock to Flyway deployment folder.
Fixed previous scripts (PremiumList- and ClaimsList-related) for
FK name change (cause by table name changes: names are quoted now).
We should consider generating foreign key names by ourselves.
Since the alpha database is empty, we dropped and recreated the schema.
Added instructions on how to submit new database incremental changes
in the README file.
Updated RegistryLock.java, removing unnecessary annotations:
- For most fields, the 'name=' property is no longer necessary not that
the naming strategy is in place. The exceptions are the two used in
the unique index.
- The @Column annotation is implicit.
* Force Gradle to use jre version of Guava
Guava is often specified by version only by other
dependencies. In such cases, Gradle usually resolves
it to the '-android' version, which lacks the collection
classes.
We use custom resolution strategy to force the selection
of the '-jre' version.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
* Add schema deployment tests
Updated flyway schema script files so that they reflect what is
currently deployed in alpha: ClaimsList and PremiumList related
elements.
Put post-schema-push pg_dump output in nomulus.golden.sql as the
authoritative schema. Also added test to verify that the schema
pushed by flyway will result in exactly the golden schema.
Upgraded testcontainers to 1.12.1.
Added a custom Truth subject for better diffing of multi-line
text blocks.
Removed claims_list.sql and premium_list.sql, as we do not have use for
them.
* Regenerate schema using tools command
Rerun GenerateSqlSchemaCommand to pick up RegistryLock
and naming strategy change.
Also updated a new license term which seems to just pop up.
