* Minor python changes
Use dataclasses instead of attrs. The former is part of the standard lib
while the latter may need to be installed separately.
Also added python3 to the list of prerequisites.
* Create a flyway index file and verify correctness
Create an index file (flyway.txt) containing the names of all of the flyway
files and verify that it is ordered and in sync with the actual contents of
the flyway directory. Also provide a target (generateFlywayIndex) to
automatically generate it.
The purpose of flyway.txt is to cause a merge conflict in the event that two
different developers add a flyway file with the same sequence number, an event
which has occurred multiple times.
* Enhance the test for forbidden Schema changes
Current test is git-based. It is difficult to maintain and does not
catch out-of-order version numbers. It is also more aggressive than
necessary, failing on changes to submitted scripts that have not been
deployed yet.
The new test starts a database, deploys the current schema to it,
then deploys the set of Flyway scripts in this repository to the database.
* Update BEAM SDK to work with Java 11
Upgraded BEAM dependencies to 2.23.0.
Updated Spec11 and invoice pipelines:
- Added the required region parameter.
- Removed the workaround code for staging.
Verified that staging is successful in alpha:
./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_spec11_pipeline --project domain-registry-alpha'
and
./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_invoicing_pipeline'
* Upgrade App Engine and webserver tests from JUnit 4 to 5
* Fix most errors
* Merge branch 'master' into junit5ification
* Fix test server by extracting non-test setup/tear-down
* Merge branch 'master' into junit5ification
* Fix backup tests
* Don't createFile(); asCharSink does it
* Increase the timeout for all WebDriver tests to 60s (helps w/ flakiness)
Add a "pseudo-task" in nom_build to do the three step process of generating
the golden schema. In the course of this, add support for pseudo-tasks in
general, improve the database directory readme and make nom_build not call
gradlew if there are no tasks.
* End-to-end Datastore to SQL pipeline
Defined InitSqlPipeline that performs end-to-end migration from
a Datastore backup to a SQL database.
Also fixed/refined multiple tests related to this migration.
* Add JUnit Params and start using it
* Convert rest of RDE tests
* Don't check headers for generated tests
* Expand visibility to fix build breakage
* Bump JUnit versions to 5.6.2
* Fix flaky tests due to Entity name conflicts
Objectify siliently replaces current registration of a given kind
when another class is registered for this kind. There are
several TestObject classes in the current code base, which by
default are all mapped to the same kind.
Tests have only been flaky because impacted tests need to run
in specific orders for failures to happen. Using multiple executors
in Gradle also reduced the likely hood of errors. To reproduce the
problem run the following tests in order (e.g., by putting them in
a test suite):
1. ExportCommitLogDiffActionTest
2. CreateAutoTimestampTest
3. RestoreCommitLogsActionTest
In this PR, we
- Made sure all entities have unique kinds.
- Made all test entities register with AppEngineRule instead of directly
with ObjectifyService.
- Added code in AppEngineRule to check for re-registrations.
- Added presumit check for forbidden direct registration.
* Create a nom_build wrapper script
nom_build is a wrapper around ./gradlew. It's purpose is to help us
deal with properties. The main problem that it is trying to solve is
that when properties are specified using -P, we don't get an error if
the property we specify isn't correct. As a result, a user or a build
agent can launch a build with unintended parameters.
nom_build consolidates all of the properties that we define into a
python script where the properties are translated to flags (actual
gradlew flags are also proxied). It also generates the property file
and warns the user if the current properties file is out of sync with
the script and includes documentation on each of the properties.
* Start using JUnit 5
This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration
In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.
* Delete no longer needed lockfiles
* Merge branch 'master' into first-junit5
* Make Gradle dependency cache shareable in GCB
Make Gradle put its caches in the source tree so that
they can be preserved across steps. When left at their
default location, caches are lost after each step.
* Upgradle JUnit to 4.13
Removed third_party/junit folder and all usage of the
JunitBackPort class. As a result, third_party is no
longer a Gradle subproject.
Minor code changes were needed to work around an
error-prone pattern: multiple statement in assertThrows'
runnable lambda.
Also third_party/activation and third_party/jsch. These
dependencies are loaded from remote maven repo. The local
copies are not in use.
* Require explict tag when starting psql docker
Defined a util class to return docker tag of desired PSQL version.
Class is defined in ':db' and shared by ':db' and ':core'. Used
an artifact declaration to exclude unnecesary compile dependencies.
Added a presubmit check for instantiations without explicit tag.
* Move soyutils_usegoog.js out of node_modules
Everytime the npmInstall runs, it removes this file from node_modules.
Move it outside the folder to prevent this from happening.
* Move karma.conf.js and soyutils_usegooge.js
* Move karma.conf.js to be under core
"Public Domain" license must be reviewed case by case.
Removed blanket acceptance and named accepted dependencies
individually.
Also added a README file to warn about this license and WTFPL.
* Regenerate schema using tools command
Rerun GenerateSqlSchemaCommand to pick up RegistryLock
and naming strategy change.
Also updated a new license term which seems to just pop up.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.
Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.
Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.
Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Corrected the type of ClaimsEntry's revision_id column.
It should be plain int8, not bigserial.
Make GenerateSqlSchemaCommand use a custom dialect that
converts all varchar type to 'text' and timestamp to
'timestamptz'.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.
Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.
Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.
* Use Flyway to deploy SQL schema to non-prod
Added Gradle tasks to deploy and drop schema in alpha
using Flyway.
Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.
Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.
* Only apply Google Java format to changed lines
* Only apply Google Java format to changed lines
* Only apply Google Java format to changed regions
Diffs are relative to origin/master.
Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
javaIncrementalFormatApply is invoked.
These tasks work from the root directory and process the buildSrc directory
too.
The Spotless Java config is removed.
* Only apply Google Java format to changed regions
Diffs are relative to origin/master.
Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
javaIncrementalFormatApply is invoked.
These tasks work from the root directory and process the buildSrc directory
too.
The Spotless Java config is removed.
* Only apply Google Java format to changed regions
Diffs are relative to origin/master.
Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
javaIncrementalFormatApply is invoked.
These tasks work from the root directory and process the buildSrc directory
too.
The Spotless Java config is removed.
* Only apply Google Java format to changed regions
Diffs are relative to origin/master.
Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
javaIncrementalFormatApply is invoked.
These tasks work from the root directory and process the buildSrc directory
too.
The Spotless Java config is removed.
* Add a generate_schema command
Add a generate_schema command to nomulus tool and add the necessary
instrumentation to EppResource and DomainBase to allow us to generate a
proof-of-concept schema for DomainBase.
* Added forgotten command description
* Revert "Added forgotten command description"
This reverts commit 09326cb8ac.
(checked in the wrong file)
* Added fixes requested during review
* Add a todo to start postgresql container
Add a todo to start a postgresql container from generate_sql_command.
* Generate code coverage report
Enable jacoco, the official Gradle code coverage plugin.
The 'build' task will write a code coverage report to
build/reports/jacoco for each subproject that has tests.
We should consider publish periodical reports to a well known
location.
This change also defines a minimum coverage verification task.
The task is for experiment only, and is not added to the build
process yet.
* Create a Gradle task to run the test server
As an artifact of the old build system, the test server relies on having
the built registrar_(bin|dbg)*(\.css)?.js in place (see ConsoleUiAction
among others). As a result, we create a Gradle task that puts those
files into the correct, readable, location before running the test
server.
* Depend on assemble rather than build
* refactor gitignores
* Check dependencies' open-source licenses
Use jk1/Gradle-License-Report to verify that all
dependencies have open-source licenses.
Note that the following dependencies are not checked:
Dependencies of official Gradle plugins
Dependencies declared in buildscript block
Dependencies of jk1/Gradle-License-Report itself
* Refactor the license check to be a generic presubmit check
This includes all of the old presubmits that were in our METADATA file
that are still valid.
* Responses to CR
Added docstrings, refactored the file finder, changed variable names,
and formatted the file
* More respones to CR
