Commit graph

41 commits

Author SHA1 Message Date
Michael Muller
6f75dfd116
Create a flyway index file and verify correctness (#819)
* Create a flyway index file and verify correctness

Create an index file (flyway.txt) containing the names of all of the flyway
files and verify that it is ordered and in sync with the actual contents of
the flyway directory.  Also provide a target (generateFlywayIndex) to
automatically generate it.

The purpose of flyway.txt is to cause a merge conflict in the event that two
different developers add a flyway file with the same sequence number, an event
which has occurred multiple times.
2020-09-29 11:26:05 -04:00
Weimin Yu
553d5717cb
Enhance the test for forbidden Schema changes (#815)
* Enhance the test for forbidden Schema changes

Current test is git-based. It is difficult to maintain and does not
catch out-of-order version numbers. It is also more aggressive than
necessary, failing on changes to submitted scripts that have not been
deployed yet.

The new test starts a database, deploys the current schema to it,
then deploys the set of Flyway scripts in this repository to the database.
2020-09-24 12:31:08 -04:00
Lai Jiang
090c233592
Some more changes to make it possible to use Java 11 to compile (#732)
Add the class paths of the source files generated by annotation processors to
the javadoc task's class path so that it doesn't complain about missing
Dagger classes.

Also remove empty <p> tags in all generated source files, because jaxb
genrerates files in multiple locations.

Lastly, for unkown reasons when the source level is set to > 8, the core
subproject throws a warning about a Gradle internal annotator processor
that only supports up to Java 8 and cause the Java compliation to fail because
we set -Werror on all java compliation tasks. I don't think there is a
strong reason that we set -Werror anyway, so this commit removes it.
2020-07-31 17:21:28 -04:00
Lai Jiang
8750c07fef
Clean up generated sources configuration (#724)
1. The Gradle apt plugin is no longer needed to process annotations.
2. Without the apt plugin, Gralde puts the source files generated by
   annotation processors in build/generated/sources/annotationProcessor.
3. Change the location of custom generated files to be consistent.
4. Fix a javadoc formatting error.
2020-07-30 15:43:42 -04:00
Ben McIlwain
caa0cd9d61
Add a "coreDev" gradle target (#667)
* Add a "buildFmt" gradle target

This does the same thing as the automatic Java build target, except instead of
failing if the code formatting isn't correct, it just automatically reformats as
necessary and continues on.

* Remove unnecessary mustRunAfters

* Make it run tests too, and add :taskTree task

* Rename task to coreDev and remove run afters

* Add task tree dependency

* Actually that may not be necessary
2020-07-10 10:03:59 -04:00
Lai Jiang
da65a38782
Add a GCB job to build and publish javadoc (#609) 2020-06-05 13:00:15 -04:00
Lai Jiang
bd443633f6
Add a task to compile javadoc across all packages (#597)
Also fixes various issues that prevent javadoc compliation.
2020-05-27 10:33:46 -04:00
Weimin Yu
6bd50421bc
Fix broken builds when Maven Central is used (#509)
* Fix broken builds when Maven Central is used

Gradle 6.2.1 apparently introduces a behavior change wrt boolean
expression: empty string used to eval to false, but now evals to
true.

Pre Gradle 6.2.1, root project's Gradle properties apparently were
not set to buildSrc. Now they are passed on to buildSrc -- mavenUrl
in buildSrc changes from null to "".

Both changes break the project when mavenUrl and/or pluginsUrl are
not set on command line.

Also added junit.jupiter-api as testCompile dependencies to projects.
This is a directly used dependency, whose absence causes a Lint
warning.
2020-03-10 11:21:03 -04:00
Ben McIlwain
bbacdb9704
Start using JUnit 5 (#488)
* Start using JUnit 5

This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration

In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.

* Delete no longer needed lockfiles

* Merge branch 'master' into first-junit5
2020-02-19 18:29:59 -05:00
Lai Jiang
76603812e3
Create kzip file for Kythe cross-referencing (#424)
This is set up per b/141716384.

TESTED=Tested on alpha and successfully uploaded the merged kzip file to
GCS.
2020-01-03 16:57:39 -05:00
Weimin Yu
6aaf081489
Create a new app to hold GenerateSqlSchemaCommand (#409)
* Create a new app to hold GenerateSqlSchemaCommand

GenerateSqlSchemaCommand starts postgresql using testcontainer.
This makes junit etc a runtime dependency, allowing them to get
into release artifacts.

By moving this command to a separate tool, we can remove junit
etc as compile/runtime dependency.
2019-12-13 16:05:35 -05:00
Weimin Yu
f48e3933f5
Run cross-release SQL integration tests (#403)
* Run cross-release SQL integration tests

Run SQL integration tests across arbitrary schema and server
releases.

Refer to integration/README.md in this change for more information.

TESTED=Cloud build changes tested with cloud-build-local
       Used the published jars to test sqlIntegration task locally.
2019-12-12 13:47:49 -05:00
Lai Jiang
d55230933b
Upgrade to Gradle 6.0 (#364) 2019-11-13 13:17:29 -05:00
Weimin Yu
e9f20b3401
Delete obsolet gradle config in root project (#354)
* Delete obsolet gradle config in root project
2019-11-08 14:32:08 -05:00
Michael Muller
8ca8fff387
Refactor environment -> project map out of build (#300)
Move the project mapping into its own file so that it can be more easily
overriden.
2019-10-07 13:34:49 -04:00
Michael Muller
31b5abb9a1
Improve error handling for environment property (#295)
* Improve error handling for environment property

Improve the error messages that we get for a bad or missing environment
property.  Move the property processing into the main build so that we do it
only once and configure all of the appengine deployment tasks to check that a
project has been defined and print a friendly error if it hasn't.

Note that even if the check isn't configured, this change prevents deployment
because the gcpProject will be set to null.  It just won't print as useful an
error message.

Tested: ran appengineDeployAll with and without the environment property, ran
"build" to verify that we don't get any complaints for non-deployment targets.

* Changes for review.

* Changes for review

* Changes for review.

* Changes for review.
2019-10-04 13:23:36 -04:00
Weimin Yu
8123c53916
Fix dependency-locking config (#276)
* Fix dependency-locking config

Reenable dependency locking after a bug errorneouly turned it off.

Removed the guava-related workaround that forcefully resolve to
the -jre distribution.

Enabled locking for buildSrc by updating its property file.

Updated all lock files.
2019-09-19 11:37:20 -04:00
Weimin Yu
471ed7caa7
Use Flyway to deploy SQL schema to non-prod (#255)
* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.

Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.

* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.

Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Updated ClaimsList.java so that Hibernate-generated
schema would use the right types.

Using 'varchar(255)' instead of 'text' for string columns
for now. We will need to investigate how to force Hibernate
to use the desired types in all cases.

* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Corrected the type of ClaimsEntry's revision_id column.
It should be plain int8, not bigserial.

Make GenerateSqlSchemaCommand use a custom dialect that
converts all varchar type to 'text' and timestamp to
'timestamptz'.

* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.

Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.

* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.

Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.

* Use Flyway to deploy SQL schema to non-prod

Added Gradle tasks to deploy and drop schema in alpha
using Flyway.

Use a custome dialect in GenerateSqlSchemaCommand to
convert varchar type to 'text' and timestamp to 'timestamptz'.

Corrected ClaimsEntry's revision_id column type to int8.
This column tracks parent table's primary key and should
not be bigserial.
2019-09-06 16:29:49 -04:00
Ben McIlwain
859b70098c
Fix typo of word "dependencies" in gradle build files (#259)
* Fix typo of word "dependencies" in gradle build files
2019-09-05 16:52:46 -04:00
Michael Muller
0daa89ae25
Remove the "showAllOutput" property from the build (#247)
* Remove the "showAllOutput" property from the build

It doesn't work very well and has been superseded by "verboseTestOutput",
which does the same thing and more.
2019-08-30 10:43:07 -04:00
Weimin Yu
b5ef99a8f8
Only apply Google Java format to changed lines (#176)
* Only apply Google Java format to changed lines

* Only apply Google Java format to changed lines

* Only apply Google Java format to changed regions

Diffs are relative to origin/master.

Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
  will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
  format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
  javaIncrementalFormatApply is invoked.

These tasks work from the root directory and process the buildSrc directory
too.

The Spotless Java config is removed.

* Only apply Google Java format to changed regions

Diffs are relative to origin/master.

Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
  will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
  format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
  javaIncrementalFormatApply is invoked.

These tasks work from the root directory and process the buildSrc directory
too.

The Spotless Java config is removed.

* Only apply Google Java format to changed regions

Diffs are relative to origin/master.

Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
  will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
  format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
  javaIncrementalFormatApply is invoked.

These tasks work from the root directory and process the buildSrc directory
too.

The Spotless Java config is removed.

* Only apply Google Java format to changed regions

Diffs are relative to origin/master.

Three tasks are added:
- javaIncrementalFormatCheck is added to the build workflow, and
  will abort build if format violations are found.
- javaIncrementalFormatApply needs to be manually invoked to correct
  format violations, the same behavior as spotlessApply.
- javaIncrementalFormatDryRun shows the changes that would happen if
  javaIncrementalFormatApply is invoked.

These tasks work from the root directory and process the buildSrc directory
too.

The Spotless Java config is removed.
2019-08-29 16:04:34 -04:00
Lai Jiang
bf52a78e89
Update Gradle wrapper to 5.6 (#232)
* Update Gradle wrapper to 5.6

* Upgrade shadowJar to 5.1 and increase JVM heap size

* Use Gradle binary on GCS
2019-08-23 22:10:55 -07:00
Michael Muller
69cb852a9c
Add a "showErrorOutput" property (#237)
* Add a "showErrorOutput" property

Add a property to let us dump test output and final status in real-time to the
console.
2019-08-22 11:10:56 -04:00
Weimin Yu
ce791e8b97
Generate code coverage report (#216)
* Generate code coverage report

Enable jacoco, the official Gradle code coverage plugin.

The 'build' task will write a code coverage report to
build/reports/jacoco for each subproject that has tests.
We should consider publish periodical reports to a well known
location.

This change also defines a minimum coverage verification task.
The task is for experiment only, and is not added to the build
process yet.
2019-08-07 11:21:04 -04:00
Lai Jiang
4fd32c8b5c
Add a Gradle task to deploy the proxy (#214)
Also refactored some common code regarding the mapping between
environments and projects.

The script to update proxy deployments and kill all pods are lifted from
https://cs.corp.google.com/piper///depot/google3/domain/registry/tools/bashrc?l=163
2019-08-06 11:45:34 -04:00
gbrodman
3488f95990
Clean up Gradle stylings and fix issues IDed in Prober (#212) 2019-08-05 15:54:20 -04:00
Lai Jiang
2ae5aba7ff
Rename dependency_license.gradle file (#196)
The file name confused the licensee app
(https://github.com/licensee/licensee) that GitHub uses to detect
license files. It thinks this file is also a license file and was not
able to determine is type.
2019-07-26 14:19:08 -04:00
Lai Jiang
0ab0a8c2e6
Use positive booleans to improve readability (#190)
This allows us to use lgtm.com to scan for vulnerabilities.
2019-07-23 22:30:46 -04:00
Lai Jiang
30f6113b05
Upgrade to Gradle 5.5.1 (#178)
Also make the default wrapper type "all" instead of "binary". This is
helpful for IDEs to understand the gradle script.
2019-07-17 17:37:44 -04:00
Weimin Yu
62eab98921
Unused deps check (#171)
* Save for later

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.
2019-07-15 16:19:34 -04:00
gbrodman
b0f6a498fd
Bump the version of the AppEngine Gradle plugin (#169)
* Bump the version of the AppEngine Gradle plugin

* deploy should depend on deployAll
2019-07-15 11:25:23 -04:00
Weimin Yu
29e4d8de05
Check dependencies' open-source licenses (#165)
* Check dependencies' open-source licenses

Use jk1/Gradle-License-Report to verify that all
dependencies have open-source licenses.

Note that the following dependencies are not checked:

Dependencies of official Gradle plugins
Dependencies declared in buildscript block
Dependencies of jk1/Gradle-License-Report itself
2019-07-11 12:08:08 -04:00
gbrodman
0e8724a48f
Require the license in Gradle files (#149) 2019-07-02 11:47:35 -04:00
gbrodman
7ce37147c0
Refactor the license check to be a generic presubmit check (#131)
* Refactor the license check to be a generic presubmit check

This includes all of the old presubmits that were in our METADATA file
that are still valid.

* Responses to CR

Added docstrings, refactored the file finder, changed variable names,
and formatted the file

* More respones to CR
2019-06-27 10:08:27 -04:00
gbrodman
3a485a03a6
Add a license-checking Python script (#123)
* Add a license-checking Python script

* Responses to CR
2019-06-21 12:25:35 -04:00
jianglai
6d5777559f Put environment check in its separate task
The check for environment cannot be done when the task is defined,
otherwise running any task will fail when the check does not pass.
Instead, check the environment when the task executes.
2019-06-19 18:46:46 -04:00
jianglai
e7b00dc4ee Only allow commandine deployment to alpha 2019-06-19 18:46:46 -04:00
gbrodman
e1db339b88
Add Spotless to the Gradle build (#88)
* Add Spotless to the Gradle build

* spotless apply

* one dot instead of two
2019-06-19 16:24:46 -04:00
gbrodman
973b29bc7a
Move the Soy files to the resources folder (#120) 2019-06-19 12:10:57 -04:00
Gus Brodman
38a47ca7d0 Apply IDEA plugin 2019-06-14 17:19:58 -04:00
Gus Brodman
a7a983bfed Refactor to be more in line with a standard Gradle project structure 2019-06-13 09:41:11 -04:00
Renamed from gradle/build.gradle (Browse further)