b19563f69d
This CL changes the Cloud Build flows to retrieve dependencies from our self-hosted GCS repository, to ensure that the release build are reproducible and hermetic (Note that it is still not truely reproducible as the dependency publishing process will override any existing artifacts in GCS with the current artifacts in Maven central. This is an issue that we should fix later). There are a couple of changes involved to get this working: 1. Changed internal repo location to pull from the new repo. 2. Remove jcenter repo. It is only used to pull in the docker gradle plugin, which is not used. We instead build the deploy jar file with Gradle and build the docker image with a Dockerfile. The docker gradle plugin artifacts uploaded to GCS cannot be read because it is using some special classifier which seems to not be preserved when uploading. The java application plugin is also removed because it is only used by the docker gradle plugin. 3. Removed netty tcnative library classifier. It does not appear to be actually used (the jar downloaded from Maven central is an uber jar) and the classifier again interferes with downloading the artifacts from GCS. 4. Removed the cyclic dependency of the util project on itself. It was added because the nebula linter wanted it, which I think is an erroneous warning which should be reported upstream. The cyclic dependency was not a problem before (for yet unknown reasons), but it seems like when we force the dependency resolution (by calling project.generateDependencyPublications during configuration stage) it exacerbated the hidden issue and caused a cyclic task dependency in the util project, which is fatal. Now Nebula will complain again, but the warning is considered benign and will not cause the build to fail. 5. Added the nebula dependency lock files. We need these files when using the GCS maven repo because the we only upload artifacts after conflict resolution to GCS. If both v1 and v2 of the same library are requested in the dependency graph, only one will be uploaded. If we do not have the lock files in place, when building from GCS maven repo, Gradle will try to first find both v1 and v2 in the repo (which fails because v1 is not present in the repo), before proceeding to select v2 to use. 6. Refactored the code to upload Maven artifacts to GCS. We need to manually edit the POM file to reproduce the dependencies for each artifact so that they are all put in the classpath during compilation. Before, the POM files do not have any dependency information, which causes compilation to fail because transitive dependencies are not loaded (even though they are present in the GCS repo). ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=233408051 |
||
|---|---|---|
| .. | ||
| config/checkstyle | ||
| core | ||
| gradle/wrapper | ||
| node_modules | ||
| proxy | ||
| services | ||
| third_party | ||
| util | ||
| z/pulldeps/google3/third_party/java_src/gtld/gradle | ||
| build.gradle | ||
| dependencies.gradle | ||
| gradlew | ||
| gradlew.bat | ||
| package.json | ||
| README.md | ||
| settings.gradle | ||
This folder contains experimental Gradle scripts as an alternative to Bazel for the open-source Nomulus project. These are work-in-progress and are expected to evolve in the near future.
Gradle is configured to use the directory containing this file as root, but use the existing Nomulus source tree.
All testing is done with Gradle v5.1.1.
Notable Issues
Test suites (RdeTestSuite and TmchTestSuite) are ignored to avoid duplicate execution of tests. Neither suite performs any shared test setup routine, so it is easier to exclude the suite classes than individual test classes. This is the reason why all test tasks in the :core project contain the exclude pattern '"/TestCase.", "/TestSuite."'
Many Nomulus tests are not hermetic: they modify global state (e.g., the shared local instance of Datastore) but do not clean up on completion. This becomes a problem with Gradle. In the beginning we forced Gradle to run every test class in a new process, and incurred heavy overheads. Since then, we have fixed some tests, and manged to divide all tests into three suites that do not have intra-suite conflicts. We will revisit the remaining tests soon.
Note that it is unclear if all conflicting tests have been identified. More may be exposed if test execution order changes, e.g., when new tests are added or execution parallelism level changes.
Initial Setup
Install Gradle on your local host, then run the following commands from this directory:
# One-time command to add gradle wrapper:
gradle wrapper
# Start the build:
./gradlew build
From now on, use './gradlew build' or './gradlew test' to build and test your changes.
To upgrade to a new Gradle version for this project, use:
gradle wrapper --gradle-version version-number
Deploy to AppEngine
Use the Gradle task 'appengineDeploy' to build and deploy to AppEngine. For now you must update the appengine.deploy.project in build.gradle to your GCP project ID.
To deploy the Gradle build, you will need the Google Cloud SDK and its app-engine-java component.