There has been a case where the CI was broken on Friday and no one
noticied or fixed it and a RC build was built with broken tests.
The tests were disabled due to unknown test failures that have since
been fixed.
Also update the machine type used by GCB to be more powerful. This is
necessary for the tests to past because N1_HIGHCPU_8 is RAM constraint
and the tests crashes. I updated all jobs to use the new type which
hopefully will make the build faster as well.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1130)
<!-- Reviewable:end -->
Defined Docker image for schema deployment.
Included schema deploymer docker in the Cloud Build release process.
Defined Cloud Build config for schema deployment.
TESTED=Used cloud-build-local to test deployment flow
TESTED=Used docker to test schema deployer image in more ways
* Merge beam and GAE configs deployment to one GCB job
Deployment of GAE configs requires that the credential used by gcloud to
have GAE admin role of the project to be managed. We do not want to
grant the GCB service account that role, because it would all *any* GCB
job to deploy anything to GAE. Instead we use a dedicated credential
originally created to deploy beam pipelines. This credential is
encrypted by KMS and stored on GCS. Since the beam pipeline deployment
GCB job already does the decryption, it make sense to add the config
deployment step there as well. The beam deployment steps are tweaked to
use the nomulus tool docker image instead of the jar file.
Also moved the content of deploy_configs_to_env.sh to the GCB yaml file
itself because the shell script is not uploaded to GC Bat the same time
as the yaml file when the job is triggered by Spinnaker.
Lastly, due to b/137891685, using GCB to deploy cron jobs does not work
as we cannot use service account credential to deploy to projects under
google.com.
* Add a Cloud Build task to update YAML configs
* CR responses
* Move config deployment to a script
* Pin builder version
* Create different beam and deploy-config files per environment
* Update comments and make a for loop
* Build docker image of nomulus tool
In the course of "gradle build", build a docker image of nomulus tool so that
users can run this to allow us to bundle the java version with the image.
* Update GCB beam deployment pipeline
Some of the texts are not really secerts because they are per-project.
Also changed the location of the credential file to `secerts` so that in
the future we may add more secerts in that folder.
The encrypted file is base64 encoded, consistent with how the proxy
certificates are encoded. Also made some changes to the other pipelines
to facilitate automation with Spinnaker
* Move terraform and kubernetes folder to be under proxy
There is no reason for them to be under proxy/src/... any more now that
we have a Gradle-idiomatic folder structure.
With https://github.com/spinnaker/spinnaker/issues/4048 Spinnaker now natively supports GCB. We are able to start a GCB job from Spinnaker, and also there is better support to consume GCB pub/sub messages. Some changes are made to remove the workaround no longer needed.
Two new workflows are added, one to rsync a GCS folder to live/ after the deployment is done (so that the nomulus.jar file can then be fetched to x20 by a []cron job), and the other to tag the proxy image as live once it is deployed.
Lastly, the docs/ folders are needed when running tests. Remove it from .gcloudignore so that when a test run is kicked off by running "gcloud builds submit" the folder is sent to GCB. Ideally .gcloudignore should be identical to .gitignore but since they both are version controlled it is hard it make one a symlink of another.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252625901
This makes it possible to pull the gradle binary from a trusted source when building the release artifacts.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245450562
The pipeline is broken into two. The first one is to be triggered when the public repo is tagged. It then tags the private repo, builds and upload the builder and base images, and push a new commit to the release (merged repo). This pipeline also does text manipulation on several files in the release repo to ensure that the images uploaded in this pipeline is always used to reproducibly build the release repo at the same commit.
The second pipeline is then triggered by commit into the release repo, which builds, signs and uploads the proxy image.
Also updated the dependency lock files to use the latest plugins dependencies, which are uploaded to the GCS repo.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244666211
