* Use JSON API for Maven Repo on GCS
The url pattern https://storage.googleapis.com/{Bucket}/{Path}
uses the legacy XML API, which seems to be less robust than
the JSON API. We have observed connection resets after a few
thousand-file download bursts over 30 minutes.
This PR changes all urls to registry's Maven repo on GCS to
gcs://{Bucket}/{Path}. Gradle uses the JSON API for such urls.
TESTED=In Cloud Build with local change
* Merge beam and GAE configs deployment to one GCB job
Deployment of GAE configs requires that the credential used by gcloud to
have GAE admin role of the project to be managed. We do not want to
grant the GCB service account that role, because it would all *any* GCB
job to deploy anything to GAE. Instead we use a dedicated credential
originally created to deploy beam pipelines. This credential is
encrypted by KMS and stored on GCS. Since the beam pipeline deployment
GCB job already does the decryption, it make sense to add the config
deployment step there as well. The beam deployment steps are tweaked to
use the nomulus tool docker image instead of the jar file.
Also moved the content of deploy_configs_to_env.sh to the GCB yaml file
itself because the shell script is not uploaded to GC Bat the same time
as the yaml file when the job is triggered by Spinnaker.
Lastly, due to b/137891685, using GCB to deploy cron jobs does not work
as we cannot use service account credential to deploy to projects under
google.com.
* Build docker image of nomulus tool
In the course of "gradle build", build a docker image of nomulus tool so that
users can run this to allow us to bundle the java version with the image.
* Move terraform and kubernetes folder to be under proxy
There is no reason for them to be under proxy/src/... any more now that
we have a Gradle-idiomatic folder structure.
With https://github.com/spinnaker/spinnaker/issues/4048 Spinnaker now natively supports GCB. We are able to start a GCB job from Spinnaker, and also there is better support to consume GCB pub/sub messages. Some changes are made to remove the workaround no longer needed.
Two new workflows are added, one to rsync a GCS folder to live/ after the deployment is done (so that the nomulus.jar file can then be fetched to x20 by a []cron job), and the other to tag the proxy image as live once it is deployed.
Lastly, the docs/ folders are needed when running tests. Remove it from .gcloudignore so that when a test run is kicked off by running "gcloud builds submit" the folder is sent to GCB. Ideally .gcloudignore should be identical to .gitignore but since they both are version controlled it is hard it make one a symlink of another.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252625901
Refactor out the build and package logic to a reusable script. Also removed the gradle task flag to skip lint check, as failing lint check is no longer a fatal error.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245296563
This alleviates the needs to have Google credentials even when downloading from
a public bucket.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245290867
The image name in the manifest needs to be the same as the name that Spinnaker trigger catches. With the new release, Spinnaker now correctly recognizes gcr.io/${PROJECT_ID}/proxy as the image name.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244845037
The pipeline is broken into two. The first one is to be triggered when the public repo is tagged. It then tags the private repo, builds and upload the builder and base images, and push a new commit to the release (merged repo). This pipeline also does text manipulation on several files in the release repo to ensure that the images uploaded in this pipeline is always used to reproducibly build the release repo at the same commit.
The second pipeline is then triggered by commit into the release repo, which builds, signs and uploads the proxy image.
Also updated the dependency lock files to use the latest plugins dependencies, which are uploaded to the GCS repo.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244666211
