google-nomulus/release/cloudbuild-proxy.yaml
Weimin Yu 20ad27cfdd Use JSON API for Maven Repo on GCS (#483)
* Use JSON API for Maven Repo on GCS

The url pattern https://storage.googleapis.com/{Bucket}/{Path}
uses the legacy XML API, which seems to be less robust than
the JSON API. We have observed connection resets after a few
thousand-file download bursts over 30 minutes.

This PR changes all urls to registry's Maven repo on GCS to
gcs://{Bucket}/{Path}. Gradle uses the JSON API for such urls.

TESTED=In Cloud Build with local change
2020-02-12 14:03:50 -05:00

67 lines
2.9 KiB
YAML

# To run the build locally, install cloud-build-local first.
# You will need access to a private registry, so be sure to install the docker
# credential helper.
# See: https://cloud.google.com/cloud-build/docs/build-debug-locally
# Then run:
# cloud-build-local --config=cloudbuild-proxy.yaml --dryrun=false --substitutions TAG_NAME=[TAG] ..
# This will create a docker image named gcr.io/[PROJECT_ID]/proxy:[TAG] locally.
# The PROJECT_ID is the current project name that gcloud uses.
#
# To manually trigger a build on GCB, run:
# gcloud builds submit --config cloudbuild-proxy.yaml --substitutions TAG_NAME=[TAG] ..
#
# To trigger a build automatically, follow the instructions below and add a trigger:
# https://cloud.google.com/cloud-build/docs/running-builds/automate-builds
steps:
# Build the proxy docker image.
- name: 'gcr.io/${PROJECT_ID}/builder:latest'
args:
- ./gradlew
- :proxy:test
- :proxy:buildProxyImage
- -PmavenUrl=gcs://domain-registry-maven-repository/maven
- -PpluginsUrl=gcs://domain-registry-maven-repository/plugins
# Tag and push the image. We can't let Cloud Build's default processing do that for us
# because we need to push the image before we can sign it in the following step.
- name: 'gcr.io/${PROJECT_ID}/builder:latest'
entrypoint: /bin/bash
args:
- -c
- |
set -e
docker tag proxy gcr.io/${PROJECT_ID}/proxy:${TAG_NAME}
docker tag proxy gcr.io/${PROJECT_ID}/proxy:latest
docker push gcr.io/${PROJECT_ID}/proxy:${TAG_NAME}
docker push gcr.io/${PROJECT_ID}/proxy:latest
dir: 'proxy'
# Get the image digest, sign it and substitute in the digest in the tagging yaml file.
- name: 'gcr.io/${PROJECT_ID}/builder:latest'
entrypoint: /bin/bash
args:
- -c
- |
set -e
digest=$(gcloud container images list-tags gcr.io/${PROJECT_ID}/proxy \
--format="get(digest)" --filter="tags = ${TAG_NAME}")
gcloud --project=${PROJECT_ID} alpha container binauthz attestations \
sign-and-create --artifact-url=gcr.io/${PROJECT_ID}/proxy@$digest \
--attestor=build-attestor --attestor-project=${PROJECT_ID} \
--keyversion-project=${PROJECT_ID} --keyversion-location=global \
--keyversion-keyring=attestor-keys --keyversion-key=signing \
--keyversion=1
sed -i s/'$${_IMAGE}'/proxy/g release/cloudbuild-tag.yaml
sed -i s/':$${TAG_NAME}'/@$digest/g release/cloudbuild-tag.yaml
# Images to upload to GCR. Even though the image has already been uploaded, we still include it
# here so that the GCB pubsub message contains it (for Spinnaker to consume).
images: ['gcr.io/${PROJECT_ID}/proxy:${TAG_NAME}']
# Config files to upload to GCS.
artifacts:
objects:
location: 'gs://${PROJECT_ID}-deploy/${TAG_NAME}'
# This cannot be regexs because of how Spinnaker constructs artifact paths.
paths:
- 'proxy/kubernetes/proxy-*.yaml'
- 'release/cloudbuild-tag.yaml'
timeout: 3600s
options:
machineType: 'N1_HIGHCPU_8'