Add docs for Cloud Build status notification (#2157)

Add documentation that describes the current Cloud Build status notification
to Google Chat, as well as how to update the configuration and the
notifier service.

release/notifications/README.md

@@ -0,0 +1,42 @@
+## About
+
+We have deployed a Cloud Build notifier on Cloud Run that sends build failure
+notifications to a Google Chat space. This folder contains the configuration and
+helper scripts.
+
+## Details
+
+The instructions for notifier setup can be found on the
+[GCP documentation site](https://cloud.google.com/build/docs/configuring-notifications/configure-googlechat).
+For the **initial** setup, Cloud Build provides a
+[script](https://cloud.google.com/build/docs/configuring-notifications/automate)
+that automates a lot of the work.
+
+With the automated procedure, the notifier is deployed as a Cloud Run service
+named `googlechat-notifier` in a single region of user's choice. The notifier
+subscribes to the `cloud-builds` Pub/sub topic for build statuses, applies our
+custom filter, and sends matching notifications to our Google Chat space.
+
+Our custom configuration for the notifier is in the `googlechat.yaml` file. It
+defines:
+
+*   The build status filter, currently matching for all triggered builds that
+    have failed or timed out. The filter semantics are explained
+    [here](https://cloud.google.com/build/docs/configuring-notifications/configure-googlechat#using_cel_to_filter_build_events).
+*   The secret name of the Chat Webhook token stored in the Secret Manager. This
+    token allows the notifier to send notifications to our Chat space. The
+    webhook token can be managed in the Google Chat client.
+
+The `googlechat.yaml` configuration file should be uploaded to the GCS bucket
+`{PROJECT_ID}-notifiers-config`. The `upload_config.sh` script can be used for
+uploading. The new configuration will take effect eventually after all currently
+running notifier instances shutdown due to inactivity, which is bound to happen
+with our workload (Note: this depends on the scale-to-zero policy, which is the
+default).
+
+To make sure the new configurations take effect immediately, the
+`update_notifier.sh`. It deploys a new revision of the notifier and shuts down
+old instances.
+
+Note that if the notifier is moved to another region, the full setup must be
+repeated.

release/notifications/googlechat.yaml

@@ -0,0 +1,14 @@
+  apiVersion: cloud-build-notifiers/v1
+  kind: GoogleChatNotifier
+  metadata:
+    name: nomulus-cloudbuild-googlechat-notifier
+  spec:
+    notification:
+      filter: has(build.build_trigger_id) && build.status in [Build.Status.FAILURE, Build.Status.TIMEOUT]
+      delivery:
+        webhookUrl:
+          secretRef: webhook-url
+    secrets:
+    - name: webhook-url
+      value: projects/_project_id_/secrets/Chat-Webhook-CloudBuildNotifications/versions/latest
+

release/notifications/update_notifier.sh

@@ -0,0 +1,44 @@
+#!/bin/bash
+# Copyright 2019 The Nomulus Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# This script redeploys the Cloud Build notifier with the latest container
+# image. It can be used to force immediate configuration change after invoking
+# `upload_config.sh`.
+
+set -e
+
+if [ $# -ne 1 ];
+then
+  echo "Usage: $0 <project_id>"
+  exit 1
+fi
+
+project_id="$1"
+
+region=$(gcloud run services list \
+         --filter="SERVICE:googlechat-notifier" \
+         --format="csv[no-heading](REGION)" \
+         --project="${project_id}")
+
+SERVICE_NAME=googlechat-notifier
+IMAGE_PATH=us-east1-docker.pkg.dev/gcb-release/cloud-build-notifiers/googlechat:latest
+DESTINATION_CONFIG_PATH="gs://${project_id}-notifiers-config/googlechat.yaml"
+
+gcloud run deploy "${SERVICE_NAME}" --image="${IMAGE_PATH}" \
+    --no-allow-unauthenticated  \
+    --update-env-vars="CONFIG_PATH=${DESTINATION_CONFIG_PATH},PROJECT_ID=${project_id}" \
+    --region="${region}" \
+    --project="${project_id}" \
+    || fail "failed to deploy notifier service -- check service logs for configuration error"

release/notifications/upload_config.sh

@@ -0,0 +1,36 @@
+#!/bin/bash
+# Copyright 2019 The Nomulus Authors. All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# This script uploads the `googlechat.yaml` configuration to the GCS bucket used
+# by the Cloud Build notifier. The new config takes effect only AFTER all
+# currently running notifier instances are shut down due to inactivity. To force
+# immediate change, use `update_notifier.sh` in this directory to redeploy the
+# service.
+
+set -e
+
+if [ $# -ne 1 ];
+then
+  echo "Usage: $0 <project_id>"
+  exit 1
+fi
+
+project_id="$1"
+
+SCRIPT_DIR="$(realpath $(dirname $0))"
+
+cat "${SCRIPT_DIR}"/googlechat.yaml  \
+    | sed  "s/_project_id_/${project_id}/g" \
+    | gcloud storage cp - "gs://${project_id}-notifiers-config/googlechat.yaml"