Make GCP proxy log in a Stackdriver logging compliant format

When not running locally, the logging formatter is set to convert the log record to a single-line JSON string that Stackdriver logging agent running in GKE will pick up and parse correctly.

Also removed redundant logging handler in the proxy frontend connection. They have two problems: 1) it is possible to leak PII when all frontend traffic is logged, such as client IPs. Even though this is less of a concern because the GCP TCP proxy load balancer masquerade source IPs. 2) We are only logging the HTTP request/response that the frontend connection is sending to/receiving from the backend connection, but the backend already has its own logging handler to log the same message that it gets from/sends to the GAE app, so the logging in the frontend connection does not really give extra information.
Logging of some potential PII information such as the source IP of a proxied connection are also removed.

Thirdly, added a k8s autoscaling object that scales the containers based on CPU load. The default target load is 80%. This, in connection with GKE cluster VM autoscaling, means that when traffic is low, we'll only have one VM running one container of the proxy.

Fixes a bug where the MetricsComponent generates a separate ProxyConfig that does not call parse method on the command line args passed, resulting default Environment always being used in constructing the metric reporter.

Lastly a little bit of cleaning of the MOE config script, no newlines are necessary as the BUILD are formatted after string substitution.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188029019
This commit is contained in:
jianglai 2018-03-06 08:55:35 -08:00
parent 09e9823b9d
commit 84eab90000
12 changed files with 317 additions and 51 deletions

View file

@ -0,0 +1,13 @@
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: proxy-autoscale
labels:
app: proxy
spec:
scaleTargetRef:
apiVersion: extensions/v1beta1
kind: Deployment
name: proxy-deployment
maxReplicas: 5
minReplicas: 1

View file

@ -42,7 +42,7 @@ spec:
- name: service-account
mountPath: /var/secrets/google
imagePullPolicy: Always
args: ["--log"]
args: ["--env", "alpha", "--log"]
env:
- name: GOOGLE_APPLICATION_CREDENTIALS
value: /var/secrets/google/service-account.json