Improve logs in the GCP proxy

Tweaked a few logging levels to not spam error level logs. Also make it easy to debug issues in case relay retry fails.

[1] Put non-fatal exceptions that should be logged at warning in their explicit sets. Also always use the root cause to determine if an exception is non-fatal, because sometimes the actual causes are wrapped inside other exceptions.

[2] Record the cause of a relay failure, and record if a relay retry is successful. This way we can look at the log and figure out if a relay is eventually successful.

[3] Add a log when the frontend connection from the client is terminated.

[4] Alway close the relay channel when a relay has failed, which, depend on if the channel is frontend or backend, will reconnect and trigger a retry.

[5] Lastly changed failure test to use assertThrows instead of fail.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=208649916
This commit is contained in:
jianglai 2018-08-14 08:22:31 -07:00
parent b552c1d115
commit 0e64015cdf
10 changed files with 154 additions and 95 deletions

View file

@ -85,7 +85,23 @@ public class RelayHandlerTest {
}
@Test
public void testSuccess_outboundClosed_enqueueBuffer() {
public void testSuccess_frontClosed() {
inboundChannel.attr(RELAY_BUFFER_KEY).set(null);
inboundChannel.attr(PROTOCOL_KEY).set(backendProtocol);
outboundChannel.attr(PROTOCOL_KEY).set(frontendProtocol);
ExpectedType inboundMessage = new ExpectedType();
// Outbound channel (frontend) is closed.
outboundChannel.finish();
assertThat(inboundChannel.writeInbound(inboundMessage)).isFalse();
ExpectedType relayedMessage = outboundChannel.readOutbound();
assertThat(relayedMessage).isNull();
// Inbound channel (backend) should stay open.
assertThat(inboundChannel.isActive()).isTrue();
assertThat(inboundChannel.attr(RELAY_BUFFER_KEY).get()).isNull();
}
@Test
public void testSuccess_backendClosed_enqueueBuffer() {
ExpectedType inboundMessage = new ExpectedType();
// Outbound channel (backend) is closed.
outboundChannel.finish();