Add short-term host/contact caching for high-QPS domain mutations

After investigating common domain create/update command usage
patterns by registrars, we noticed that it is frequent for a
given registrar to reuse both hosts (using a standardized set of
nameservers) as well as contacts (e.g. for privacy/proxy
services). With these usage patterns, potential per-registrar
throughput during high volume scenarios (i.e. first moments of
General Availability) suffers from hitting hot keys in Datastore.

The solution, implemented in this CL, is to add short-term
in-memory caching for contacts and hosts, analogous to how we are
already caching Registry and Registrar entities.  These new
cached paths are only used inside domain flows to determine
existence and deleted/pending delete status of contacts and
hosts. This is a potential loss of transactional consistency, but
in practice it's hard to imagine this having negative effects, as
contacts or hosts that are in use cannot be deleted, and caching
would primarily affect widely used contacts and hosts.

Note that this caching can be turned on or off through a
configuration option, and by default would be off. We'd only want
it on when we really needed it, i.e. during a big launch.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187093378
This commit is contained in:
mcilwain 2018-02-26 15:42:45 -08:00 committed by jianglai
parent cd9bd35a08
commit ce136f9285
13 changed files with 461 additions and 16 deletions

View file

@ -1018,6 +1018,10 @@ public final class RegistryConfig {
* asynchronously fails the delete). Without this delay, the mapreduce might have started before
* the domain flow committed, and could potentially miss the reference.
*
* <p>If you are using EPP resource caching (eppResourceCachingEnabled in YAML), then this
* duration should also be longer than that cache duration (eppResourceCachingSeconds).
*
* @see google.registry.config.RegistryConfigSettings.Caching
* @see google.registry.flows.async.AsyncFlowEnqueuer
*/
@Provides
@ -1324,6 +1328,27 @@ public final class RegistryConfig {
return CONFIG_SETTINGS.get().caching.staticPremiumListMaxCachedEntries;
}
public static boolean isEppResourceCachingEnabled() {
return CONFIG_SETTINGS.get().caching.eppResourceCachingEnabled;
}
@VisibleForTesting
public static void overrideIsEppResourceCachingEnabledForTesting(boolean enabled) {
CONFIG_SETTINGS.get().caching.eppResourceCachingEnabled = enabled;
}
/**
* Returns the amount of time an EPP resource or key should be cached in memory before expiring.
*/
public static Duration getEppResourceCachingDuration() {
return Duration.standardSeconds(CONFIG_SETTINGS.get().caching.eppResourceCachingSeconds);
}
/** Returns the maximum number of EPP resources and keys to keep in in-memory cache. */
public static int getEppResourceMaxCachedEntries() {
return CONFIG_SETTINGS.get().caching.eppResourceMaxCachedEntries;
}
/** Returns the email address that outgoing emails from the app are sent from. */
public static String getGSuiteOutgoingEmailAddress() {
return CONFIG_SETTINGS.get().gSuite.outgoingEmailAddress;