Improve efficiency of async contact and host deletion with batching

This allows handling of N asynchronous deletion requests simultaneously instead
of just 1.  An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once.  This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336

java/google/registry/config/ConfigModule.java

@@ -665,15 +665,33 @@ public final class ConfigModule {
     return config.getContactAutomaticTransferLength();
   }
 
-  @Provides
-  @Config("asyncDeleteFlowMapreduceDelay")
-  public static Duration provideAsyncDeleteFlowMapreduceDelay(RegistryConfig config) {
-    return config.getAsyncDeleteFlowMapreduceDelay();
-  }
-
   @Provides
   @Config("maxChecks")
   public static int provideMaxChecks(RegistryConfig config) {
     return config.getMaxChecks();
   }
+
+  /**
+   * Returns the delay before executing async delete flow mapreduces.
+   *
+   * <p>This delay should be sufficiently longer than a transaction, to solve the following problem:
+   * <ul>
+   *   <li>a domain mutation flow starts a transaction
+   *   <li>the domain flow non-transactionally reads a resource and sees that it's not in
+   *       PENDING_DELETE
+   *   <li>the domain flow creates a new reference to this resource
+   *   <li>a contact/host delete flow runs and marks the resource PENDING_DELETE and commits
+   *   <li>the domain flow commits
+   * </ul>
+   *
+   * <p>Although we try not to add references to a PENDING_DELETE resource, strictly speaking that
+   * is ok as long as the mapreduce eventually sees the new reference (and therefore asynchronously
+   * fails the delete). Without this delay, the mapreduce might have started before the domain flow
+   * committed, and could potentially miss the reference.
+   */
+  @Provides
+  @Config("asyncDeleteFlowMapreduceDelay")
+  public static Duration getAsyncDeleteFlowMapreduceDelay() {
+    return Duration.standardSeconds(90);
+  }
 }