Use 10 workers instead of the default 100 for re-save all EPP resources (#904)

* Use 10 workers instead of the default 100 for re-save all EPP resources

The intended/desired effect is to have a larger number of GCS commit log diffs
spread out over a longer period of time, with each diff itself being
significantly smaller. This should retain roughly the same amount of total work
for the async Cloud SQL replication action to have to deal with, but spread
across 10X as much time.
This commit is contained in:
Ben McIlwain 2020-12-10 15:25:25 -05:00 committed by GitHub
parent 83ed448741
commit 6a9929019a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -50,11 +50,21 @@ public class ResaveAllEppResourcesAction implements Runnable {
@Inject Response response; @Inject Response response;
@Inject ResaveAllEppResourcesAction() {} @Inject ResaveAllEppResourcesAction() {}
/**
* The number of shards to run the map-only mapreduce on.
*
* <p>This is less than the default of 100 because we only run this action monthly and can afford
* it being slower, but we don't want to write out lots of large commit logs in a short period of
* time because they make the Cloud SQL migration tougher.
*/
private static final int NUM_SHARDS = 10;
@Override @Override
public void run() { public void run() {
mrRunner mrRunner
.setJobName("Re-save all EPP resources") .setJobName("Re-save all EPP resources")
.setModuleName("backend") .setModuleName("backend")
.setDefaultMapShards(NUM_SHARDS)
.runMapOnly( .runMapOnly(
new ResaveAllEppResourcesActionMapper(), new ResaveAllEppResourcesActionMapper(),
ImmutableList.of(EppResourceInputs.createKeyInput(EppResource.class))) ImmutableList.of(EppResourceInputs.createKeyInput(EppResource.class)))