Use 10 workers instead of the default 100 for re-save all EPP resources (#904)

* Use 10 workers instead of the default 100 for re-save all EPP resources

The intended/desired effect is to have a larger number of GCS commit log diffs
spread out over a longer period of time, with each diff itself being
significantly smaller. This should retain roughly the same amount of total work
for the async Cloud SQL replication action to have to deal with, but spread
across 10X as much time.
This commit is contained in:
Ben McIlwain 2020-12-10 15:25:25 -05:00 committed by GitHub
parent 1b4e9dcd3c
commit c66a061396

View file

@ -50,11 +50,21 @@ public class ResaveAllEppResourcesAction implements Runnable {
@Inject Response response;
@Inject ResaveAllEppResourcesAction() {}
/**
* The number of shards to run the map-only mapreduce on.
*
* <p>This is less than the default of 100 because we only run this action monthly and can afford
* it being slower, but we don't want to write out lots of large commit logs in a short period of
* time because they make the Cloud SQL migration tougher.
*/
private static final int NUM_SHARDS = 10;
@Override
public void run() {
mrRunner
.setJobName("Re-save all EPP resources")
.setModuleName("backend")
.setDefaultMapShards(NUM_SHARDS)
.runMapOnly(
new ResaveAllEppResourcesActionMapper(),
ImmutableList.of(EppResourceInputs.createKeyInput(EppResource.class)))