Commit graph

10 commits

Author SHA1 Message Date
mcilwain
6e74ba0587 Wrap ModulesService in new AppEngineServiceUtils
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).

This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.

This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
2018-08-10 13:44:25 -04:00
mcilwain
ded40851d3 Use locking on async mapreduces
This ensures that only one will run at a time, which should help fix the
clogged up mapreduces we've seen on sandbox.

In order to do this, the UnlockerOutput is introduced. This unlocks the
given Lock after all reducer shards have finished.

Also increases the lease duration of the DNS refresh action from 20 to
240 minutes. 20 minutes isn't long enough; when there's a lot of domains
and decent system load the mapreduce could take longer than that in the
ordinary case.

TESTED=Deployed to alpha and verified that more than one copy of the
mapreduce wouldn't run simultaneously, and also that the lock is
released when the mapreduce is finished.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205887554
2018-08-10 13:44:25 -04:00
mcilwain
6f2e663b72 Add asynchronous scheduled actions to re-save entities
This is used in the domain transfer and delete flows, both of which are
asynchronous flows that have implicit default actions that will be taken at some
point in the future. This CL adds scheduled re-saves to take place soon after
those default actions would become effective, so that they can be re-saved
quickly if so.

Unfortunately the redemption grace period on our TLDs is 35 days, which exceeds
the 30 day maximum task ETA in App Engine, so these won't actually fire.  That's
fine though; the deletion is actually effective as of 5 days, and this is just
removing the grace period.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201345274
2018-06-27 15:28:52 -04:00
mcilwain
5d80f124ca Remove unnecessary "throws" declarations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201058582
2018-06-18 18:17:56 -04:00
mcilwain
7dc224627f Automatically refactor more exception testing to use new JUnit rules
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178911894
2017-12-27 10:42:36 -05:00
mcilwain
03c782f38e Replace ExceptionRule with ExpectedException
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.

Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
2017-12-13 12:43:45 -05:00
mcilwain
bb67841884 Add metrics for async batch operation processing
We want to know how long it's actually taking to process asynchronous
contact/host deletions and DNS refreshes on host renames. This adds
instrumentation. Five metrics are recorded as follows:

* An incrementable metric for each async task processed (split out by
  type of task and result).
* Two event metrics for processing time between when a task is enqueued
  and when it is processed -- tracked separately for contact/host
  deletion and DNS refresh on host rename.
* Two event metrics for batch size every time the two mapreduces are
  run (this is usually 0). Tracked separately for contact/host deletion
  and DNS refresh on host rename.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157001310
2017-06-05 18:17:09 -04:00
nickfelt
65aaeccfc6 Fix TaskQueueHelper param matching for pull queue tasks
This fixes TaskQueueHelper methods and MatchableTaskInfo so that the .param() matching works for pull queues, by parsing the payload for URL-encoded parameters more liberally.  As such, it updates all the places where formerly we were hacking around this by manually constructing the expected payloads and using TaskMatcher.payload() instead.

It also adds a TaskQueueHelper.assertTasksEnqueued() overload that accepts an Iterable<TaskStateInfo> so that you can cleanly assert that a queue contains the same tasks that were returned via a previous call to getQueueInfo("queue").getTaskInfo().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156604901
2017-05-23 17:22:49 -04:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mountford
669c3dc975 Move the async [] classes out of the flows package
We should be able to remove the dependency on the App Engine [] library from the frontend service, since no []s actually run there. In order to do this, we need to remove the various []-reliant classes from the frontend service build. This CL begins the process by moving the two async "flows" to a different package which is not included in the frontend service.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142159568
2016-12-19 11:09:19 -05:00
Renamed from javatests/google/registry/flows/async/RefreshDnsOnHostRenameActionTest.java (Browse further)