Commit graph

23 commits

Author SHA1 Message Date
cgoldfeder
9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
mcilwain
01bb3a30f2 Fix handling around size of batches of mapreduce entities to process
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149562105
2017-03-13 10:42:44 -04:00
mcilwain
d2ca4b7234 Refactor main loop of MapreduceEntityCleanupAction
This also tightens up some error-checking conditions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149552065
2017-03-13 10:38:21 -04:00
mountford
5d4287a375 Add mapreduce cleanup action
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149432516
2017-03-13 10:24:57 -04:00
cgoldfeder
91049d2c53 Replace 'host.getSubordinateHost() != null' with 'host.isSubordinate()'
This is a cleanup in preparation for the next change that does a lot
of work with subordinate hosts, to make it easier to reason about in
complex code.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146689904
2017-02-06 16:45:23 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
ctingue
f30b87d40e Add ExpandRecurringBillingEventsAction cron job
Note that this merely starts this MR on a daily schedule -- the billing queries that ultimately consume the synthetic OneTime events are filtering out the events at this time, so we're still relying on query-time expansion of Recurrings.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144450565
2017-01-18 11:03:32 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00
ctingue
8b297fa099 Clean up stray commented code in ExpandRecurringBillingEventsAction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143696298
2017-01-09 11:59:04 -05:00
ctingue
00dab551d7 Fix deleted domain bug in recurring billing expansion
Currently, the recurring billing expansion chokes when looking for existing OneTime events for domains that are deleted or transferred at the MR execution time. To fix this, change the FKI query to look for a key at either execution time or recurrence end time, whichever is earliest.

(This one is distinct from [] in that this bug occurs when domains are [] deleted.)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143693153
2017-01-09 11:59:04 -05:00
ctingue
0cf62be403 Fix recurring expansion when handling transferred or deleted domains
The recurring billing event expansion fails when handling domains that are transferred or deleted before the initial renew. Quietly ignore these recurring events.

(This one is distinct from [] in that this bug occurs when domains currently exist under new ownership.)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143594317
2017-01-09 11:59:04 -05:00
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
mountford
0afafeff03 Move VerifyEntityIntegrityAction to batch package
By moving []s into the batch package, which is not included in the frontend service, we pave the way to remove the dependency of frontend on the [] library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142265351
2016-12-19 11:09:20 -05:00
mountford
669c3dc975 Move the async [] classes out of the flows package
We should be able to remove the dependency on the App Engine [] library from the frontend service, since no []s actually run there. In order to do this, we need to remove the various []-reliant classes from the frontend service build. This CL begins the process by moving the two async "flows" to a different package which is not included in the frontend service.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142159568
2016-12-19 11:09:19 -05:00
mcilwain
2b7d580bb3 Run buildifier on codebase to format BUILD files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140362453
2016-11-28 18:15:21 -05:00
ctingue
35d88a9f8c Rethrow exceptions in ExpandRecurringBillingEventsAction
Also fixes bug in handling of future-dated Recurring billing events.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140048357
2016-11-28 11:34:30 -05:00
jart
2e81de9954 Make essential Bazel packages publicly visible
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
2016-10-24 11:57:00 -04:00
ctingue
3c36b6b928 Clean up billing MR work
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136196221
2016-10-14 17:47:46 -04:00
shikhman
f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
mcilwain
dfe0ba32cb Reduce Datastore load of delete prober data mapreduce
This eliminates the transactional load of ForeignKeyIndexes and
EppResourceIndexes, the latter of which was problematic because it is parented
on the EppResourceIndexBucket entity group, and can cause concurrent
modification exceptions on live code paths. By removing the transactional load
and only touching that entity group on the delete, the number of potential
concurrent modification exceptions is significantly reduced.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135706974
2016-10-11 11:27:49 -04:00
mcilwain
b65b855067 Always use the constructor to make Immutable Collection Builders
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135359669
2016-10-07 14:37:02 -04:00
mcilwain
b6fbe1f3df Consolidate expand billing events action into 'batch' package
Now it lives alongside the delete prober data action, as well as any future
batch/maintenance tasks that should run periodically.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134435668
2016-10-03 15:49:35 -04:00