Commit graph

12 commits

Author SHA1 Message Date
larryruili
552ab12314 Prepare billing pipeline for production
This makes a few cosmetic changes that prepares the pipeline for production.

Namely:
- Converts file names to include the input yearMonth, mostly mirroring the original invoicing pipeline.
- Factors out the yearMonth logic from the reporting module to the more common backend module. We will likely use the default yearMonth logic in other backend tasks (such as spec11 reporting).
- Adds the "withTemplateCompatability" flag to the Bigquery read, which allows multiple uses of the same template.
- Adds the 'billing' task queue, which retries up to 5 times every 3 minutes, which is about the rate we desire for checking if the pipeline is complete.
- Adds a shell 'invoicing upload' class, which tests the retry semantics we want for post-generation work (e-mailing the invoice to crr-tech, and publishing detail reports)

While this cl may look big, it's mostly just a refactor and setting up boilerplate needed to frame the upload logic.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179849586
2017-12-27 11:39:21 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
mcilwain
6636e02d57 Clean up and consolidate some unused queues
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136170276
2016-10-14 17:37:21 -04:00
mcilwain
94c549d960 Delete obsoleted non-batched DNS refresh on host rename action
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135681285
2016-10-11 11:27:48 -04:00
mcilwain
e192c11adf Add a batched [] for DNS refreshing on host renames
This will replace the existing DnsRefreshForHostRenameAction.

This is stage one of a three stage migration process. It adds the new queue and
[] but doesn't call them yet. Stage two will cut over to using the new
functionality, and stage three will remove the old functionality.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134793963
2016-10-03 16:11:00 -04:00
mcilwain
00ea99960a Improve efficiency of async contact and host deletion with batching
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1.  An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once.  This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
2016-09-19 11:47:55 -04:00
shikhman
cee08d48f2 Remove unused dns-cron queue
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884314
2016-09-14 15:44:28 -04:00
mcilwain
c06133435b Consolidate some cron groups and use fanout action for more retrying
This is the first step in consolidating our task queues down into a smaller
number.  We have lots of tasks that run quite infrequently, and they can all run
in the same queue to get retry semantics without needing a new queue for each
cron entry.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131990472
2016-09-02 13:56:23 -04:00
Ben McIlwain
38e7b07c93 Remove the "mail" task queue
I can't find any remaining uses of it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127942602
2016-07-21 14:54:57 -04:00
Ben McIlwain
a879afa075 Delete the "export" task queue
Near as I can tell, it's no longer used by anything.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127863557
2016-07-21 14:54:12 -04:00
Ben McIlwain
8f986d6f55 Get rid of deprecated rde-staging queue
It's no longer used by the current RDE system.  Also, remove an unused
Dagger @PRovides method for the rde-upload queue.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=122164826
2016-05-13 23:25:54 -04:00
Justine Tunney
5012893c1d mv com/google/domain/registry google/registry
This change renames directories in preparation for the great package
rename. The repository is now in a broken state because the code
itself hasn't been updated. However this should ensure that git
correctly preserves history for each file.
2016-05-13 18:55:08 -04:00
Renamed from java/com/google/domain/registry/env/common/default/WEB-INF/queue.xml (Browse further)