Commit graph

29 commits

Author SHA1 Message Date
guyben
8beb10c2a3 Update sandbox / alpha cron.xml to be in line with production
There are 2 types of changed done here:
- reorder the existing cron jobs to be in the same order as production (for
  easier diffing)
- add missing cron-jobs to either alpha or sandbox

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=183232936
2018-02-01 21:57:39 -05:00
larryruili
fbdb148540 Add billing pipeline to cron
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181793243
2018-01-19 14:38:38 -05:00
mcilwain
46aa638b74 Rationalize prod/sandbox instance numbers to 50/5/50
That's 50 each for frontend and backend and 5 for tools. Since the
MetricExporter bug has been fixed for awhile now, we aren't gaining anything by
artificially keeping the instance number low, whereas we might benefit from
higher instance counts, e.g. for load-testing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179432038
2017-12-27 11:13:42 -05:00
larryruili
8dcc2d6833 Chain ICANN report upload after staging
This converts the upload task from a cron job to a task chained after staging.
This ensures the upload job only occurs when its dependencies are met, and
provides a faster turnaround time to verify both the staging and upload jobs
are complete.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175045489
2017-11-21 18:16:08 -05:00
larryruili
486c348a00 Add reporting cron jobs to production
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173569808
2017-11-07 17:25:46 -05:00
mcilwain
c24d5b8a88 Increase the frontend service idle timeout from 10 to 30 minutes
This should help reduce the occurrence of requests taking a long time
to process because a new instance is being spun up. We might consider
increasing this further to 60 minutes in the future if necessary.

This also increases the number of frontend instances on production to 8
from 6, since it appears like the issue we were attempting to mitigate
with that change is now fixed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173440059
2017-11-07 17:07:10 -05:00
mcilwain
62dcf2f1a7 Temporarily tune down production frontend instances to 6
We'll revert this once the stuck instance issue in Java 8 is fixed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173183426
2017-10-24 16:53:47 -04:00
mcilwain
c69b409b30 Add java8 runtime option to production appengine-web.xml files
It occurs to me that we can't have this setting different between sandbox
and production, otherwise we can end up with a situation where we push code
that works on sandbox but then fails only when it is pushed to production.
Sandbox and production need to always be set up similarly for this reason.

We'll just have to pay a greater amount of attention to the release process
next week than normal, and continue playing around in alpha for the mean
time with a fully Java 8 build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170197703
2017-10-04 16:16:45 -04:00
mcilwain
7dc1940cdb Move ResaveAllEppResources mapreduce from tools service to backend
It makes sense for all mapreduces to run in backend, especially onces
that are scheduled regularly to run in cron like this one now. We don't
have many instances configured for the tools service anymore on some
of our environments, so backend is the friendliest place for a mapreduce
to run.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882122
2017-09-20 10:27:17 -04:00
guyben
d7214b58fc Re-enable DeleteOldCommitLogs cron job
Also adds a "resave all epp" cron job that's needed for the delete to work correctly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168879965
2017-09-20 10:27:17 -04:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
mcilwain
fb10005cba Remove unneeded WEB-INF/*.yaml resource paths
The YAML configuration files are now being built directly into the
JAR, and not stored in the WEB-INF/ directory, so this is unnecessary.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146815937
2017-02-08 17:42:49 -05:00
mcilwain
bf068e61d9 Move the environment configuration YAML files into the main JAR
This allows configuration to work properly from the nomulus tool.

TESTED=I built and ran it against several environments, and all worked
properly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146697124
2017-02-06 16:46:52 -05:00
mcilwain
8830224cc5 Add MOE equivalence for sync on 2017-02-02
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146422375
2017-02-06 16:45:23 -05:00
mcilwain
16f51f0a5c Add initial implementation of YAML config file
This implements the basic framework that allows global YAML
configuration, per-environment custom configuration, and unit-
test-specific configuration.

TESTED=I deployed to alpha, ran some EPP commands through the
nomulus tool, and verified no errors.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145422680
2017-01-30 15:03:53 -05:00
mcilwain
6c11ac5392 Move more configuration options into YAML config files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145452311
2017-01-25 12:25:54 -05:00
shikhman
d3fe6be385 Remove ECatcher from GAE code
It's superceded by Stackdriver Trace.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145440930
2017-01-25 12:25:41 -05:00
mcilwain
b5cf58bf2c Add initial implementation of YAML config file
This implements the basic framework that allows global YAML
configuration, per-environment custom configuration, and unit-
test-specific configuration.

TESTED=I deployed to alpha, ran some EPP commands through the
nomulus tool, and verified no errors.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145422680
2017-01-25 12:25:05 -05:00
ctingue
f30b87d40e Add ExpandRecurringBillingEventsAction cron job
Note that this merely starts this MR on a daily schedule -- the billing queries that ultimately consume the synthetic OneTime events are filtering out the events at this time, so we're still relying on query-time expansion of Recurrings.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144450565
2017-01-18 11:03:32 -05:00
mountford
890354938c Switch TMCH SMDRL fetch job to start at 00:15 and 12:15
The job was starting at midnight and noon, which is exactly when the files are changing. This resulted in intermittent failures, as the files are temporarily missing during the changeover.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139081163
2016-11-15 11:02:42 -05:00
mountford
994747a8e5 Enable built-in session cleanup in production
[] enabled the built-in App Engine session cleanup servlet in alpha and sandbox, and it appears to be deleting expired sessions at the expected rate of 100 every 15 minutes. So enable it for production as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138071390
2016-11-10 11:13:52 -05:00
mcilwain
6636e02d57 Clean up and consolidate some unused queues
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136170276
2016-10-14 17:37:21 -04:00
mcilwain
e192c11adf Add a batched [] for DNS refreshing on host renames
This will replace the existing DnsRefreshForHostRenameAction.

This is stage one of a three stage migration process. It adds the new queue and
[] but doesn't call them yet. Stage two will cut over to using the new
functionality, and stage three will remove the old functionality.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134793963
2016-10-03 16:11:00 -04:00
mcilwain
7f0cb4eae5 Move prober deletion action to backend module
Also creates a new package named 'batch' to house it.

TESTED=I deployed it to alpha, sent a POST request to the task URL, and it
successfully ran the [].

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134332999
2016-10-03 15:41:38 -04:00
mcilwain
00ea99960a Improve efficiency of async contact and host deletion with batching
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1.  An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once.  This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
2016-09-19 11:47:55 -04:00
mcilwain
c06133435b Consolidate some cron groups and use fanout action for more retrying
This is the first step in consolidating our task queues down into a smaller
number.  We have lots of tasks that run quite infrequently, and they can all run
in the same queue to get retry semantics without needing a new queue for each
cron entry.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131990472
2016-09-02 13:56:23 -04:00
Justine Tunney
7cc7dc4af2 Use zip_rule() for EAR/WAR deploy archives
These build rules allow Bazel to generate the .ear and .war files which
appcfg.sh (a tool that comes with the App Engine SDK) can then use to
perform a deployment.

Included in this CL are configurations for five separate production
environments: production, sandbox, alpha, crash, and local.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129163010
2016-08-02 21:00:39 -04:00
Michael Muller
c458c05801 Rename Java packages to use the .google TLD
The dark lord Gosling designed the Java package naming system so that
ownership flows from the DNS system. Since we own the domain name
registry.google, it seems only appropriate that we should use
google.registry as our package name.
2016-05-13 20:04:42 -04:00
Justine Tunney
5012893c1d mv com/google/domain/registry google/registry
This change renames directories in preparation for the great package
rename. The repository is now in a broken state because the code
itself hasn't been updated. However this should ensure that git
correctly preserves history for each file.
2016-05-13 18:55:08 -04:00