We never really used it and it'll be obsolete come Registry 3.0 anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213274520
Defines cron job in crash, sandbox and production environments.
Job already exists in alpha.
Job is not added to qa environment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212878436
This was only supposed to stay commented out until load-testing was complete.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=210087917
*** Reason for rollback ***
It's still having the same issues from b/79463634 in sandbox, so we don't want to deploy it to prod.
*** Original change description ***
Switch pubapi/default service to basic scaling in prod/sandbox
Also goes back up to 100 max instances.
Hopefully this'll work better this time.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206975159
Also goes back up to 100 max instances.
Hopefully this'll work better this time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204783809
It only needs the error page HTML files; everything else isn't used by endpoints
served by the backend service and only serves to increase build times
(especially compiling all that JS).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202229514
RdeUpload works on alpha and sandbox by sending the data to a google internal
server. It is still running (and succeeding) after each RdeStaging, but the
lack of the cron job means it is currently lagging behind.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202181148
Currently, we have two different ways to parse a "set" parameter:
key=value1&key=value2&key=value3...
and
keys=value1,value2,value3
This is error prone for several reasons:
- different parts of the code must be "synchronized" to use the same style (the
place that creates the request, and the place that parses the request)
- for the key=value1&key=value2, we often use the same key name for the single
value and the set value. This can result in subtle bugs where part of the
code will successfully read the key assuming there's only one key (and will
get the first key=value1, ignoring the rest)
Here we transition everything to the keys=value1,value2,value3 method. This one
was chosen because:
- it's shorter
- it's more intuitive for users
- the key name is plural, differentiating it from the singular key=value that
other requests might need
-----------------------------------
To make sure there are not "transition issues", we will continue to support
(with warnings) the key=value1&key=value2 parameter parsing until we're sure we
haven't forgotten to update any part of the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198810681
The migration plan is as follows:
1. This CL, which adds the new "pubapi" service that serves the check API, WHOIS, and RDAP.
2a. Update our public facing sites to switch over to use the new service.
2b. (either order) Rewrite the check API to remove dependencies on flows.
3. ... eventually, once the frontend service is no longer being hit by this traffic, remove its handling of these public endpoints.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197716346
100 is way overkill with manual scaling. 30 is most likely still overkill too,
but we want to tune incrementally rather than all at once. Note that at 30
instances we're expecting around 3 QPS per instance, which is still an order
of magnitude less than each instance can actually handle.
This also fixes the instance type on sandbox to be the same as on prod.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196875876
This should decrease the average wait time when running nomulus tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=195465469
This also removes RDE tasks that shouldn't/can't run on non-production environments, like upload/reporting.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192177779
This also reduces the interval of the commitLogCheckpoint cron job to once
every three minutes, as this job needs to load all commit log bucket entities.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191613858
TldFanoutAction fans out a given endpoint to all TLDs (either TEST, REAL, or
both).
However, it is also used to delegate a single endpoint request that we want set
in a specific queue (so we can control retries). We do that by setting the TLD
list to "runInEmpty" rather than "forEachRealTld" or "forEachTestTld".
Currently, using "runInEmpty" would still specify a TLD - but that TLD would be
the empty string. This is a bug: it sets the TLD parameter to a bad value. It
worked only because none of the endpoints called with "runInEmpty" were using
the TLD parameter.
However, this will (and does) break if either (a) the endpoint accepts an
optional TLD parameter (like deleteProberData does), or (b) the given endpoint
already has a TLD parameter in it (we want to run the endpoint with a single
TLD, but still use the "fanout" to set the right queue).
This CL fixes several things:
- if runInEmpty is given, no TLD parameter is added
- 'runInEmpty' is now mutually exclusive with 'forEach*Tld' and 'excludes'
- we do some sanity checks and added logging
- removed the buggy and unused "':tld' in path is replaced by TLD"
- in the cron.xml, removed documentation for :tld and the broken :registrar
Note that none of the endpoints that were used with runInEmpty fanout had the TLD parameter prior to deleteProberData
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189954585
The higher the number the better for serious launches. These used to be 100
but had been detuned because instances weren't dying correctly when no longer
needed, thus contributing to higher costs than necessary. That problem was
fixed when we migrated to the Java 8 runtime, however, so there's no reason
not to use the higher number.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184742738
There are 2 types of changed done here:
- reorder the existing cron jobs to be in the same order as production (for
easier diffing)
- add missing cron-jobs to either alpha or sandbox
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=183232936
That's 50 each for frontend and backend and 5 for tools. Since the
MetricExporter bug has been fixed for awhile now, we aren't gaining anything by
artificially keeping the instance number low, whereas we might benefit from
higher instance counts, e.g. for load-testing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179432038
Loadtesting data is identified as "prober data" by this job (it removes
anything under ".test", not only prober data)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177309096
This mirrors production in hopes of triggering b/67508570 to test the fix.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175295742
This should help reduce the occurrence of requests taking a long time
to process because a new instance is being spun up. We might consider
increasing this further to 60 minutes in the future if necessary.
This also increases the number of frontend instances on production to 8
from 6, since it appears like the issue we were attempting to mitigate
with that change is now fixed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173440059
It makes sense for all mapreduces to run in backend, especially onces
that are scheduled regularly to run in cron like this one now. We don't
have many instances configured for the tools service anymore on some
of our environments, so backend is the friendliest place for a mapreduce
to run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882122
Also adds a "resave all epp" cron job that's needed for the delete to work correctly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168879965
My continuing investigations into necessary resources for running
our environments seems to indicate that four instances should be
sufficient for our purposes. If it's not, we can always revert.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155607688
We want to lower the maximum number of service instances as much as
possible without affecting service reliability so that we can make
stronger statements about what the maximum cost of running a typical
Nomulus environment might be. This first step likely won't affect the
frontend and tools modules in practice because they aren't typically
running even this many instances, but it will clamp down on the
number of backend instances (which should be fine; it just means the
mapreduces will take longer).
Alpha is tuned down the same as sandbox for consistency reasons.
This also standardizes on the B4 size (which has 512 MiB RAM) for
all instances. Most instances were already using this, and the
deviations from it were seemingly at random. Crucially, backend,
which is likely most sensitive to this because it uses the mapreduce
library, is already on the smaller memory size.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154537995
The YAML configuration files are now being built directly into the
JAR, and not stored in the WEB-INF/ directory, so this is unnecessary.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146815937
This allows configuration to work properly from the nomulus tool.
TESTED=I built and ran it against several environments, and all worked
properly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146697124
This implements the basic framework that allows global YAML
configuration, per-environment custom configuration, and unit-
test-specific configuration.
TESTED=I deployed to alpha, ran some EPP commands through the
nomulus tool, and verified no errors.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145422680
This implements the basic framework that allows global YAML
configuration, per-environment custom configuration, and unit-
test-specific configuration.
TESTED=I deployed to alpha, ran some EPP commands through the
nomulus tool, and verified no errors.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145422680
Note that this merely starts this MR on a daily schedule -- the billing queries that ultimately consume the synthetic OneTime events are filtering out the events at this time, so we're still relying on query-time expansion of Recurrings.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144450565
The job was starting at midnight and noon, which is exactly when the files are changing. This resulted in intermittent failures, as the files are temporarily missing during the changeover.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139081163
App Engine provides a servlet which deletes up to 100 expired _ah_SESSION entities from DataStore. This CL adds a cron job to call the servlet every 15 minutes in both alpha and sandbox. Assuming all goes well, we will turn it on in production.
I originally learned about this servlet here:
http://www.radomirml.com/blog/2011/03/26/cleaning-up-expired-sessions-from-app-engine-datastore/
But it appears that we do not need a servlet definition, just a cron entry.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137533532
This will replace the existing DnsRefreshForHostRenameAction.
This is stage one of a three stage migration process. It adds the new queue and
[] but doesn't call them yet. Stage two will cut over to using the new
functionality, and stage three will remove the old functionality.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134793963
Also creates a new package named 'batch' to house it.
TESTED=I deployed it to alpha, sent a POST request to the task URL, and it
successfully ran the [].
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134332999
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1. An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once. This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
This is the first step in consolidating our task queues down into a smaller
number. We have lots of tasks that run quite infrequently, and they can all run
in the same queue to get retry semantics without needing a new queue for each
cron entry.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131990472
These build rules allow Bazel to generate the .ear and .war files which
appcfg.sh (a tool that comes with the App Engine SDK) can then use to
perform a deployment.
Included in this CL are configurations for five separate production
environments: production, sandbox, alpha, crash, and local.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129163010
