These shouldn't ever be null, but we have some bad data in production
for prober TLDs left over from the Registry 2.0 transition. Ignoring
null values here is required to finish cleanup for this old data, which
currently cannot even be deleted because it's throwing an NPE when
trying to update these values.
This commit will be reverted after the bad data is cleaned up, likely
sometime next week.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158546840
Now that we've been live with the new code, all of the old tasks
have been processed, and the fallback code to handle the situation
of those fields not being specified is not necessary.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158530336
It turns out that StackdriverWriter uses reflection on the class of
the DistributionFitter instance, so rather than giving it custom
handling for FibonacciFitters, it's easier to turn FibonacciFitter
into a single utility method that takes advantage of existing support
for CustomFitters.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158190147
PDT testing revealed a couple ways in which our WHOIS output was non-compliant. First, the Consistent Labeling & Display policy dictates that the contact IDs must be ROIDs. See rule 11 in https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en. Second, PDT tests expect that a WHOIS response will treat missing values either by omitting the line entirely, or by including the line with a blank value, but not both. So this is legal:
Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568
Fax Number Ext:
and this is legal:
Phone Number: 123-4567
Fax Number: 123-4568
but this is not:
Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568
In the last example, one extension line is present with a blank value, while the other extension line is omitted. We cannot do both. Therefore, we should update our code to omit lines with no value. Since we can't guarantee that we will always emit all lines that the parse might expect to see, it is safe to use the policy of omitting lines with no value.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158184150
Memcache is already off but now it's not in the code anymore.
This includes removing domain creation failfast, since that is actually
slower now than just running the flow - all you gain is a non-transactional
read over a transactional read, but the cost is that you always pay that
read, which is going to drive up latency.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158183506
Changed [] to use v1 instead of v1beta1, and replaced v1beta1 with v1 in all the java files.
If there is special build rules for open-source etc. that also need to be updated, or non "TAP-able" tests that need to be run, please check and see if they are OK.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157895888
The command was set up such that an update without any contact types specified would clear out the list, instead of leaving them unchanged, as it should.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157766429
[] opened up the settings on RdeStaging, in order to make it usable by the nomulus tool. But in retrospect, we think that all we needed to do was support the POST method, not loosen the auth settings, since nomulus invokes RdeStaging via a task queue. Removing the looser auth settings will bring this action into line with other backend actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157262595
This brings the affected actions into line with the settings on other similar actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157259842
Our build got broken by the deprecation of goog.structs.Map in [] This is a quick fix.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157236744
We want to know how long it's actually taking to process asynchronous
contact/host deletions and DNS refreshes on host renames. This adds
instrumentation. Five metrics are recorded as follows:
* An incrementable metric for each async task processed (split out by
type of task and result).
* Two event metrics for processing time between when a task is enqueued
and when it is processed -- tracked separately for contact/host
deletion and DNS refresh on host rename.
* Two event metrics for batch size every time the two mapreduces are
run (this is usually 0). Tracked separately for contact/host deletion
and DNS refresh on host rename.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157001310
A Fibonacci fitter is useful in situations where you want more precision on the
low end than an ExponentialFitter with exponent base 2 provides without the
hassle of dealing with non-integer boundaries, such as would be created by an
exponential fitter with a base of less than 2. Fibonacci fitters are ideal for
integer metrics that are bounded across a certain range, e.g. integers between 1
and 1,000.
This also cleans up some unit test comments.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156773367
Move the "restoreCommitLogs" command from the backend module to the tools
module so it's easier to access with nomulus.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156768389
The StatusProhibitsOperationException superclass constructor just wants a raw message to return to the client, so we should pass it an explanation rather than just the superordinate domain name. In fact, I think the superordinate domain name isn't really necessary since it should be obvious from the hostname in the create/update request what the superordinate domain would be.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156747185
It was setting the TLD and watermark date parameters as comma-separated lists. What it really needs to do is set the parameters multiple times, once for each value.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156741051
This was needed to correct bad data (LINKED status values on EppResources). The code has been fixed to no longer persist LINKED on any resources and I ran a resave all action yesterday to remove all persisted LINKED status values, so the migration @OnLoad can be safely removed now.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156580334
HttpServlet#destroy() is not called on instance shutdown, so the metrics code was never shut down correctly before.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156360866
WHOIS disclaimer is read from the config yaml file directly and the line breaks
within the text does not contain carriage return, which ICANN requires. This
CL fixes the non-compliance.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156359526
*** Reason for rollback ***
This change is crashing alpha
*** Original change description ***
Change metrics ThreadFactory to produce daemon threads
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156336467
This replaces the memcache caching, which we think is overall a bad idea.
We load all registrars at once instead of caching each as needed, so that
the loadAllCached() methods can be cached as well, and therefore will
always produce results consistent with loadByClientIdCached()'s view of the
registrar's values. All of our prod registrars together total 300k of data
right now, so this is hardly worth optimizing further, and in any case this
will likely reduce latency even further since most requests will be
served out of memory.
While I was in the Registrar file I standardized the error messages for incorrect
password and clientId length to be the same format, and cleaned up a few
random things I noticed in the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156151828
We will not want to run this under normal circumstances, but in cases (such as PDT testing in sandbox) where it's desirable to generate an escrow deposit even if it isn't technically valid XML, this gives us that option. Manual-mode RDE generation is also changed so that, if no watermark date is specified, it defaults to the previous midnight, to better support running of RDE in sandbox to catch data problems.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156108295
Now that we've updated the oauth java6 package in the open source build, we
can safely replace "Exception" with "IOException" in the throws clause of the
method that calls AuthorizationCodeInstalledApp.authorize().
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156057190
Upgrade to the latest versions of omit_com_google_oauth_client_java6 and
omit_com_google_oauth_client_jetty and fix the broken stuff in our local
mirroring procedures. Specifically:
- Document that users need to add a GCS url.
- Document how to run the check_bazel_deps script.
- Document that we don't use the in-piper repo.
- Remove the broken ibiblio link for the "truth" package.
- Delete sync_bazel_deps (which doesn't currently work, isn't python-style
compliant and is no longer relevant since we don't use the in-piper repo)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155898485
For the currency specific billing account ids to work properly, any REAL registrar that is allowed to register domains in a certain TLD must have a billing account id for the currency of that TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864711
This helps creating test domains. Also fixed a bunch of bad test cases.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864065
This tool finds all registrars that have a null name, or a null localizedAddress field, and populates them as follows:
- The registrar's name is updated to its clientId
- The registrar's empty localizedAddress fields are replaced with fake data
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155739946
Moving to KmsKeyring, any command that uses the Keyring must be a
RemoteApiCommand to work.
TESTED: successfully encrypted a file with no logged errors.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155673003
My continuing investigations into necessary resources for running
our environments seems to indicate that four instances should be
sufficient for our purposes. If it's not, we can always revert.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155607688
This should be %SOURCE_DATASET% so that it's adjusted appropriately based on the parameter value.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155582718
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
These annotations are part of guava's API, and they cannot safely be
optimized off the compile-time classpath.
This works around google/error-prone#615.
See []
H/T @cushon
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155433554
Frontend validation: ensures that only one WHOIS abuse contact exist per registrar. Any existing WHOIS abuse contact will be overridden when a new one is designated.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155289097
We've determined that getting correctness semantics right, even
in the few cases that it is possible to do so (see linked bug for
audit) is not worth the bother in terms of highly complicated code
and potential bugs. This CL turns off memcache at the Ofy level
but doesn't rip out the annotations etc. so that we can quickly
turn it back on if this turns out to have been a mistake.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155227761
If there are gaps or forks in the commit log history files, raise an exception
and display the possible sets of files in the log files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155219410
Also added corresponding getters and setters for the new field. Note that
nothing has changed on the RDAP front for now, as the CL&D only concerns WHOIS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155116134
