Replace KeystoreKeyring with ComparatorKeyring between KeystoreKeyring and
KmsKeystore. In the opensource version, will replace DummyKeyring with
KmsKeyring directly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152893767
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
For TLDs with domain create restriction. SERVER_TRANSFER_PROHIBITED and SERVER_UPDATE_PROHIBITED status codes
are automatically applied to newly created domains to make them immutable. When there is a legitimate for an update on a domain, the registry must first run nomulus update_server_locks to remove status before the registrar can request an update via EPP.
To eliminate the risk of the registry forgetting to reapply the codes after a update, we automatically re-apply these codes after a success update.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152533379
RdeStagingAction always processed all RDE and BRDA deposits currently outstanding, updating the cursors appropriately and kicking off the upload job. Sometimes we don't want all that. We just want to create a specific deposit by hand, without modifying the cursors or uploading. This CL adds parameters to support that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152415959
This is better than calling assertTldExists() inside a for loop because you can throw a single exception reporting all bad TLDs at once rather than only getting as far as the first failure. And then it's also a one-liner instead of 3 lines.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152412876
The code to authenticate and authorize incoming requests (including via OAuth) has been in the system. This CL actually turns it on, since we are satisfied from logging information that it is not unjustly denying access.
Auth settings are also updated on a few commands missed earlier.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152381820
This fixes recording of number of attempts and command name on EPP
flows, which was broken because a separate metric builder was
being injected in two places, EppController and FlowRunner, with the
one injected into FlowRunner being discarded rather than having changes
applied to the same instance as in EppController.
This also adds a test that the metric is created successfully inside
a flow. Note that tests already exist for EppController to ensure that
the metric is recorded correctly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152306596
RegistrarContacts are allowed to have no "types" specified (they can also have multiple types). However, the registrar console displays the contacts for the logged-in registrar grouped by type into sections, and contacts with no type were simply being omitted from the listing, which was confusing because you can't even log into the console unless your email is listed as a contact, and yet you might then visit the contact settings page and see (apparently) no configured contacts.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152302159
When a TLD is domain create restricted, every domain that is created under it will have both SERVER_TRANSFER_PROHIBITED and SERVER_UPDATE_PROHIBITED status applied on it. This way after a domain is created no registrar can change any settings on it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152266535
TldFanoutAction now returns an HTTP response detailing the actions it's taking.
The format is as follows:
OK: Launched the following 3 tasks in queue the-queue
- Task: task-a6ad250b-a9d8-427d-bbf7-eb736e6f4dcb, tld: com, endpoint: /the/servlet/com
- Task: task-cf6c4bb4-0542-411e-ae4d-723beec09e9c, tld: net, endpoint: /the/servlet/net
- Task: task-57899661-fc3f-4049-a265-d6604051406e, tld: org, endpoint: /the/servlet/org
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152264954
Since domain create restriction only applies to closed TLDs, flows like domain application create and domain application update does not apply, as the TLD never goes through sunrise period. Removing checks for domain create restrictions in these flows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152260673
We now attempt to retry Whois queries in the event of a short-lived error. Currently, we consider 'DatastoreTimeoutException' and 'DatastoreFailureException' as transient.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152044934
As part of b/36599833, this makes FlowRunner log the appropriate ICANN activity
report field name for each flow it runs as part of a structured JSON log
statement which can be parsed to generate ICANN activity reports (under the key
"icannActivityReportField").
In order to support this, we introduce an annotation for Flow classes called
@ReportingSpec and a corresponding enum of values for this annotation, which is
IcannReportingTypes.ActivityReportField, that stores the mapping of constant
enum values to field names.
The mapping from flows to fields is fairly obvious, with three exceptions:
- Application flows are all accounted under domains, since applications are
technically just deferred domain creates within the EPP protocol
- ClaimsCheckFlow is counted as a domain check
- DomainAllocateFlow is counted as a domain create
In addition, I've added tests to all the corresponding flows that we are
indeed logging what we expect.
We'll also need to log the TLD for this to be useful, but I'm doing that in a
follow-up CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151283411
This is a follow-up to Lai's refactoring of the get reservation types
code to return a set rather than a single type. Since we're always
returning a set now, the more natural way to represent a label that is
not reserved is to return an empty set rather than a set containing
UNRESERVED.
Also fixes some minor style issues I ran across regarding static
importing and test method naming that I ran across (no logic
implications).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151132116
It was previously only using the name of the inner command XML element,
e.g. "Create", "Delete", "Update", etc. This wasn't very useful because
there was no way to discriminate between operations on different types
of EPP resources.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151131491
A CurrencyUnit-to-BillingAccountEntry map is persisted in the Registrar entity. It provides flexibility for billing systems that assign different account ids for accounts under different currencies of the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151022753
Currently, the message you get when using e.g. assertAboutContacts().that(alice).isEqualTo(bob) is not very helpful, because we override the "actual" toString (for alice) to return just an abbreviated version, and dump the entire full ImmutableObject toString for bob:
java.lang.AssertionError: Not true that <ContactResource with foreign key 'contact_id'> is equal to ContactResource (@2125903542): {
authInfo=ContactAuthInfo (@2128169374): ...
// giant blob of contact info here
}
So you can't even figure out why they aren't equal. Rather than reverting the "actual" object's toString representation in this case, which would just require you to visually compare two giant blobs of properties, this special-cases to let you compare using the prettyPrintEntityDeepDiff() helper we use in MutatingCommand.
With the new version, you see exactly what differs:
java.lang.AssertionError: Not true that <ContactResource with foreign key 'contact_id'> is equal to ContactResource (@2125903542): {
authInfo=ContactAuthInfo (@2128169374): ...
// giant blob of contact info here
}
It differs as follows:
transferData.pendingTransferExpirationTime -> [2017-03-27T16:01:39.882Z, 2020-05-08T16:01:39.882Z]
lastTransferTime -> [2017-03-27T16:01:39.882Z, 2020-05-08T16:01:39.882Z]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151004823
Also adds the "test" prefix to all other tests so that our presubmit
rules can prevent similar future errors in this file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150875528
When updating domains, make sure that if the domains are nameserver restricted, the updated nameservers set on the domains are still consistent with the restriction.
When updating domains of a domain created restricted TLD, validate if the domain is still on the reserved list with nameserver restricted reservation. If it is not, there's likely some conflicting states of the domain that needs to be reconciled (e. g.the domain is removed from the reserved list after being created). Throws an exception in this case.
Also added missing tests for TLDs with nameserver whitelist.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150781935
During domain create/applicationcreate/allocate, domains that are on the reserved list(s) with nameserver restricted reservation type must set nameservers that are part of the allowed nameservers for that domain in the reserved list(s) applied to that TLD.
Additionally a boolean is added to Registry to indicate if a TLD is restricting domain create. If it is, only domains that are nameserver restricted can be registered.
For consistency with a similar feature that validates a TLD-wide nameserver whitelist, the per-domain nameserver validation is performed even when the operation is in super-user mode. Similarly, if a domain is nameserver restricted, nameservers must be supplied (i. e. the nameservers set cannot be empty) when registering the domain.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150641269
This is the third step of migrating to the new format: removing support
for the legacy format.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150512556
Now that transfers are always restricted to 1 year, it's unnecessary to store
extendedRegistrationYears on TransferData - it will always be equal to 1. This
simplifies logic in a few other places, e.g. RdeDomainImportAction.
I verified in BigQuery that no DomainBases exist with extendedRegistrationYears
values that aren't either null or equal to 1. At some point we should remove
the persisted fields from datastore via e.g. resaving all those domains, but
it's low priority and can wait until we have some more pressing migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150373897
This CL restricts domain transfer pricing lookups (on domain check and info) to
only support a 1-year period for inquiring about transfer fees. That treatment
matches what we do for domain restores, which are also always one year. This is
a followup to [] which disallowed actual transfer request flows from
specifying multi-year periods.
Since it's no longer necessary, this CL also changes the domain transfer pricing
logic to drop the years parameter, including removing the parameter from the
custom pricing logic TransferPriceParameters object.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150367839
EPP host spec section 3.2.5 requires that attempts to update hosts that are linked to different registrars must fail with error code 2305. This is complicated to do, as linked status is eventually consistent, and even more painful when checking links to those of different registrars.
This change forbids external-to-anything renames entirely.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150336754
This tweaks the logic that prohibits domain renews during pending transfers to just use the regular verifyNoDisallowedStatuses() check instead of a special check on TransferData with a custom exception. This is simpler and produces a better error message: we get "Operation disallowed by status: pendingTransfer" instead of "Object with given ID (foo.com) already has a pending transfer" (which is intended for use when denying a transfer request for an object already being transferred, not for this case).
For the record, we originally prohibited renews for domains in pending transfer because there's no good reason to do such a renew: b/12533793. But in fact our transfer server-approve logic relies heavily on this behavior, because otherwise the domain's expiration time computed in cloneProjectedAtTime() will reflect the transfer year added to the post-renew expiration time, whereas all the transfer server approve entities (e.g. new autorenew billing event) will reflect the pre-renew expiration time at the moment the transfer was requested. As such, it would be quite difficult to ever support a renew during pending transfer, since it would need to change many fields within the transfer server approve entities.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150325501
This changes ResourceStatusProhibitsOperationException so that we print out the list of StatusValues using their XML names rather than the literal enum name, i.e. we use "pendingDelete" rather than "PENDING_DELETE".
This seems more correct given that EPP clients will be used to seeing the status values in the XML representation, and it also matches the existing ResourceHasClientUpdateProhibitedException that hardcodes "clientUpdateProhibited":
http://[]/third_party/java_src/gtld/java/google/registry/flows/exceptions/ResourceHasClientUpdateProhibitedException.java?l=22&rcl=146111211
Also reorganized related test methods and added some missing tests, including for ContactTransferRequestFlow which previously had none. I also renamed the "clientProhibitedStatusValue" tests to instead say "statusValueNotClientSettable" to be clearer about what's being tested, and that it's not related to the "clientXXProhibited" statuses.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150248562
It turns out that this ICANN policy appears to prohibit transfers with
registration extensions other than 1 year (section A.8):
https://www.icann.org/resources/pages/policy-2012-03-07-en
This is backed up by the practical fact that we've never seen a registrar
request a transfer for any period other than one year.
And removing the support for multi-year transfers vastly simplifies
transfer logic and eliminates a bunch of annoying corner cases. Users
still can achieve the same thing by doing a 1-year transfer plus a
manual renewal afterwards for the remainder of the desired extension.
This change leaves in place lots of infrastructure to support multi-year
transfers that is now obsolete (e.g. TransferData.extendedRegistrationYears).
This should all be cleaned up, but it's a lower priority than fixing the
gap itself and insulating ourselves against needing to handle any real
multi-year transfer case. Once this CL goes in, we can start ignoring
extendedRegistrationYears entirely because it'll always be 1 year, which
makes the cleanup process easier.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150212864
There will now be "resultToString" and "resultEquals" to stringify / compare
the method results.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150135475
A new field (allowedNameservers) is added to ReservedListEntry that stores the allow nameservers for the label. The field itself is a comma separated string, but the actual lines within a reserved list file (from which the field is parsed) uses colon to separate nameservers, to avoid conflicting with the commas used as primary separators in a CSV file.
Combined with upcoming update(s) that enables locking down an entire TLD to only delegate domains with a nameserver restricted reservation type, this change will enable us to restrict domain delegation to nameservers specifically specified in the allowed nameservers list, in order to prevent malicious delegation in case the registrar for a brand TLD is compromised.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149989330
This CL defines metrics for both premium and reserved lists, but actually uses only the reserved list metrics. The premium list metrics will be used in a future CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149982639
A fake Keyring implementation is needed to test Keyring related tools now, so
I've refactored the fake RdeKeyring to support all Keyring API methods and
moved it to the common testing package.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149977968
This is the second step of migrating to our new XSRF token format. The
first step ([] made validate() start accepting new tokens
(basically, dual-read). This step cuts over our "writing" to write the
new token format. The third and final step will drop support for
validating the old token format (back to single-read). We'll do that
in a subsequent push so that we don't invalidate all the current XSRF
tokens that people might have in their browsers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149790648
Instead of only returning the most severe one, return all applicable ones. This is because the reserved list has grown to a list of types that are not strictly comparable but orthogonal to each other. We can no longer depend on the fact that the most severe type incorporates all properties of those beneath it. Therefore returning all of them and treat them one by one in the calling site is the correct behavior.
Due to constraint imposed in eppcom.xsd, during domain checks the response can only contain a reservation reason of fewer than 32 characters, therefore we are returning the message for the type with highest severity, in case of multiple reservation types for a label.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149776106
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.
Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
A test has been added to RequestHandlerTest, making sure that, while we merely log errors for the time being, the correct dummy AuthResult is being created.
Most actions use the default settings, which have been changed to INTERNAL / APP / IGNORED. Actions with non-default settings are:
INTERNAL/NONE/PUBLIC (non-auth public endpoints)
CheckApiAction
WhoisHttpServer
Rdap*Action
INTERNAL,API/APP/ADMIN (things currently protected by web.xml)
EppTlsAction
EppToolAction
CreateGroupsAction
CreatePremiumListAction
DeleteEntityAction
List*sAction
UpdatePremiumListAction
VerifyOteAction
WhoisServer
INTERNAL,API,LEGACY/USER/PUBLIC (registrar console)
RegistrarPaymentAction
RegistrarPaymentSetupAction
RegistrarSettingsAction
EppConsoleAction
INTERNAL,API,LEGACY/NONE/PUBLIC (registrar console main page)
ConsoleUiAction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149761652
This is an alternative to the "manually constructed" version in []
This generator recieves two objects implementing some interface. It returns a new implementation that exactly duplicates the behavior (return values, exceptions etc.) of the first ("original") object, but also logs any differences (in either return value or thrown exception) from the second object.
Nothing in the second object's execution will affect the results - not even thrown exceptions - it's only used for logging any differences.
This is useful when switching backend architecture for the some interface (say... from keystore to datastore+KMS for the Keyring interface :)) as we can make sure everything was OK for some extended period of real data before doing the switch.
This is better than just comparing the outputs once because having a completely different backend out of our control can result in "glitches" that are rare but relevant: maybe the new backend has lower availability, or sometimes fails and gives wrong results, or... well, that's it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149706165
