Commit graph

103 commits

Author SHA1 Message Date
mcilwain
4b99fae1dd Properly handle NAME_COLLISION domains in start-date sunrise
Domains that are reserved with type NAME_COLLISION can be registered defensively
during sunrise only, but DNS can never resolve for them. Correspondingly, we
need to apply the SERVER_HOLD status for such registrations. We also send the
registrar a poll message informing them of this act.

This brings us up to feature parity with end-date sunrise (implemented in
DomainAllocateFlow), which already has all of this handling.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205277728
2018-08-10 13:44:25 -04:00
mcilwain
abfc60ab39 Consolidate wildcarded domain_create_response testdata XMLs
This is a follow-up to [] It also removes a couple of the
doSuccessfulTest() overloads (having that many was overkill), and statically
imports a few constants for line length reasons.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204971899
2018-07-17 22:06:48 -04:00
mcilwain
e32b1a571a Consolidate wildcarded domain_create testdata XMLs
It's simpler to have fewer of these, and it's clearer from callsites if it's
always explicit exactly which domain is being created.

I also removed two irregularities: domain_create.xml included hostnames but
domain_create_wildcard.xml did not, and the former called the domain name
%DOMAIN% whereas the latter called it %HOSTNAME%.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204956433
2018-07-17 22:05:24 -04:00
mcilwain
df5d9957f7 Move the AllocationToken entity to the token subpackage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204485366
2018-07-14 01:37:03 -04:00
guyben
b4cb3e0a6f Inject clock for Tmch validity using InjectRule instead of manually
Basically a less patchy version of []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202138086
2018-06-27 15:28:53 -04:00
guyben
44c517f111 Set clock used to validate Tmch signature in tests
The signature has expired on Mon Jun 25 23:59:59 UTC 2018.

Instead of fixing all the test files with a new signed mark, we inject the
clock used to validate the signature.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202116721
2018-06-27 15:28:52 -04:00
mcilwain
ed910455b0 Add more absent clTrid unit tests
In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.

This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
2018-06-27 15:28:52 -04:00
mcilwain
ad73f3d167 Remove more unnecessary "throws" declarations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201243722
2018-06-27 15:28:52 -04:00
mcilwain
5d80f124ca Remove unnecessary "throws" declarations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201058582
2018-06-18 18:17:56 -04:00
larryruili
446617e4e5 Add proper flagging for start-date sunrise billing
This will allow us to check in actual SUNRISE billing policies per launch (15% discount), instead of relying on ad-hoc timestamps.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200077926
2018-06-18 17:59:07 -04:00
guyben
5aeee19699 Send out Lordn during start-date sunrise
Also prevents signed marks from being used in non-sunrise TldStates.

Currently, we send out a Lordn update only when there's a ClaimNotice, or if
we're in end-date sunrise.

But EPPs can contain a SignedMark instead of a ClaimsNotice for trademarked
domains - in which case we aren't sending out Lordn update. This also applies
to start-date sunrises.

We also change the SignedMark behavior for superusers. Currently, if a
mismatched signed mark is given as superuser, we accept it. That causes
problems when we want to send the Lordn update.

Instead - we no longer allow superusers to give a mismatched SignedMark (just
as we don't allow users to give a bad ClaimNotice). A super user can still
create a domain WITHOUT a signed mark - but if one is provided, it must match.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199783411
2018-06-18 17:45:44 -04:00
larryruili
1c71487e5d Set EAP billing event to one-year period
This is consistent with how we treat RESTORE billing events as well- in
general, fees are considered to be amortized over the course of a year (by the
invoicing team).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199684843
2018-06-18 17:42:29 -04:00
jianglai
e5538cfe35 Handle missing expected fee type in domain create
Also added a couple of more tests to make sure that we cover all edge cases.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=195872013
2018-05-17 21:52:35 -04:00
jianglai
82ded21b9e Make EAP fee description check more flexible
Currently we determine fee type from the fee extension description by checking if the format string of the FeeType contains the description we received. The formatting string for EAP is "Early Access Period, fee expires: %s", so the fee description generated by a domain check command, like "Early Access Period, fee expires: 2022-03-01T00:00:00.000Z", is not recognized as EAP.

This CL adds the ability to add arbitrary extra description strings to the FeeType for a description to match against. It also changes the match to "the given description contains any of the strings from the list of format string plus extra description strings".

For EAP, we added an extra description string "Early Access Period", so any fee description that contains "Early Access Period" will be matched to EAP FeeType, including the specific description (that contains the expiry time) that we send in a domain check.

Also improved error message on multiple fee type matching.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194149162
2018-05-05 23:26:33 -04:00
larryruili
f56355c9e8 Enforce anchor tenant domain create period
This verifies anchor tenant creates by registrars are for exactly 2 years, to simplify billing down the line (anchor tenants get 2 years of free domain creates).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193931858
2018-04-23 15:11:37 -04:00
mcilwain
2c0fb6d5a6 Enforce canonicalization of premium/reserved list labels
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193401336
2018-04-23 14:53:10 -04:00
guyben
b5ae37c5cc Return more informative errors when signed mark is invalid at this time
A "mark" tells us that the holder owns the trademark for a given domain name. It is signed for authentication.

If the signature's certificate is either "not yet valid" or "expired", we return explicit errors to that effect.

But in addition to the signature's certificate, the mark itself might not be valid yet or already expired. Right now if that happens - we return an error saying "the mark doesn't match the domain name".

That is wrong - as the mark can match the domain name, just be expired. Returning "the mark doesn't match the domain name" in that case is misleading.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190069976
2018-04-02 16:29:24 -04:00
guyben
edcb725a18 Default missing type= argument to "registration" in launch:create EPP extension
<launch:create> has an optional type argument, that can take either "application" or "registration":
https://tools.ietf.org/html/rfc8334#section-3.3.1

We get that type via createExtension.get().getCreateType(), where if the type= argument isn't given, the function returns null.

In that case, we need to decide based on the TLD - application for end-date sunrise, and registration for start-date sunrise.

For now we can't do that, because FlowPicker doesn't have access to the TLD information. Until that is fixed we decide as follows:

- landrush and sunrush phases will default to APPLICATION, because there's no possible
registration for it.

- sunrise defaults to REGISTRATION because we're currenly launching start-date sunrise that uses registration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189942568
2018-04-02 16:22:54 -04:00
mcilwain
ceed5bdd1c Make return value of ClaimsListShard.getClaimKey() Optional
It was nullable all along, but wasn't tagged as such, and thus it was
possible to misuse the method from its call sites.

Also adds an assertion about no NORDN tasks being enqueued in a failing
domain create test for a required signed mark.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187649865
2018-03-06 19:15:52 -05:00
guyben
24799b394d Allow the same LaunchPhase to be used for both start-date and end-date sunrise
Also changed the name of "verifyRegistryStateAllowsLaunchFlows" to "verifyRegistryStateAllowsApplicationFlows", because there are now launch flows that don't use applications (start-date sunrise).

Finally, added a test to showcase the "super-user" power that EPPs with Anchor Tenants have. There's no change in behavior in that regard in this CL - we just add a test to make it explicit.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187517199
2018-03-06 19:12:56 -05:00
mcilwain
ce136f9285 Add short-term host/contact caching for high-QPS domain mutations
After investigating common domain create/update command usage
patterns by registrars, we noticed that it is frequent for a
given registrar to reuse both hosts (using a standardized set of
nameservers) as well as contacts (e.g. for privacy/proxy
services). With these usage patterns, potential per-registrar
throughput during high volume scenarios (i.e. first moments of
General Availability) suffers from hitting hot keys in Datastore.

The solution, implemented in this CL, is to add short-term
in-memory caching for contacts and hosts, analogous to how we are
already caching Registry and Registrar entities.  These new
cached paths are only used inside domain flows to determine
existence and deleted/pending delete status of contacts and
hosts. This is a potential loss of transactional consistency, but
in practice it's hard to imagine this having negative effects, as
contacts or hosts that are in use cannot be deleted, and caching
would primarily affect widely used contacts and hosts.

Note that this caching can be turned on or off through a
configuration option, and by default would be off. We'd only want
it on when we really needed it, i.e. during a big launch.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187093378
2018-03-06 19:05:20 -05:00
cushon
606b470cd0 Merge JUnitBackport's expectThrows into assertThrows
More information: https://github.com/junit-team/junit5/issues/531

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187034408
2018-03-06 18:56:15 -05:00
jianglai
ff221fba96 Validate individual fee types
Currently we validate the fee extension by summing up all fees present in the extension and comparing it against the total fee to be charged. While this works in most cases, we'd like the ability to individually validate each fee. This is especially useful during EAP when two fees are charged, a regular "create" fee that would also be amount we charge during renewal, and a one time "EAP" fee.

Because we can only distinguish fees by their descriptions, we try to match the description to the format string of the fee type enums. We also only require individual fee matches when we are charging more than one type of fees, which makes the change compatible with most existing use cases where only one fees is charged and the description field is ignored in the extension.

We expect the workflow to be that a registrar sends a domain check, and we reply with exactly what fees we are expecting, and then it will use the descriptions in the response to send us a domain create with the correct fees.

Note that we aggregate fees within the same FeeType together. Normally there will only be one fee per type, but in case of custom logic there could be more than one fee for the same type. There is no way to distinguish them as they both use the same description. So it is simpler to just aggregate them.

This CL also includes some reformatting that conforms to google-java-format output.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=186530316
2018-03-06 18:48:39 -05:00
guyben
b0cbc0f60d Add the START_DATE_SUNRISE phase
The START_DATE_SUNRISE phase allows registration of domains only with a signed mark. In all other respects - it is identical to the GENERAL_AVAILABILITY phase.

Note that Anchor Tenants bypass all checks, and are hence able to register domains without a signed mark.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185534793
2018-02-20 15:45:27 -05:00
jianglai
eb82566785 Add test to confirm fee validation when creating/checking a premium domain during EAP
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185002798
2018-02-20 15:26:13 -05:00
mcilwain
98a61b8181 Add the ability to require premium fee acking for a registrar
When enabled for a registrar, all EPP operations on premium domains that have
costs (e.g.  creates, renews, transfers) will fail unless the EPP fee extension
is used to explicitly ack the amount of fee as part of the EPP transaction.

This ack is required regardless of whether premium fee acking is required at
the registry level. No data migration is necessary since false is the desired
default for this new attribute.

This CL also contains some slight refactoring of static utility methods used to
perform fee verification; there was short-circuiting at call-sites in two
places when what was really needed was two methods, one implementing additional
functionality on top of the other, and calling the inner method in the places
where short-circuiting had previously been necessary.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184229363
2018-02-01 22:07:11 -05:00
mcilwain
9d532cb507 Add extensibility framework for allocation tokens
This uses an extensibility mechanism similar to that of WhoisCommandFactory
and CustomLogicFactory, namely, that a fully qualified Java class is
specified in the YAML file for each environment with the allocation token
custom logic to be used.  By default, this points to a no-op base class
that does nothing.  Users that wish to add their own allocation token
custom logic can simply create a new class that extends
AllocationTokenCustomLogic and then configure it in their .yaml config
files.

This also renames the existing *FlowCustomLogic *Flow instance variables
from customLogic to flowCustomLogic, to avoid the potential confusion with
the new AllocationTokenCustomLogic class that also now exists.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=183003112
2018-02-01 21:54:29 -05:00
mcilwain
c416b3892d Migrate final try/catch test assertions to use assert/expectThrows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182091814
2018-01-19 14:43:12 -05:00
mcilwain
e07d011bc6 Add basic AllocationToken validation/redemption for domain creates
The next step is to add them for domain checks as well (which is simpler
because it doesn't involve validation).

This requires the addition of a TrimWhitespaceAdapter for XML JAXB objects,
which will prove useful for other @XmlValue attributes in the future.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181526726
2018-01-19 14:28:26 -05:00
mcilwain
5f62947691 Add and wire up the AllocationTokenExtension XML class
The extension itself isn't used for anything yet; allocation tokens can be
passed for domain creates and checks but are ignored if present. This will
be changed in a subsequent CL that adds AllocationToken entities and related
logic. Usage of this extension in any other EPP flow will throw an
UnsupportedExtensionException.

The relevant spec is https://tools.ietf.org/html/draft-gould-allocation-token-04

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181343188
2018-01-19 14:23:59 -05:00
mcilwain
d538dca2e0 Re-add EppException marshalling assertions in domain flow tests
Apologies for the reformatting, but this refactoring is quite rote and it's
definitely a bigger use of total time to perform the reformatting individually
than to simply do it file-wide.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179238852
2017-12-27 11:07:18 -05:00
mcilwain
fbe11ff33c Use method references instead of lambdas when possible
The assertThrows/expectThrows refactoring script does not use method
references, apparently.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179089048
2017-12-27 10:55:03 -05:00
mcilwain
9157930983 Automatically refactor more exception testing to use new JUnit rules
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179072309
2017-12-27 10:50:20 -05:00
mcilwain
52ce49a02c Manually migrate some ExceptionRules to Junit 4.13 style asserts
These testing helper functions can't be handled by the automatic refactoring
tool because they're taking in expected exception details as parameters.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178832406
2017-12-13 12:43:45 -05:00
mcilwain
b825a2b5a8 Get rid of custom ExceptionRule methods
The only remaining methods on ExceptionRule after this are methods that
also exist on ExpectedException, which will allow us to, in the next CL,
swap out the one for the other and then run the automated refactoring to
turn it all into assertThrows/expectThrows.

Note that there were some assertions about root causes that couldn't
easily be turned into ExpectedException invocations, so I simply
converted them directly to usages of assertThrows/expectThrows.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178623431
2017-12-13 12:43:45 -05:00
guyben
b8970bfe1b Rename all testdata loading files to conform to a single naming scheme
The scheme is:
- loadBytes: returns a ByteSource of the data
- loadFile: returns a string using UTF8 encoding, optionally applying
  substitutions

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177606406
2017-12-01 22:14:06 -05:00
mcilwain
ebfa27b3ad Require fee extension when registering domain in EAP
Failing to use the fee extension during EAP can result in charges to registrars
that are radically different than what they may have been expecting.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177597883
2017-12-01 22:14:06 -05:00
mcilwain
2aa897e698 Remove unnecessary generic type arguments
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175155365
2017-11-21 18:17:31 -05:00
kak
6f9b039e72 Migrates users from the unitless, ambiguous plus(long) and minus(long) methods on various Joda-Time APIs to their Duration-accepting overloads.
​
This makes the units explicit, which prevents confusion and bugs.

More information: []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172455602
2017-10-24 16:53:47 -04:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
mcilwain
1c4e79f99e Don't allow non-active registrars to create domains or applications
Specifically, this prevents suspended registrars from creating domains or applications. Pending registrars already can't perform these actions because they get an error message when attempting to log in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170481338
2017-10-04 16:16:45 -04:00
larryruili
c86fd96654 Ignore test TLDs when logging transaction creates/deletes
The probers make a constant stream of create and delete calls, which we don't
want to account for when constructing transaction reports. This change will
cause only real TLDs to log create and delete transaction records.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166737801
2017-08-29 17:18:39 -04:00
larryruili
97581e519d Unnest transactionFieldAmount from DomainTransactionRecord
After working further with domain deletes, I realized we'll need to record multiple reportingTimes under a single historyEntry when issuing a -1 counter to cancel grace-period adds. Since the TLD would be the only shared component within a record, we'll just duplicate it across all records to save an unnecessary layer of hierarchy.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166261413
2017-08-29 17:05:15 -04:00
larryruili
c40dc67c5b Record transaction for domain adds, renews and allocates
This is the second of many cls adding explicit logging in all our domain
mutation flows to facilitate transaction reporting.

Adds and renews each result in a +1 counter for the NET_ADDS/RENEWS_#_YR field,
which I've added simple (# of years, add or renew) -> Enum functions to get.
Allocates are just a special case of adds, and are counted in a similar manner.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165963249
2017-08-29 16:56:19 -04:00
bbilbo
e786c8d6ff Add better testing of domain and host creation using multi-part TLDs
Added validation on domain creation, preventing a domain from being created if
it equals an existing TLD. Added domain create tests for domains using
multi-part TLDs that shared suffixes and prefixes. Added host create tests for
hosts using multi-part TLDs that shared suffixes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164297749
2017-08-29 15:47:50 -04:00
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
cgoldfeder
ae039aa0d8 Remove all vestiges of memcache
Memcache is already off but now it's not in the code anymore.

This includes removing domain creation failfast, since that is actually
slower now than just running the flow - all you gain is a non-transactional
read over a transactional read, but the cost is that you always pay that
read, which is going to drive up latency.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158183506
2017-06-14 10:28:24 -04:00
cgoldfeder
275d6ddc10 Disable memcache completely
We've determined that getting correctness semantics right, even
in the few cases that it is possible to do so (see linked bug for
audit) is not worth the bother in terms of highly complicated code
and potential bugs. This CL turns off memcache at the Ofy level
but doesn't rip out the annotations etc. so that we can quickly
turn it back on if this turns out to have been a mistake.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155227761
2017-05-17 11:35:19 -04:00
cgoldfeder
d7da112c19 Put back accidentally deleted assertion
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154300634
2017-05-03 10:35:44 -04:00
cgoldfeder
3f9501e469 Remove obsolete comment.
init() is from pre-flattening days

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154256803
2017-04-26 11:21:35 -04:00