Commit graph

439 commits

Author SHA1 Message Date
gbrodman
8af2afc616 Use a potential discount in the AllocationToken when determining domain create price
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245458027
2019-04-27 00:08:57 -04:00
gbrodman
29b9f04aaa Fix up a comment based on investigation into billing events
Please see the linked bug for more context--I believe this is correct based on what I've observed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245423464
2019-04-27 00:05:49 -04:00
gbrodman
6ee34a79b1 Store a reference to an allocation token in the OneTime billing event
We will need to be able to find all redemptions associated with a particular token and this will allow us to do that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292865
2019-04-26 23:57:56 -04:00
mcilwain
b46a6b6d55 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
gbrodman
74971144d2 Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
mcilwain
d7fbd93dca Remove authcode fallback for registering anchor tenants
We won't be using this for future launches, so get rid of it now.

Also reorganizes existing anchor tenant test cases and adds a new one for
creation of a TMDBed anchor tenant domain in sunrise.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239669950
2019-03-25 11:59:38 -04:00
mcilwain
d7306652eb Allow registrars to be completely DISABLED
Disabled registrar cannot perform any actions via EPP and cannot log in to the
registrar web console.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239606389
2019-03-21 15:03:33 -04:00
gbrodman
2a18e705a2 Add generic XML syntax testing to a flow test
Adding it to one test is sufficient because we use the same loading logic across all flows.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239506003
2019-03-21 14:58:31 -04:00
gbrodman
6e01d400cd Test UnknownCurrencyEppException in the flow tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239279335
2019-03-20 14:25:28 -04:00
weiminyu
c8197dd19c Remove unused soy template
Generated class DomainCheckFeeEppSoyInfo.class is not used anywhere.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238450318
2019-03-20 14:25:28 -04:00
shicong
bc3bdc7347 Upgrade the version of google/errorprone plug-in
This CL upgraded google/errorprone plug-in to 2.3.3 and resolved
some warnings detected from the plug-in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238047862
2019-03-20 14:25:28 -04:00
gbrodman
44e3668510 Display pricing tier even when a domain is reserved
Even though it's not directly related, I also fixed a typo in PREMIUM because it bugged me.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237796791
2019-03-20 14:25:28 -04:00
guyben
847795d58d Remove the web console EPP endpoint
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.

We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.

Also, this is a security risk, as it allowed to do "billable actions" (creating a new domain for example) with the only authentication being access to the registrar's G Suite account.

This bypassed the certificate, IP whitelist, and EPP password, which is bad.

PUBLIC:
Remove the web console EPP endpoint

This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.

We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236244195
2019-03-05 14:20:42 -05:00
jianglai
4418d45595 Do not require auth info in super user transfer
Super users can look up auth info in Datastore or BigQuery backup anyway.
Requiring it only adds friction when using the super user extension, without
any additional security benefit.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=235786090
2019-03-05 14:17:42 -05:00
gbrodman
7ddd8b55d2 Set domain create fee cost to be 0 for anchor tenants
We already properly remove EAP fees for anchor tenants.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=235529469
2019-03-05 14:11:36 -05:00
mcilwain
94a2681127 Send deletion poll messages when requested by superuser
Otherwise, registrars will never receive a notification through EPP that a
domain has been synchronously deleted by us.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=234172289
2019-02-18 17:34:36 -05:00
mcilwain
49ac4e3e69 Improve Datastore efficiency of duplicate contact messages
I should have caught this in the review, but [] is loading *ALL*
contacts individually from Datastore on every domain update. This will add a
large number of Datastore round trips and thus significantly reduce update
performance.

This CL changes the behavior to *ONLY* load contacts when there is a duplicate
(which is needed to determine the contact's display name to generate the error
message), and loads all of them in a single batch rather than individually.

This also makes some minor changes around domain getters returning empty sets
instead of null.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233128140
2019-02-11 11:22:34 -05:00
shicong
71d65ed73a Add detailed log when DuplicateContactForRoleException is thrown
This change also added a test to verify that EPP request to modify
both contacts and registrant at same time can be handled as expected.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232935690
2019-02-08 11:20:21 -05:00
gbrodman
93ff541006 Handle sunrise reservations properly during sunrise TLDs
We figure out the TLD state so that we properly check whether or not we can provision sunrise domains in that TLD. We also change the message slightly so that it's a bit more clear when we aren't in sunrise.

Note: it is deliberate that NAME_COLLISION reservations are provisionable in sunrise.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232742813
2019-02-06 18:06:21 -05:00
gbrodman
3cf26ff9b6 Fix various Error Prone errors that were found by the FOSS build
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
2019-01-28 16:05:09 -05:00
mcilwain
e2528875b2 Merge DomainResource into DomainBase
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.

It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.

This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
2019-01-28 15:57:10 -05:00
mcilwain
37aa1d1815 Always require acknowledgment of premium fees
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.

Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
2019-01-17 19:07:51 -05:00
guyben
a4f85c33c0 Add the App Engine service used in the Action definition
Our goal is to be able to address every Action by looking at the class itself, and to make it clearer at a glance what you need to access the Action's endpoint

Currently, we can know from the @Action annotation:
- the endpoint path
- the Method needed
- the authentication level needed

This CL adds the service where the Action is hosted, which also translates to the URL.

NOTE - currently we don't have any Action hosted on multiple services. I don't think we will ever need it (since they do the same thing no matter which service they are on, so why host it twice?), but if we do we'll have to update the code to allow it.

The next step after this is to make sure all the @Parameters are defined on the Action itself, and then we will be able to craft access to the endpoint programatically (or at least verify at run-time we crafted a correct URL)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229375735
2019-01-17 18:59:16 -05:00
mcilwain
170980db2f Statically import commonly used TldState enum values
Takes advantage of the fact that the default state of a TLD created in tests is GENERAL_AVAILABILITY.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228916164
2019-01-14 16:23:15 -05:00
mcilwain
580302898d Delete end-date sunrise, landrush, and sunrush phases
This also deletes the associated commands and domain application specific
entities.

We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.

Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
2019-01-10 16:23:35 -05:00
guyben
c74ffd7559 Fix @VisibleForTesting given the newly deployed enforcement
Generated code is now also covered by @VisibleForTesting, including Dagger @Inject

This CL is a cleanup of auto-generated code by ghm@ from the Error Prone team

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228748874
2019-01-10 16:23:35 -05:00
guyben
898448b8a0 Reverse dependency between /flows/ and /batch/
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.

However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.

This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/

Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/

It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
2019-01-10 16:23:35 -05:00
mcilwain
4491b7b909 Make loadByForeignKey() and related methods return Optional
This is safer and addresses a common source of confusion in the codebase because it's always explicit that the resource returned may not be present, whether because it's soft-deleted when projected to the given time or because it never existed in the first place.

In production code, the presence of the returned value is always checked. In test code, its presence is assumed using .get() where that is expected and convenient, as it not being present will throw an NPE that will cause the test to fail anyway.

Note that the roughly equivalent reloadResourceByForeignKey(), which is widely used in test code, is not having this same treatment applied to it. That is out of the scope of this CL, and has much smaller returns anyway because it's only used in tests (where the unexpected absence of a given resource would just cause the test to fail).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225424002
2018-12-14 11:24:49 -05:00
mcilwain
400994237c Require SSL certificate hash on login by default
Note that it's possible to set a config option to disable this functionality
on a per-environment basis (we're disabling it for sandbox), but in general
SSL certificate hashes should be required for increased security.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225053496
2018-12-12 13:22:34 -05:00
jianglai
0a44ef0dca Refactor EppXmlTransformer to be in the model/ package
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
2018-12-12 13:22:34 -05:00
mcilwain
f58211402a Add an unrenew_domain command to nomulus tool
This is used to reduce the expiration time of domain(s) by some number of years
(if enough length remains in the registration term to do so). This does not back
out the previously saved BillingEvent entities as they may have already been
sent out and invoiced, so any related refunds must be handled out of band.

In addition to reducing the registration expiration time on the domain itself,
this command writes out a new history entry, one-time poll message informing the
registrar of this change, auto-renew billing event and poll message, and
updates/ends the old auto-renew billing event and poll message.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224999285
2018-12-12 13:22:34 -05:00
shicong
5d2bb892f3 Update lastEppUpdateTime and lastEppUpdateClientId when necessary
EppResource's lastEppUpdateTime and lastEppUpdateClientId need
to be updated whenever the domain is updated, renewed, deleted or
transfered.

This commit applied the change to the following domain EPP commands:

 - Update (already implemented)
 - Renew
 - Delete
 - Restore
 - Transfer request
 - Transfer approve
 - Transfer reject
 - Transfer cancel

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223344758
2018-12-03 19:13:55 -05:00
mcilwain
11da64bee2 Deny non-active registrars access to non-free EPP commands
Registrars that are PENDING or SUSPENDED should not have access to
EPP commands that cost money, as in either case it's not likely we'd
actually be able to get payment from said registrar. For this reason
we already prevented access to the domain create flow for non-active
registrars. This commit extends that to other commands that cost
money, including renewals, restores, and transfer requests.

Note that implicit autorenews will still occur for suspended
registrars, as in our point-in-time data model there's no good way
to prevent them. So when a registrar is suspended for non-payment,
the game plan is to get all of their domains transferred out to a
registrar that will pay as soon as possible.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223173677
2018-12-03 19:10:45 -05:00
guyben
5f283ebd09 Use AuthenticatedRegistrarAccessor in EppConsoleAction
EppConsoleAction still "manually" checks access by going over the
RegistrarContacts. We need it to use AuthenticatedRegistrarAccessor just like
every other part of the registrar console.

We still need to remove the (now unneeded) login EPP sent by the console, but that's left for a followup CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222404208
2018-12-03 18:51:40 -05:00
jianglai
e39cc04110 Test AsyncFlowMetrics without mock
Instead of verifying interactions on the mocks, we instead assert on the real test subject directly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218209556
2018-10-22 19:10:01 -04:00
jianglai
b254269d2f Refactor LordnTask to LordnTaskUtils
Made it clear that it is a util class and moved some of the functions only used in NordnUploadAction (to NordnUploadAction). Also used Retrier to handle retries when leasing tasks.

These changes allow us to no longer use InjectRule in related unit tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217761117
2018-10-22 18:59:48 -04:00
jianglai
3daf62a486 Remove the use of InjectRule in IdnLabelValidatorTest
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217707263
2018-10-22 18:54:53 -04:00
mcilwain
f43125bb04 Add a RESERVED billing event flag for creates of reserved domains
This is an additional signal we can use for the billing/invoicing pipeline that
will be helfpul in targeting invoice rows that may need adjustment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217524250
2018-10-22 18:38:23 -04:00
weiminyu
9e02502fd4 Fix bugs exposed by testing with Gradle
The following issues are addressed:
- XML sanitizer should preserve input encoding. Gradle loses any that is not UTF-8. Bazel loses any that is not ASCII.
- Verify that XML sanitizer works with non-UTF8 encoding
- GpgSystemCommandRule breaks when $TMPDIR env variable is not set
- TestDataHelper throws exception when loading resources if resources are plain files on default file system as opposed to being in a jar file.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216537258
2018-10-17 11:44:41 -04:00
mcilwain
218c4517eb Stop exporting EPP flow metrics to BigQuery
These are simply too costly in their current form now that we are handling double-digit QPS, so at a minimum we'd want to refactor these for batched exports using a background thread (like how Stackdriver metrics work). However, upon further review, that work isn't worth doing if this BigQuery table isn't actually being used for anything, and it seems that we aren't using it anymore given that ICANN transaction reporting no longer requires it.

So the simplest thing to do is simply to get rid of this entirely, and just use a combination of Stackdriver metrics and App Engine logs. The eppMetrics BigQuery table is ~1.2 billion rows and takes up 223 GB, so that's not an insignificant GCP billings saving if we can delete it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215905466
2018-10-08 16:59:29 -04:00
mcilwain
1586813398 Bypass EAP fees for anchor tenants
Note that the check flow does not yet handle any kind of allocation token
handling at all. Step 2 will be to add allocation token handling there, so a
RESERVED_FOR_ANCHOR_TENANT or RESERVED_FOR_SPECIFIC_USE domain will show as
available instead of reserved if the right token is specified using the
extension. Then once that's done, we can use that information to adjust the
price accordingly as well.

Right now the behavior with a domain check is that reserved domains always show
as reserved, even if they're anchor tenants.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215599350
2018-10-08 16:59:29 -04:00
jianglai
8d675a4b8c Remove checking of SNI headers
This is only useful when we used the [] proxy because the GFE requires SNI during handshake in order to request the client certificate. The GCP proxy does not need this (it always requests the client certificate). We do not need to check for its existence.

Also removed the checking of internal headers for ssl cert hash used only by the [] proxy.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213059027
2018-09-14 21:34:44 -04:00
mcilwain
8ddbf88151 Remove the deprecated MISTAKEN_PREMIUM reservation type
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
2018-09-14 21:28:23 -04:00
mcilwain
1b3df82fb3 Allow creation of reserved domains using allocation tokens
Unlike anchor tenants, these domains can be registered for any number of years,
but only during GA, as third parties cannot register domains pre-GA except
through the anchor tenant program.

Since this is new functionality, unlike creation of anchor tenants, there is no
fallback provided to send codes through the domain authcode; they must be sent
using the allocation token extension.

And note that, like with anchor tenants, providing the domain-specific
allocation token overrides any other reserved types that might apply to that
domain.

No changes are necessary to the domain application create flow because of the
above restriction to GA.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212310701
2018-09-11 14:17:08 -04:00
mcilwain
7b87ba41c7 Delete all Limited Release Program (LRP) code
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.

Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
2018-08-20 14:28:16 -04:00
mcilwain
d2f849ac0f Add new reserved domain creation from allocation tokens mechanism
Note that this gets rid of anchor tenant codes in reserved lists (yay!), which
are no longer valid.  They have to come from allocation tokens now.

This removes support for LRP from domain application create flow (that's fine,
we never used it and I'm going to delete all of LRP later).  It also uses
allocation tokens from EPP authcodes as a fallback, for now, but that will be
removed later once we switch fully to the allocation token mechanism.

This doesn't yet allow registration of RESERVED_FOR_SPECIFIC_USE domains using
the allocation token extension; that will come in the next CL.  Ditto for
showing these reserved domains as available on domain checks when the allocation
token is specified.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209019617
2018-08-20 14:22:06 -04:00
weiminyu
81fce674d2 Sanitize EPP XML requests and responses
Masks user credentials (tags 'pw' and 'newPW') in EPP XML messages.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207953894
2018-08-10 13:46:48 -04:00
mcilwain
6e74ba0587 Wrap ModulesService in new AppEngineServiceUtils
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).

This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.

This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
2018-08-10 13:44:25 -04:00
mcilwain
4b99fae1dd Properly handle NAME_COLLISION domains in start-date sunrise
Domains that are reserved with type NAME_COLLISION can be registered defensively
during sunrise only, but DNS can never resolve for them. Correspondingly, we
need to apply the SERVER_HOLD status for such registrations. We also send the
registrar a poll message informing them of this act.

This brings us up to feature parity with end-date sunrise (implemented in
DomainAllocateFlow), which already has all of this handling.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205277728
2018-08-10 13:44:25 -04:00