Commit graph

407 commits

Author SHA1 Message Date
gbrodman
4895f455d8 Add additional tests for the AllocationToken status transitions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292932
2019-04-26 23:59:32 -04:00
gbrodman
6ee34a79b1 Store a reference to an allocation token in the OneTime billing event
We will need to be able to find all redemptions associated with a particular token and this will allow us to do that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292865
2019-04-26 23:57:56 -04:00
mcilwain
b46a6b6d55 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
jianglai
77eb7610e8 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
cfee7e7fd5 Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
mcilwain
d5b540dfe7 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
gbrodman
397f222521 Automated g4 rollback of changelist 240574585.
*** Reason for rollback ***

The inconsistent class loading is breaking the tests

*** Original change description ***

Validate provided email addresses when creating a Registrar

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241014945
2019-03-29 16:23:59 -04:00
gbrodman
308d5eb76b Validate provided email addresses when creating a Registrar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240574585
2019-03-29 16:14:22 -04:00
gbrodman
5d25290d42 Remove compareTo() from DelegationSignerData
The only reason why it existed was so that we could get tests to print information in a consistent order and there are other ways of doing that. By removing compareTo we can use the properties of the extended ImmutableObject properly and properly implement the RFC https://tools.ietf.org/html/rfc5910#page-18

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240170488
2019-03-29 15:56:15 -04:00
mcilwain
d7306652eb Allow registrars to be completely DISABLED
Disabled registrar cannot perform any actions via EPP and cannot log in to the
registrar web console.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239606389
2019-03-21 15:03:33 -04:00
gbrodman
4544aa1efe Enforce abuse WHOIS contact for REAL registrars when adding TLDs
We do not enforce this for non-REAL registrars or in any environment other than UNITTEST or PRODUCTION. This is similar but separate to [] since we can add allowed TLDs in either location.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239510275
2019-03-21 15:00:16 -04:00
shicong
bc3bdc7347 Upgrade the version of google/errorprone plug-in
This CL upgraded google/errorprone plug-in to 2.3.3 and resolved
some warnings detected from the plug-in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238047862
2019-03-20 14:25:28 -04:00
mcilwain
49ac4e3e69 Improve Datastore efficiency of duplicate contact messages
I should have caught this in the review, but [] is loading *ALL*
contacts individually from Datastore on every domain update. This will add a
large number of Datastore round trips and thus significantly reduce update
performance.

This CL changes the behavior to *ONLY* load contacts when there is a duplicate
(which is needed to determine the contact's display name to generate the error
message), and loads all of them in a single batch rather than individually.

This also makes some minor changes around domain getters returning empty sets
instead of null.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233128140
2019-02-11 11:22:34 -05:00
shicong
c2445c3797 Rename loadByForeignKeys to loadByForeignKeysCached
This is to remind the user that the function actually uses cache, and also
for naming consistency with EppResourceUtils.loadByForeignKeyCached().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232870846
2019-02-08 11:17:38 -05:00
gbrodman
93ff541006 Handle sunrise reservations properly during sunrise TLDs
We figure out the TLD state so that we properly check whether or not we can provision sunrise domains in that TLD. We also change the message slightly so that it's a bit more clear when we aren't in sunrise.

Note: it is deliberate that NAME_COLLISION reservations are provisionable in sunrise.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232742813
2019-02-06 18:06:21 -05:00
mcilwain
c6e58d3bff Fix some issues caught by IntelliJ static code analysis
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
2019-01-28 16:08:24 -05:00
gbrodman
3cf26ff9b6 Fix various Error Prone errors that were found by the FOSS build
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
2019-01-28 16:05:09 -05:00
mcilwain
e2528875b2 Merge DomainResource into DomainBase
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.

It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.

This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
2019-01-28 15:57:10 -05:00
mcilwain
37aa1d1815 Always require acknowledgment of premium fees
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.

Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
2019-01-17 19:07:51 -05:00
mcilwain
8ac8ecf8f6 Rationalize OT&E client ID validation rules
This makes the validation rules much simpler, thus placing less cognitive load on the users of the console and nomulus tool.  The changes are:

1. Don't allow hyphens. No real registrars use hyphens in their client IDs, and it's better to reserve these solely as the delimiter between the base client ID and the number representing the environment.
2. Allow the first character to be a number.  This has affected multiple real registrars, causing their OT&E and production client IDs to be different.  There's no reason for this restriction, as the only reason motivating it was that there are no TLDs that start with a number.  However, the OT&E TLDs are created only in sandbox and never have DNS syncing enabled, so this restriction is purposeless.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229187198
2019-01-14 16:28:12 -05:00
mcilwain
170980db2f Statically import commonly used TldState enum values
Takes advantage of the fact that the default state of a TLD created in tests is GENERAL_AVAILABILITY.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228916164
2019-01-14 16:23:15 -05:00
weiminyu
a80a44cd06 Define TestRule that manages cache use in tests
All current tests that use caches with custom data expiry values
now restore the default config when teardown. We need to prevent
new unsafe uses from being introduced.

Restoration code have also been added to a few other tests that modifies
static fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228888041
2019-01-11 11:50:33 -05:00
guyben
a4fca18657 Set short redemption/add/delete periods for all OT&E TLDs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228791942
2019-01-11 10:59:21 -05:00
mcilwain
580302898d Delete end-date sunrise, landrush, and sunrush phases
This also deletes the associated commands and domain application specific
entities.

We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.

Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
2019-01-10 16:23:35 -05:00
guyben
c74ffd7559 Fix @VisibleForTesting given the newly deployed enforcement
Generated code is now also covered by @VisibleForTesting, including Dagger @Inject

This CL is a cleanup of auto-generated code by ghm@ from the Error Prone team

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228748874
2019-01-10 16:23:35 -05:00
guyben
898448b8a0 Reverse dependency between /flows/ and /batch/
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.

However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.

This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/

Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/

It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
2019-01-10 16:23:35 -05:00
Shicong Huang
a79e254e49 Add an action to check the status of OT&E verification
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228333195
2019-01-08 16:46:41 -05:00
jianglai
2e7d71b238 Refactor most of OT&E verification code to exist in utils class
This does not change existing functionality but will allow us to use this common code in the yet-to-be-created web console action as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227159346
2019-01-02 11:58:36 -05:00
guyben
2777018d6a Add the ability to setup OT&E from the web console
We create a new endpoint with a simple form that will let admins (including
support) setup OT&E for registrars.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226570568
2019-01-02 11:56:59 -05:00
mcilwain
56b61ad5a2 Invalidate premium list cache on update
This will only affect the tools service, the primary use case being (1) I go to
create a domain through nomulus tool, realize it's premium, (2) update the
premium list to not include that domain, (3) kill the tools service instance to
wipe out the cached premium value, then (4) create the domain at standard. This
commit eliminates step 3.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226180160
2018-12-20 07:46:33 -05:00
guyben
9d6a7ef66a Create OT&E entities directly, instead of calling sub-commands
This is in preparation for having a web-console endpoint to create OTE.

In addition - we streamline the code:

- we remove support for different premium lists
- we remove support for different DNS writers - we never want a "real" DnsWriter for OTE
- we remove support of --eap_only, because we don't need it anymore
- We use a single password for all the Registrars

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225841694
2018-12-20 07:46:33 -05:00
mcilwain
4491b7b909 Make loadByForeignKey() and related methods return Optional
This is safer and addresses a common source of confusion in the codebase because it's always explicit that the resource returned may not be present, whether because it's soft-deleted when projected to the given time or because it never existed in the first place.

In production code, the presence of the returned value is always checked. In test code, its presence is assumed using .get() where that is expected and convenient, as it not being present will throw an NPE that will cause the test to fail anyway.

Note that the roughly equivalent reloadResourceByForeignKey(), which is widely used in test code, is not having this same treatment applied to it. That is out of the scope of this CL, and has much smaller returns anyway because it's only used in tests (where the unexpected absence of a given resource would just cause the test to fail).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225424002
2018-12-14 11:24:49 -05:00
mcilwain
015c854a92 Make loadByForeignKeyCached() return an Optional type
Next up (and a much larger commit) will be giving loadByForeignKey() the same
treatment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225182377
2018-12-12 13:22:34 -05:00
jianglai
0a44ef0dca Refactor EppXmlTransformer to be in the model/ package
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
2018-12-12 13:22:34 -05:00
guyben
4dad0a8a73 Fix checkState failing to trigger retrier's retry
We have a retrier wanting to call a function, retrying on
IllegalStateExceptions (thrown by checkState()).

However, one of the checkStates is called inside a Concurrent.transform, so
when the checkState fails, the resulting IllegalStateException is wrapped in an
UncheckedExecutionException and isn't caught by the retrier.

We unwrap the IllegalStateException to make sure it's caught.

Q: Why not just catch UncheckedExecution exception?
A: Because it might wrap a different exception which we don't want to retry on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224862573
2018-12-12 13:22:34 -05:00
shicong
5bc70cbc99 Update lastEppUpdateTime after certain grace period is passed
The lastEppUpdateTime should be updated asynchronously in the situations below:

 - Implicit transfer success after 5 day pending transfer period
 - Implicit end of any grace period

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224831791
2018-12-12 13:22:34 -05:00
mcilwain
b317aab22f Clean up some areas in preparation for externalized codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221454055
2018-11-16 16:51:11 -05:00
jianglai
589e98a1db Explicitly set the shard size when saving the claims list
This allows us to get rid of the use of InjectRule in tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218221409
2018-10-22 19:13:11 -04:00
mcilwain
f43125bb04 Add a RESERVED billing event flag for creates of reserved domains
This is an additional signal we can use for the billing/invoicing pipeline that
will be helfpul in targeting invoice rows that may need adjustment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217524250
2018-10-22 18:38:23 -04:00
mcilwain
9faf7181c3 Index the allocation token's redemption history entry field
We need this to efficiently query which tokens have and have not been redeemed.

I'm also marking it as nullable for consistency with the domain name field (it's
already null for unredeemed tokens).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214366534
2018-10-03 11:52:31 -04:00
mcilwain
633b30725a Handle Purchase Order numbers explicitly in billing pipeline
Note that these aren't used for most registrars.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213631863
2018-09-20 11:19:36 -04:00
mcilwain
8ddbf88151 Remove the deprecated MISTAKEN_PREMIUM reservation type
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
2018-09-14 21:28:23 -04:00
weiminyu
80b0e6297b Export Premium names to Drive
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212509587
2018-09-14 11:47:38 -04:00
mcilwain
7b87ba41c7 Delete all Limited Release Program (LRP) code
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.

Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
2018-08-20 14:28:16 -04:00
mcilwain
d2f849ac0f Add new reserved domain creation from allocation tokens mechanism
Note that this gets rid of anchor tenant codes in reserved lists (yay!), which
are no longer valid.  They have to come from allocation tokens now.

This removes support for LRP from domain application create flow (that's fine,
we never used it and I'm going to delete all of LRP later).  It also uses
allocation tokens from EPP authcodes as a fallback, for now, but that will be
removed later once we switch fully to the allocation token mechanism.

This doesn't yet allow registration of RESERVED_FOR_SPECIFIC_USE domains using
the allocation token extension; that will come in the next CL.  Ditto for
showing these reserved domains as available on domain checks when the allocation
token is specified.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209019617
2018-08-20 14:22:06 -04:00
mcilwain
d80f431e21 Add domain name support to AllocationToken entities
The design doc is at []
The next step will be to tie this into the domain create flow, and if the domain
name is on a reserved list, allow it to be created if the token is specified that
has the given domain name on it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207884521
2018-08-10 13:46:48 -04:00
mcilwain
f614044681 Add an auto update time field to the Cursor entity
Also adjusts the nomulus list_cursors command to output the value of this field.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206646117
2018-08-10 13:44:25 -04:00
mcilwain
ded40851d3 Use locking on async mapreduces
This ensures that only one will run at a time, which should help fix the
clogged up mapreduces we've seen on sandbox.

In order to do this, the UnlockerOutput is introduced. This unlocks the
given Lock after all reducer shards have finished.

Also increases the lease duration of the DNS refresh action from 20 to
240 minutes. 20 minutes isn't long enough; when there's a lot of domains
and decent system load the mapreduce could take longer than that in the
ordinary case.

TESTED=Deployed to alpha and verified that more than one copy of the
mapreduce wouldn't run simultaneously, and also that the lock is
released when the mapreduce is finished.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205887554
2018-08-10 13:44:25 -04:00
mcilwain
df5d9957f7 Move the AllocationToken entity to the token subpackage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204485366
2018-07-14 01:37:03 -04:00