Commit graph

395 commits

Author SHA1 Message Date
shicong
bc3bdc7347 Upgrade the version of google/errorprone plug-in
This CL upgraded google/errorprone plug-in to 2.3.3 and resolved
some warnings detected from the plug-in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238047862
2019-03-20 14:25:28 -04:00
mcilwain
49ac4e3e69 Improve Datastore efficiency of duplicate contact messages
I should have caught this in the review, but [] is loading *ALL*
contacts individually from Datastore on every domain update. This will add a
large number of Datastore round trips and thus significantly reduce update
performance.

This CL changes the behavior to *ONLY* load contacts when there is a duplicate
(which is needed to determine the contact's display name to generate the error
message), and loads all of them in a single batch rather than individually.

This also makes some minor changes around domain getters returning empty sets
instead of null.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233128140
2019-02-11 11:22:34 -05:00
shicong
c2445c3797 Rename loadByForeignKeys to loadByForeignKeysCached
This is to remind the user that the function actually uses cache, and also
for naming consistency with EppResourceUtils.loadByForeignKeyCached().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232870846
2019-02-08 11:17:38 -05:00
gbrodman
93ff541006 Handle sunrise reservations properly during sunrise TLDs
We figure out the TLD state so that we properly check whether or not we can provision sunrise domains in that TLD. We also change the message slightly so that it's a bit more clear when we aren't in sunrise.

Note: it is deliberate that NAME_COLLISION reservations are provisionable in sunrise.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232742813
2019-02-06 18:06:21 -05:00
mcilwain
c6e58d3bff Fix some issues caught by IntelliJ static code analysis
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
2019-01-28 16:08:24 -05:00
gbrodman
3cf26ff9b6 Fix various Error Prone errors that were found by the FOSS build
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
2019-01-28 16:05:09 -05:00
mcilwain
e2528875b2 Merge DomainResource into DomainBase
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.

It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.

This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
2019-01-28 15:57:10 -05:00
mcilwain
37aa1d1815 Always require acknowledgment of premium fees
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.

Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
2019-01-17 19:07:51 -05:00
mcilwain
8ac8ecf8f6 Rationalize OT&E client ID validation rules
This makes the validation rules much simpler, thus placing less cognitive load on the users of the console and nomulus tool.  The changes are:

1. Don't allow hyphens. No real registrars use hyphens in their client IDs, and it's better to reserve these solely as the delimiter between the base client ID and the number representing the environment.
2. Allow the first character to be a number.  This has affected multiple real registrars, causing their OT&E and production client IDs to be different.  There's no reason for this restriction, as the only reason motivating it was that there are no TLDs that start with a number.  However, the OT&E TLDs are created only in sandbox and never have DNS syncing enabled, so this restriction is purposeless.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229187198
2019-01-14 16:28:12 -05:00
mcilwain
170980db2f Statically import commonly used TldState enum values
Takes advantage of the fact that the default state of a TLD created in tests is GENERAL_AVAILABILITY.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228916164
2019-01-14 16:23:15 -05:00
weiminyu
a80a44cd06 Define TestRule that manages cache use in tests
All current tests that use caches with custom data expiry values
now restore the default config when teardown. We need to prevent
new unsafe uses from being introduced.

Restoration code have also been added to a few other tests that modifies
static fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228888041
2019-01-11 11:50:33 -05:00
guyben
a4fca18657 Set short redemption/add/delete periods for all OT&E TLDs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228791942
2019-01-11 10:59:21 -05:00
mcilwain
580302898d Delete end-date sunrise, landrush, and sunrush phases
This also deletes the associated commands and domain application specific
entities.

We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.

Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
2019-01-10 16:23:35 -05:00
guyben
c74ffd7559 Fix @VisibleForTesting given the newly deployed enforcement
Generated code is now also covered by @VisibleForTesting, including Dagger @Inject

This CL is a cleanup of auto-generated code by ghm@ from the Error Prone team

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228748874
2019-01-10 16:23:35 -05:00
guyben
898448b8a0 Reverse dependency between /flows/ and /batch/
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.

However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.

This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/

Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/

It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
2019-01-10 16:23:35 -05:00
Shicong Huang
a79e254e49 Add an action to check the status of OT&E verification
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228333195
2019-01-08 16:46:41 -05:00
jianglai
2e7d71b238 Refactor most of OT&E verification code to exist in utils class
This does not change existing functionality but will allow us to use this common code in the yet-to-be-created web console action as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227159346
2019-01-02 11:58:36 -05:00
guyben
2777018d6a Add the ability to setup OT&E from the web console
We create a new endpoint with a simple form that will let admins (including
support) setup OT&E for registrars.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226570568
2019-01-02 11:56:59 -05:00
mcilwain
56b61ad5a2 Invalidate premium list cache on update
This will only affect the tools service, the primary use case being (1) I go to
create a domain through nomulus tool, realize it's premium, (2) update the
premium list to not include that domain, (3) kill the tools service instance to
wipe out the cached premium value, then (4) create the domain at standard. This
commit eliminates step 3.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226180160
2018-12-20 07:46:33 -05:00
guyben
9d6a7ef66a Create OT&E entities directly, instead of calling sub-commands
This is in preparation for having a web-console endpoint to create OTE.

In addition - we streamline the code:

- we remove support for different premium lists
- we remove support for different DNS writers - we never want a "real" DnsWriter for OTE
- we remove support of --eap_only, because we don't need it anymore
- We use a single password for all the Registrars

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225841694
2018-12-20 07:46:33 -05:00
mcilwain
4491b7b909 Make loadByForeignKey() and related methods return Optional
This is safer and addresses a common source of confusion in the codebase because it's always explicit that the resource returned may not be present, whether because it's soft-deleted when projected to the given time or because it never existed in the first place.

In production code, the presence of the returned value is always checked. In test code, its presence is assumed using .get() where that is expected and convenient, as it not being present will throw an NPE that will cause the test to fail anyway.

Note that the roughly equivalent reloadResourceByForeignKey(), which is widely used in test code, is not having this same treatment applied to it. That is out of the scope of this CL, and has much smaller returns anyway because it's only used in tests (where the unexpected absence of a given resource would just cause the test to fail).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225424002
2018-12-14 11:24:49 -05:00
mcilwain
015c854a92 Make loadByForeignKeyCached() return an Optional type
Next up (and a much larger commit) will be giving loadByForeignKey() the same
treatment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225182377
2018-12-12 13:22:34 -05:00
jianglai
0a44ef0dca Refactor EppXmlTransformer to be in the model/ package
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
2018-12-12 13:22:34 -05:00
guyben
4dad0a8a73 Fix checkState failing to trigger retrier's retry
We have a retrier wanting to call a function, retrying on
IllegalStateExceptions (thrown by checkState()).

However, one of the checkStates is called inside a Concurrent.transform, so
when the checkState fails, the resulting IllegalStateException is wrapped in an
UncheckedExecutionException and isn't caught by the retrier.

We unwrap the IllegalStateException to make sure it's caught.

Q: Why not just catch UncheckedExecution exception?
A: Because it might wrap a different exception which we don't want to retry on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224862573
2018-12-12 13:22:34 -05:00
shicong
5bc70cbc99 Update lastEppUpdateTime after certain grace period is passed
The lastEppUpdateTime should be updated asynchronously in the situations below:

 - Implicit transfer success after 5 day pending transfer period
 - Implicit end of any grace period

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224831791
2018-12-12 13:22:34 -05:00
mcilwain
b317aab22f Clean up some areas in preparation for externalized codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221454055
2018-11-16 16:51:11 -05:00
jianglai
589e98a1db Explicitly set the shard size when saving the claims list
This allows us to get rid of the use of InjectRule in tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218221409
2018-10-22 19:13:11 -04:00
mcilwain
f43125bb04 Add a RESERVED billing event flag for creates of reserved domains
This is an additional signal we can use for the billing/invoicing pipeline that
will be helfpul in targeting invoice rows that may need adjustment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217524250
2018-10-22 18:38:23 -04:00
mcilwain
9faf7181c3 Index the allocation token's redemption history entry field
We need this to efficiently query which tokens have and have not been redeemed.

I'm also marking it as nullable for consistency with the domain name field (it's
already null for unredeemed tokens).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214366534
2018-10-03 11:52:31 -04:00
mcilwain
633b30725a Handle Purchase Order numbers explicitly in billing pipeline
Note that these aren't used for most registrars.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213631863
2018-09-20 11:19:36 -04:00
mcilwain
8ddbf88151 Remove the deprecated MISTAKEN_PREMIUM reservation type
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
2018-09-14 21:28:23 -04:00
weiminyu
80b0e6297b Export Premium names to Drive
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212509587
2018-09-14 11:47:38 -04:00
mcilwain
7b87ba41c7 Delete all Limited Release Program (LRP) code
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.

Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
2018-08-20 14:28:16 -04:00
mcilwain
d2f849ac0f Add new reserved domain creation from allocation tokens mechanism
Note that this gets rid of anchor tenant codes in reserved lists (yay!), which
are no longer valid.  They have to come from allocation tokens now.

This removes support for LRP from domain application create flow (that's fine,
we never used it and I'm going to delete all of LRP later).  It also uses
allocation tokens from EPP authcodes as a fallback, for now, but that will be
removed later once we switch fully to the allocation token mechanism.

This doesn't yet allow registration of RESERVED_FOR_SPECIFIC_USE domains using
the allocation token extension; that will come in the next CL.  Ditto for
showing these reserved domains as available on domain checks when the allocation
token is specified.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209019617
2018-08-20 14:22:06 -04:00
mcilwain
d80f431e21 Add domain name support to AllocationToken entities
The design doc is at []
The next step will be to tie this into the domain create flow, and if the domain
name is on a reserved list, allow it to be created if the token is specified that
has the given domain name on it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207884521
2018-08-10 13:46:48 -04:00
mcilwain
f614044681 Add an auto update time field to the Cursor entity
Also adjusts the nomulus list_cursors command to output the value of this field.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206646117
2018-08-10 13:44:25 -04:00
mcilwain
ded40851d3 Use locking on async mapreduces
This ensures that only one will run at a time, which should help fix the
clogged up mapreduces we've seen on sandbox.

In order to do this, the UnlockerOutput is introduced. This unlocks the
given Lock after all reducer shards have finished.

Also increases the lease duration of the DNS refresh action from 20 to
240 minutes. 20 minutes isn't long enough; when there's a lot of domains
and decent system load the mapreduce could take longer than that in the
ordinary case.

TESTED=Deployed to alpha and verified that more than one copy of the
mapreduce wouldn't run simultaneously, and also that the lock is
released when the mapreduce is finished.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205887554
2018-08-10 13:44:25 -04:00
mcilwain
df5d9957f7 Move the AllocationToken entity to the token subpackage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204485366
2018-07-14 01:37:03 -04:00
mcilwain
32b3563126 Delete all Braintree code
We never launched this, don't planning on launching it now anyway, and it's rotted over the past two years anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202993577
2018-07-14 01:37:03 -04:00
mcilwain
43ed2cd7b3 Clean up annotation imports in BUILD files
This affects JSR305, JSR330, and Guava annotations.

The exact command run to generate this CL was:

build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
2018-06-27 15:28:53 -04:00
mcilwain
bee3d6a5a4 Fix BUILD glob rules to not include non-wildcarded paths
This is a Buildozer warning

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202311866
2018-06-27 15:28:53 -04:00
mcilwain
4c7bc3b18c Improve internal build system speed
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202016862
2018-06-27 15:28:52 -04:00
mcilwain
ed910455b0 Add more absent clTrid unit tests
In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.

This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
2018-06-27 15:28:52 -04:00
mcilwain
6706b99828 Delete remnants of registrar-level credits
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
2018-06-27 15:28:52 -04:00
mcilwain
d3364b0387 Migrate away from VoidWorks
This is one last hanging piece of work left over from last year's Java 8
migration. There's no functionality changes in this CL, just refactoring.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201947600
2018-06-27 15:28:52 -04:00
mcilwain
8b263baefa Delete MultiplyingCloudDnsWriter
Now that the large zone re-signing test is complete, we no longer need it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
2018-06-27 15:28:06 -04:00
mcilwain
5d80f124ca Remove unnecessary "throws" declarations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201058582
2018-06-18 18:17:56 -04:00
guyben
646c6fa069 Print GAE-UserId even when null
Also explicitly state that contacts missing GAE-UserId can't access the
registrar console

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402953
2018-06-18 18:06:23 -04:00
larryruili
1c71487e5d Set EAP billing event to one-year period
This is consistent with how we treat RESTORE billing events as well- in
general, fees are considered to be amortized over the course of a year (by the
invoicing team).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199684843
2018-06-18 17:42:29 -04:00
mcilwain
e1bcc2e64d Add relevant exception message for checkArgumentNotNull usage
I ran into this while writing some other code and having the exception message
would have made it easier to debug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199292321
2018-06-06 15:13:10 -04:00