This removes a LOT of "almost duplicate" test files, and moves some of the info into the java test itself making it clearer what's actually different between tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240230982
The only reason why it existed was so that we could get tests to print information in a consistent order and there are other ways of doing that. By removing compareTo we can use the properties of the extended ImmutableObject properly and properly implement the RFC https://tools.ietf.org/html/rfc5910#page-18
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240170488
Moving some of the info into the java test file itself makes it more easily readable. Maybe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240148747
We won't be using this for future launches, so get rid of it now.
Also reorganizes existing anchor tenant test cases and adds a new one for
creation of a TMDBed anchor tenant domain in sunrise.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239669950
Disabled registrar cannot perform any actions via EPP and cannot log in to the
registrar web console.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239606389
Adding it to one test is sufficient because we use the same loading logic across all flows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239506003
In [] a change would have been made to your project that is incompatible with
your open source integration. To make sure the open source variant of your
project remains working, we have eagerly updated your open source copy to use
Mockito 2. This CL integrates that change into [].
Please read []and
understand the consequences of this change.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=238445356
Even though it's not directly related, I also fixed a typo in PREMIUM because it bugged me.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237796791
Mockito in third_party is updated to 1.10. We do not need to backport this rule anymore.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237496086
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.
We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.
Also, this is a security risk, as it allowed to do "billable actions" (creating a new domain for example) with the only authentication being access to the registrar's G Suite account.
This bypassed the certificate, IP whitelist, and EPP password, which is bad.
PUBLIC:
Remove the web console EPP endpoint
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.
We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236244195
Super users can look up auth info in Datastore or BigQuery backup anyway.
Requiring it only adds friction when using the super user extension, without
any additional security benefit.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=235786090
Otherwise, registrars will never receive a notification through EPP that a
domain has been synchronously deleted by us.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=234172289
I should have caught this in the review, but [] is loading *ALL*
contacts individually from Datastore on every domain update. This will add a
large number of Datastore round trips and thus significantly reduce update
performance.
This CL changes the behavior to *ONLY* load contacts when there is a duplicate
(which is needed to determine the contact's display name to generate the error
message), and loads all of them in a single batch rather than individually.
This also makes some minor changes around domain getters returning empty sets
instead of null.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=233128140
This change also added a test to verify that EPP request to modify
both contacts and registrant at same time can be handled as expected.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232935690
We figure out the TLD state so that we properly check whether or not we can provision sunrise domains in that TLD. We also change the message slightly so that it's a bit more clear when we aren't in sunrise.
Note: it is deliberate that NAME_COLLISION reservations are provisionable in sunrise.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232742813
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.
It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.
This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.
Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
Takes advantage of the fact that the default state of a TLD created in tests is GENERAL_AVAILABILITY.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228916164
This also deletes the associated commands and domain application specific
entities.
We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.
Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
Fix a couple more cases when caching related configs are changed.
This does not fix all our problems with Gradle builds.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228735790
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.
However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.
This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/
Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/
It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
This is safer and addresses a common source of confusion in the codebase because it's always explicit that the resource returned may not be present, whether because it's soft-deleted when projected to the given time or because it never existed in the first place.
In production code, the presence of the returned value is always checked. In test code, its presence is assumed using .get() where that is expected and convenient, as it not being present will throw an NPE that will cause the test to fail anyway.
Note that the roughly equivalent reloadResourceByForeignKey(), which is widely used in test code, is not having this same treatment applied to it. That is out of the scope of this CL, and has much smaller returns anyway because it's only used in tests (where the unexpected absence of a given resource would just cause the test to fail).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225424002
Note that it's possible to set a config option to disable this functionality
on a per-environment basis (we're disabling it for sandbox), but in general
SSL certificate hashes should be required for increased security.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225053496
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
This is used to reduce the expiration time of domain(s) by some number of years
(if enough length remains in the registration term to do so). This does not back
out the previously saved BillingEvent entities as they may have already been
sent out and invoiced, so any related refunds must be handled out of band.
In addition to reducing the registration expiration time on the domain itself,
this command writes out a new history entry, one-time poll message informing the
registrar of this change, auto-renew billing event and poll message, and
updates/ends the old auto-renew billing event and poll message.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224999285
The lastEppUpdateTime should be updated asynchronously in the situations below:
- Implicit transfer success after 5 day pending transfer period
- Implicit end of any grace period
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224831791
This also raises their access level so they can be called by other classes that
extend EppTestCase (which I'm writing in a follow-up CL).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223512618
I'm adding another EPP lifecycle test that will need to be in the tools package
because it has to call tools as part of the lifecycle. This commit consolidates
common functionality within the EppTestCase abstract base class (and increases
visibility) so that it can easily be referenced by more extending classes, even
ones in a different package.
This also explicitly loads the test files from the testdata directory collocated
with EppTestCase, so that new tests in other packages won't have to duplicate
lots of these same test files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223365398
EppResource's lastEppUpdateTime and lastEppUpdateClientId need
to be updated whenever the domain is updated, renewed, deleted or
transfered.
This commit applied the change to the following domain EPP commands:
- Update (already implemented)
- Renew
- Delete
- Restore
- Transfer request
- Transfer approve
- Transfer reject
- Transfer cancel
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223344758
Registrars that are PENDING or SUSPENDED should not have access to
EPP commands that cost money, as in either case it's not likely we'd
actually be able to get payment from said registrar. For this reason
we already prevented access to the domain create flow for non-active
registrars. This commit extends that to other commands that cost
money, including renewals, restores, and transfer requests.
Note that implicit autorenews will still occur for suspended
registrars, as in our point-in-time data model there's no good way
to prevent them. So when a registrar is suspended for non-payment,
the game plan is to get all of their domains transferred out to a
registrar that will pay as soon as possible.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223173677
EppConsoleAction still "manually" checks access by going over the
RegistrarContacts. We need it to use AuthenticatedRegistrarAccessor just like
every other part of the registrar console.
We still need to remove the (now unneeded) login EPP sent by the console, but that's left for a followup CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222404208
The main goal of this CL is to make the clock constructor injectable (so that tests do not need to use InjectRule to override the clock). The complication is that the clock is used by two static caches (ROOT_CACHE directly and CRL_CACHE indirectly). The clock is not actually used to construct the lock, but rather to verify that the root certificate is within its validity period.
For ROOT_CACHE we move the verification to its call sites. This adds a bit overhead because the validity check happens every time the cache is called, not just when the cache is built or refreshed. However this check is rather cheap. Also the resources are included in the jar and the cache is valid for 1 year. Given that we deploy every week, there's not much point making it an expiring cache rather than a static map.
For CRL_CACHE we change the key to a tuple of TmchCaMode and X509Certificate. The certificate is no longer provided from the ROOT_CACHE directly and must be verified before it is provided as a cache key. We left the CRL verification inside the cache loader because it (signature verification) is more expensive compared to simple expiration check, and we do not want to do this every time the cache is called.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218385684
Instead of verifying interactions on the mocks, we instead assert on the real test subject directly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218209556
Made it clear that it is a util class and moved some of the functions only used in NordnUploadAction (to NordnUploadAction). Also used Retrier to handle retries when leasing tasks.
These changes allow us to no longer use InjectRule in related unit tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217761117
This is an additional signal we can use for the billing/invoicing pipeline that
will be helfpul in targeting invoice rows that may need adjustment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217524250
The following issues are addressed:
- XML sanitizer should preserve input encoding. Gradle loses any that is not UTF-8. Bazel loses any that is not ASCII.
- Verify that XML sanitizer works with non-UTF8 encoding
- GpgSystemCommandRule breaks when $TMPDIR env variable is not set
- TestDataHelper throws exception when loading resources if resources are plain files on default file system as opposed to being in a jar file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216537258
These are simply too costly in their current form now that we are handling double-digit QPS, so at a minimum we'd want to refactor these for batched exports using a background thread (like how Stackdriver metrics work). However, upon further review, that work isn't worth doing if this BigQuery table isn't actually being used for anything, and it seems that we aren't using it anymore given that ICANN transaction reporting no longer requires it.
So the simplest thing to do is simply to get rid of this entirely, and just use a combination of Stackdriver metrics and App Engine logs. The eppMetrics BigQuery table is ~1.2 billion rows and takes up 223 GB, so that's not an insignificant GCP billings saving if we can delete it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215905466
Note that the check flow does not yet handle any kind of allocation token
handling at all. Step 2 will be to add allocation token handling there, so a
RESERVED_FOR_ANCHOR_TENANT or RESERVED_FOR_SPECIFIC_USE domain will show as
available instead of reserved if the right token is specified using the
extension. Then once that's done, we can use that information to adjust the
price accordingly as well.
Right now the behavior with a domain check is that reserved domains always show
as reserved, even if they're anchor tenants.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215599350
This is only useful when we used the [] proxy because the GFE requires SNI during handshake in order to request the client certificate. The GCP proxy does not need this (it always requests the client certificate). We do not need to check for its existence.
Also removed the checking of internal headers for ssl cert hash used only by the [] proxy.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213059027
Unlike anchor tenants, these domains can be registered for any number of years,
but only during GA, as third parties cannot register domains pre-GA except
through the anchor tenant program.
Since this is new functionality, unlike creation of anchor tenants, there is no
fallback provided to send codes through the domain authcode; they must be sent
using the allocation token extension.
And note that, like with anchor tenants, providing the domain-specific
allocation token overrides any other reserved types that might apply to that
domain.
No changes are necessary to the domain application create flow because of the
above restriction to GA.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212310701
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.
Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
Note that this gets rid of anchor tenant codes in reserved lists (yay!), which
are no longer valid. They have to come from allocation tokens now.
This removes support for LRP from domain application create flow (that's fine,
we never used it and I'm going to delete all of LRP later). It also uses
allocation tokens from EPP authcodes as a fallback, for now, but that will be
removed later once we switch fully to the allocation token mechanism.
This doesn't yet allow registration of RESERVED_FOR_SPECIFIC_USE domains using
the allocation token extension; that will come in the next CL. Ditto for
showing these reserved domains as available on domain checks when the allocation
token is specified.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209019617
Masks user credentials (tags 'pw' and 'newPW') in EPP XML messages.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207953894
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).
This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.
This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
