Commit graph

831 commits

Author SHA1 Message Date
mcilwain
27b9244126 Fix non-determinism in RegistryTest duplicate auth code test
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149576311
2017-03-13 11:22:55 -04:00
mcilwain
815dae2749 Refactor WhoisReader.readCommand() method to take a now param
This obviates the need for the use of @AutoFactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149570411
2017-03-13 10:45:40 -04:00
mcilwain
01bb3a30f2 Fix handling around size of batches of mapreduce entities to process
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149562105
2017-03-13 10:42:44 -04:00
shikhman
1dfb6afa4b Add command description to get_resource_by_key command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149561503
2017-03-13 10:41:15 -04:00
mcilwain
d2ca4b7234 Refactor main loop of MapreduceEntityCleanupAction
This also tightens up some error-checking conditions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149552065
2017-03-13 10:38:21 -04:00
shikhman
b05afd2153 Fix nomulus tool NPEs on login/logout commands when printing usage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149535358
2017-03-13 10:35:23 -04:00
jart
0c7443815a Add generates_api to Auto Factory too
We're going to play it safe, since the third_party definition of Auto Factory seems to have it too.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149534930
2017-03-13 10:32:24 -04:00
mcilwain
3fcb564251 Record metrics for WHOIS commands
Note that this does not write out metrics for invocations of the
nomulus tool.

This requires a slight refactoring of the existing WhoisResponse
interface so as to also support returning the number of results found
by the WHOIS query.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149461208
2017-03-13 10:30:53 -04:00
mmuller
4eef02f17f Inject token manager into LoadTestAction constructor
Inject into constructor instead of an instance variable, since it gets used
in the constructor before the instance variables get injected.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149452717
2017-03-13 10:29:22 -04:00
mcilwain
ce4f3c0d56 Don't allow setting reserved lists with conflicting auth codes
This is an error condition that will soon throw an exception when
attempting to register the domain name, so it's good to let the registry
operator know of the error when it is first introduced.

Unfortunately there's still a backdoor that allows duplicate labels
that's harder to protect against (that this commit doesn't cover): the
case where reserved lists are already applied to a TLD, then one of the
reserved lists is updated to add another auth code, which then conflicts
with one on a different reserved list.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149443007
2017-03-13 10:27:51 -04:00
mountford
5d4287a375 Add mapreduce cleanup action
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149432516
2017-03-13 10:24:57 -04:00
mcilwain
bd7db61606 Use Dagger to @Inject WhoisReader instances
This is a precursor to adding metrics to WHOIS queries (as I'd like
to be able to @Inject the metrics builders).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149418018
2017-03-07 14:02:46 -05:00
mcilwain
53785c2fc7 Add WHOIS metrics classes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149344437
2017-03-07 14:01:16 -05:00
mmuller
b6f3d60e0f Refactor command running around auth exceptions
Refactor command and component code in RegistryCli so that we can handle a
LoginRequiredException from whereever we are likely to ever get one.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149329171
2017-03-07 13:59:40 -05:00
Ben McIlwain
c0e195e144 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260890
2017-03-07 13:56:41 -05:00
Ben McIlwain
65fb0aee6a Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260762
2017-03-07 13:55:13 -05:00
Ben McIlwain
0cbc2bdab6 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260759
2017-03-07 13:53:46 -05:00
Ben McIlwain
2bba5660c2 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149172966
2017-03-07 13:50:53 -05:00
mountford
034312c17d Fix bug in authentication code
[] changed RequestHandler to log a warning instead of returning an error to the user when authentication failed. Unfortunately, it didn't handle the resulting absent value of AuthResult. I don't yet know why the tests pass, and plan to add a test to check for this situation, but I wanted to start by fixing the problem as soon as possible, as it seems like this would result in exceptions any time a request was handled which didn't pass authentication.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149136265
2017-03-07 13:49:27 -05:00
mountford
9a15f08b3a Add Truth subjects for incrementable and event metrics
This CL adds Truth framework subjects to some metrics in the Stackdriver metrics library, in a contrib subpackage. It doesn't deal with gauge metrics, and for event metrics, the assertions can only be that a metric has or does not have a distribution for a particular set of label values. Asserting more fine-grained propositions regarding the distribution will require a distribution subject.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149112692
2017-03-07 13:46:38 -05:00
mmuller
5614760d53 Implement login/logout commands
Refactor the auth code into its own dagger module, add tests and use the new interfaces to implement the login and logout commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149108266
2017-03-07 13:45:11 -05:00
mountford
ee2bd594c8 Change new authorization logic to log a warning rather than rejecting the request
This is the first step in rolling out the changes so that we can check via logging whether turning on the logic would reject anything it should not.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149050878
2017-03-07 13:42:16 -05:00
nickfelt
3a7f67b7f3 Fix DomainTransferRequestFlow to correctly cancel autorenew graces
This fixes longstanding bug b/19430703 in which domain transfers that were
server-approved would only handle the autorenew grace period correctly if
the autorenew grace period was going to start within the transfer window.
If the autorenew grace period was already active (e.g. the domain had
recently autorenewed, before the transfer was requested), the logic would
miss it, even if it was going to be active throughout the transfer window
(i.e. it would still be active at the server-approval time).

When the autorenew grace period is active at the time a transfer is approved
(whether by the server or explicitly via DomainTransferApproveFlow), the
correct behavior is to essentially "cancel" the autorenew - the losing registrar
receives a refund for the autorenew charge, and the gaining registrar's transfer
extended registration years are applied to the expiration time as it was prior
to that autorenew.  The way we implement this is that we just have the transfer
essentially "subsume" the autorenew - we deduct 1 year from the transfer's
extended registration years before extending the registration period from what
the expiration time is post-autorenew at the moment of transfer approval.

See b/19430703#comment17 for details on the policy justification; the only real
ICANN document about this is https://www.icann.org/news/advisory-2002-06-06-en,
but registrars informally document in many places that transfers will trigger
autorenew grace, e.g. see https://support.google.com/domains/answer/3251236

There are still a few parts of this bug that remain unfixed:

  1) RdeDomainImportAction repeats a lot of logic when handling imported domains
     that are in pending transfer, so it will also need to address this case in
     some way, but the policy choices there are unclear so I'm waiting until we
     know more about RDE import goals to figure out how to fix that.

  2) Behavior at the millisecond edge cases is inconsistent - specifically, for
     the case where a transfer is requested such that the automatic transfer
     time is exactly the domain's expiration time (down to the millisecond),
     the correct behavior is a little unclear and this CL for now ignores this
     issue in favor of getting a fix for 99.999% of the issue into prod.  See
     newly created b/35881941 for the gory details.

Also, there are parts of this bug that will be fixed as parts of either
b/25084229 (transfer exDate computations) or b/35110537 (disallowing transfers
with extended registration years other than 1), both of which are less pressing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149024269
2017-03-07 13:39:15 -05:00
nickfelt
f663f00251 Add test and commentary for "phantom autorenew" in pendingDelete
This documents some slightly spooky behavior around domains that have an expiration time within their pendingDelete window (meaning the whole period from DomainDeleteFlow running to the actual deletionTime, not just the 5-day pendingDelete grace period).  They will experience an autorenew in terms of expiration time and grace period status due to cloneProjectedAtTime(), but without the usual artifacts of an autorenew (billing event and poll message).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149019980
2017-03-07 13:37:40 -05:00
Ben McIlwain
fa5607c3e1 Use Immutable types for static final fields
For constant field declarations, you should use the immutable type (such as ImmutableList) instead of the general collection interface type (such as List). This communicates to your callers important semantic guarantees ([]

This change replaces constants (static final CONSTNAT_CASE) declaration type which use the general collection interface (e.g. List) with an immutable type (e.g. ImmutableList).

For more info, see: []

Cleanup change automatically generated by javacflume/refactory
Refactoring: //third_party/java_src/error_prone/project/core/src/main/java/com/google/errorprone/bugpatterns:MutableConstantField_refactoring
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149010021
2017-03-07 13:34:48 -05:00
nickfelt
2e969d6ed1 Change XsrfTokenManager to support new HMAC token format
This follows up on Brian's work to transition not just to a new format
with an empty scope value, but instead to replace the existing format
entirely with a new one that:

  1) includes a version number to support future format migrations
  2) doesn't include a field for the scope at all, since scoping the
     tokens adds no real security benefit and just makes verification
     more difficult
  3) replaces the raw SHA-256 hash with a SHA-256 HMAC instead, as a
     best practice to avoid length-extension attacks [1], even though
     in our particular case they would only be able to extend the
     timestamp and would thus be relatively innocuous

The new format will be produced by calling generateToken(), and the
scope-accepting version is renamed to generateLegacyToken() in addition
to its existing deprecation, for maximum clarity.

I changed the validateToken() logic to stop accepting a scope entirely;
when validating a legacy-style token, we'll test it against the two
existing legacy scope values ("admin" and "console") and accept it if
it matches either one.

Note that this means the xsrfScope parameter in @Action is now wholly
obsolete; I'll remove it in a follow-up to avoid bringing extra files
into this CL.

After this CL hits production, the next one will replace all calls to
generateLegacyToken() with generateToken().  Once that CL is deployed,
the last step will be removing the legacy fallback in validateToken().

[1] See https://en.wikipedia.org/wiki/Length_extension_attack

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148936805
2017-03-07 13:31:46 -05:00
nickfelt
499f1e7dbc Rewrite ServerSecret caching and accessor logic
I'm working on some changes to XsrfTokenManager (b/35388772) and ServerSecret
was crufty enough that I ended up rewriting it.  Now it uses a LoadingCache
with a transaction instead of needlessly race-condition-y static init logic.

It also now supports retrieving its value as either a UUID (the old format
used by XsrfTokenManager) or a byte[].  The latter is more flexible and can
be directly used with HMAC which the new XsrfTokenManager format will employ.

And lastly, I added tests.  In addition, I tested this code on alpha and
verified appropriate operation (XSRF tokens still work from the console and
from regtool; if you remove ServerSecret from datastore and memcache, it
persists a new one).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148931620
2017-03-07 13:30:18 -05:00
nickfelt
c56959b62b Add new DomainResource.getGracePeriodsOfType() method
This adds a new method which will be used in an upcoming CL affecting domain
transfer logic.  It also removes two older methods that are unused (they were
originally going to be used for TLD-specific logic which is now obsolete).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148928965
2017-03-07 13:28:49 -05:00
jart
13249db5cf Set generates_api=1 for Dagger component processor
Context: [] and []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148885749
2017-03-07 13:27:21 -05:00
shikhman
51a613bca6 Fix bugs in KmsConnectionImpl
A few errors emerged when doing an integration test against the actual API. I've updated the unit tests to reflect the correct behavior.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148793856
2017-03-07 13:25:51 -05:00
nickfelt
726e925b4a Refactor a few new XsrfTokenManager methods
Followup to comments on []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148792464
2017-03-07 13:24:21 -05:00
mcilwain
822cbc0494 Add missing newline to default-config.yaml
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148778586
2017-03-07 13:22:52 -05:00
shikhman
ddfe4c8a47 Add KMS parameters in configuration
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148778412
2017-03-07 13:21:23 -05:00
mountford
c7a62e9b98 Add XSRF protection to legacy authentication mechanism
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148689952
2017-03-07 13:18:04 -05:00
ctingue
a5932c0fc3 Drop FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148688861
2017-03-07 13:16:34 -05:00
ctingue
023a76349e Add ROID flag in FixDomainNameserverKeysCommand
There's a deleted domain with bad host resource keys. Since FKIs won't help us here, load a domain directly by key (i.e. by ROID) if we so choose.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148685240
2017-03-07 13:15:06 -05:00
nickfelt
5a00317dcc Exclude SYNTHETIC flag from BillingData BigQuery view
The BillingData view is intended to be the stable, clean source of data for the invoicing pipeline and detail report output.  As such, cutting over to the synthetically created OneTimes is an implementation detail and shouldn't be visible in the output (esp. since that goes straight to the detail reports that registrars receive, and it'd probably just be confusing for them).  I think it's fair to generically exclude the SYNTHETIC flag from showing up here - it will still be in BigQuery on the individual OneTimes if we want to look for it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148682719
2017-03-07 13:13:37 -05:00
mcilwain
64bfbea324 Remove duplicate version of lookupSuperordinateDomain() function
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148657943
2017-03-07 13:12:09 -05:00
mcilwain
90114858fa Perform minor refactors on premium list code
Principally, this moves a load method into DatastoreHelper that is now
only used by tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148649087
2017-02-27 11:30:45 -05:00
mcilwain
8d84397e80 Fix Nomulus build errors
This fixes Java7 generic type inference errors in premium list code
and upgrades the Truth library to v0.32, because we're now using
assert(e).hasMessageThat() which is not in prior versions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148501657
2017-02-27 11:30:45 -05:00
mcilwain
ea4e471c04 Move premium list static helper methods into their own class
It was kind of messy having all of that logic living alongside the
entities themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148498024
2017-02-27 11:30:45 -05:00
shikhman
388dd1055e Abstract KMS code with KmsConnection and create a fake KmsConnection
This simplifies the tests for KmsKeyring and KmsUpdater.

This is a followup to []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496758
2017-02-27 11:30:45 -05:00
Wolfgang Meyers
9f90597691 Handle pending transfers on domain imports from RDE deposits
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496597
2017-02-27 11:30:40 -05:00
mcilwain
3ca9bb6aeb Read from bloom filter for premium pricing checks
This also cleans up the PremiumList API so that it only has one
method for checking premium prices, which is by TLD, rather than two.

I will be refactoring a lot of the static methods currently residing in
the PremiumList class into a separate utils class, but I don't want to
include too many changes in this one CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148475345
2017-02-27 11:22:21 -05:00
mountford
3ac74fa449 Add more tests to new authentication framework
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148459400
2017-02-27 11:19:27 -05:00
mcilwain
dd400f30f5 Don't allow duplicates in premium/reserved lists
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148458642
2017-02-27 11:17:58 -05:00
nickfelt
ea3a8dfa9d Make Router reject classes with no @Action-returning methods
This provides a safeguard against using TypeInstantiator to resolve the component class, where if resolution is done incorrectly, you end up with java.lang.Object.  Formerly, that would have "succeeded" in generating a Router for Object, which of course has no methods that return @Action classes.  Such a router is pretty useless, so it's better to make Router stricter and have it fail if you give it such a class by accident.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148353224
2017-02-27 11:16:30 -05:00
nickfelt
1e7fc4d64d Remove Builder type param on RequestComponentBuilder/RequestHandler
It turns out this type parameter was never necessary.  A builder only needs the reflexive second type parameter when you want to have a builder inheritance hierarchy where the descendant builders have methods that the ancestor builder doesn't.  In that case, the type param enables the ancestor builder's setter methods to automatically return the correct derived type, so that if you start with a derived builder, you can call a setter method inherited from an ancestor and then continue the chain with setters from the derived builder (e.g. new ContactResource.Builder().setCreationTime(now).setContactId(), which otherwise would have returned an EppResource.Builder from setCreationTime(), at which point the call to setContactId() would not compile).

Even then, it's not strictly necessary to use the type parameter, since you could instead just have each derived type override every inherited method to specify itself as the return type.  But that would be a lot of extra boilerplate and brittleness.

Anyway, in this case, there is a builder hierarchy, but RequestComponentBuilder specifies all the methods that we're ever going to want on our builders, so there's never any need to be able to call specific derived builder methods.  We only even need the individual builder classes so that Dagger can generate them separately for each component.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148269178
2017-02-27 11:12:07 -05:00
nickfelt
16832323d0 Make ListObjectsAction return 200 when sending JSON error
This fixes a bug in the interaction between ListObjectsAction and ListObjectsCommand/AppEngineConnection.  ListObjectsAction was returning HTTP status code 400 when it caught an IAE, but also attempting to return a JSON response payload of {"status": "error", "error": "<exception message>"}.  However, AppEngineConnection treats any HTTP error response as more like a crash on the server side - it attempts to scrape the error message out of the autogenerated HTML that AppEngine produces for uncaught exceptions, and throws an exception, killing ListObjectsCommand before it can extract the JSON which contains the nicer error (that stating the missing field, etc versus just "400 Bad Request").

The fix is just to have ListObjectsAction return a 200 and the error message so that ListObjectsCommand can correctly handle it.

I also de-scoped the catch to only catching IAE, since catching Exception was overbroad, and the only "expected" exception to be thrown is an IAE from the checkArgument() that tests if the requested fields all exist.  Any other kinds of exceptions should actually just bubble up and kill the action, and get the regular AppEngineConnection error treatment.

I also added "billingId" as an alias for "billingIdentifier", parallel to clientId/clientIdentifier, since that's why I came across this issue in the first place.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148248834
2017-02-27 11:10:37 -05:00
mmuller
acc7d19106 Put the path to client_secrets in a config option.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148214513
2017-02-22 11:49:03 -05:00