Commit graph

836 commits

Author SHA1 Message Date
mountford
1f000b94e6 Set correct auth settings for all actions
A test has been added to RequestHandlerTest, making sure that, while we merely log errors for the time being, the correct dummy AuthResult is being created.

Most actions use the default settings, which have been changed to INTERNAL / APP / IGNORED. Actions with non-default settings are:

INTERNAL/NONE/PUBLIC (non-auth public endpoints)

CheckApiAction
WhoisHttpServer
Rdap*Action

INTERNAL,API/APP/ADMIN (things currently protected by web.xml)

EppTlsAction
EppToolAction
CreateGroupsAction
CreatePremiumListAction
DeleteEntityAction
List*sAction
UpdatePremiumListAction
VerifyOteAction
WhoisServer

INTERNAL,API,LEGACY/USER/PUBLIC (registrar console)

RegistrarPaymentAction
RegistrarPaymentSetupAction
RegistrarSettingsAction
EppConsoleAction

INTERNAL,API,LEGACY/NONE/PUBLIC (registrar console main page)

ConsoleUiAction

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149761652
2017-03-13 11:22:55 -04:00
Ben McIlwain
f5e76868f0 Add generic implementation comparator
This is an alternative to the "manually constructed" version in []

This generator recieves two objects implementing some interface. It returns a new implementation that exactly duplicates the behavior (return values, exceptions etc.) of the first ("original") object, but also logs any differences (in either return value or thrown exception) from the second object.

Nothing in the second object's execution will affect the results - not even thrown exceptions - it's only used for logging any differences.

This is useful when switching backend architecture for the some interface (say... from keystore to datastore+KMS for the Keyring interface :)) as we can make sure everything was OK for some extended period of real data before doing the switch.

This is better than just comparing the outputs once because having a completely different backend out of our control can result in "glitches" that are rare but relevant: maybe the new backend has lower availability, or sometimes fails and gives wrong results, or... well, that's it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149706165
2017-03-13 11:22:55 -04:00
mcilwain
fa5d23bc19 Clarify what affects the size of the premium list entry cache
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149680785
2017-03-13 11:22:55 -04:00
mcilwain
3b2a014524 Don't log premium list nomulus tool usage errors as SEVERE
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149661618
2017-03-13 11:22:55 -04:00
mcilwain
ad840ad4d6 Better handle syncing contacts when creating RegistrarContacts
This also adds a test that should've been there already.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149583500
2017-03-13 11:22:55 -04:00
mcilwain
27b9244126 Fix non-determinism in RegistryTest duplicate auth code test
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149576311
2017-03-13 11:22:55 -04:00
mcilwain
815dae2749 Refactor WhoisReader.readCommand() method to take a now param
This obviates the need for the use of @AutoFactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149570411
2017-03-13 10:45:40 -04:00
mcilwain
01bb3a30f2 Fix handling around size of batches of mapreduce entities to process
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149562105
2017-03-13 10:42:44 -04:00
shikhman
1dfb6afa4b Add command description to get_resource_by_key command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149561503
2017-03-13 10:41:15 -04:00
mcilwain
d2ca4b7234 Refactor main loop of MapreduceEntityCleanupAction
This also tightens up some error-checking conditions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149552065
2017-03-13 10:38:21 -04:00
shikhman
b05afd2153 Fix nomulus tool NPEs on login/logout commands when printing usage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149535358
2017-03-13 10:35:23 -04:00
jart
0c7443815a Add generates_api to Auto Factory too
We're going to play it safe, since the third_party definition of Auto Factory seems to have it too.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149534930
2017-03-13 10:32:24 -04:00
mcilwain
3fcb564251 Record metrics for WHOIS commands
Note that this does not write out metrics for invocations of the
nomulus tool.

This requires a slight refactoring of the existing WhoisResponse
interface so as to also support returning the number of results found
by the WHOIS query.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149461208
2017-03-13 10:30:53 -04:00
mmuller
4eef02f17f Inject token manager into LoadTestAction constructor
Inject into constructor instead of an instance variable, since it gets used
in the constructor before the instance variables get injected.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149452717
2017-03-13 10:29:22 -04:00
mcilwain
ce4f3c0d56 Don't allow setting reserved lists with conflicting auth codes
This is an error condition that will soon throw an exception when
attempting to register the domain name, so it's good to let the registry
operator know of the error when it is first introduced.

Unfortunately there's still a backdoor that allows duplicate labels
that's harder to protect against (that this commit doesn't cover): the
case where reserved lists are already applied to a TLD, then one of the
reserved lists is updated to add another auth code, which then conflicts
with one on a different reserved list.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149443007
2017-03-13 10:27:51 -04:00
mountford
5d4287a375 Add mapreduce cleanup action
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149432516
2017-03-13 10:24:57 -04:00
mcilwain
bd7db61606 Use Dagger to @Inject WhoisReader instances
This is a precursor to adding metrics to WHOIS queries (as I'd like
to be able to @Inject the metrics builders).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149418018
2017-03-07 14:02:46 -05:00
mcilwain
53785c2fc7 Add WHOIS metrics classes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149344437
2017-03-07 14:01:16 -05:00
mmuller
b6f3d60e0f Refactor command running around auth exceptions
Refactor command and component code in RegistryCli so that we can handle a
LoginRequiredException from whereever we are likely to ever get one.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149329171
2017-03-07 13:59:40 -05:00
Ben McIlwain
c0e195e144 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260890
2017-03-07 13:56:41 -05:00
Ben McIlwain
65fb0aee6a Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260762
2017-03-07 13:55:13 -05:00
Ben McIlwain
0cbc2bdab6 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149260759
2017-03-07 13:53:46 -05:00
Ben McIlwain
2bba5660c2 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149172966
2017-03-07 13:50:53 -05:00
mountford
034312c17d Fix bug in authentication code
[] changed RequestHandler to log a warning instead of returning an error to the user when authentication failed. Unfortunately, it didn't handle the resulting absent value of AuthResult. I don't yet know why the tests pass, and plan to add a test to check for this situation, but I wanted to start by fixing the problem as soon as possible, as it seems like this would result in exceptions any time a request was handled which didn't pass authentication.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149136265
2017-03-07 13:49:27 -05:00
mountford
9a15f08b3a Add Truth subjects for incrementable and event metrics
This CL adds Truth framework subjects to some metrics in the Stackdriver metrics library, in a contrib subpackage. It doesn't deal with gauge metrics, and for event metrics, the assertions can only be that a metric has or does not have a distribution for a particular set of label values. Asserting more fine-grained propositions regarding the distribution will require a distribution subject.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149112692
2017-03-07 13:46:38 -05:00
mmuller
5614760d53 Implement login/logout commands
Refactor the auth code into its own dagger module, add tests and use the new interfaces to implement the login and logout commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149108266
2017-03-07 13:45:11 -05:00
mountford
ee2bd594c8 Change new authorization logic to log a warning rather than rejecting the request
This is the first step in rolling out the changes so that we can check via logging whether turning on the logic would reject anything it should not.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149050878
2017-03-07 13:42:16 -05:00
nickfelt
3a7f67b7f3 Fix DomainTransferRequestFlow to correctly cancel autorenew graces
This fixes longstanding bug b/19430703 in which domain transfers that were
server-approved would only handle the autorenew grace period correctly if
the autorenew grace period was going to start within the transfer window.
If the autorenew grace period was already active (e.g. the domain had
recently autorenewed, before the transfer was requested), the logic would
miss it, even if it was going to be active throughout the transfer window
(i.e. it would still be active at the server-approval time).

When the autorenew grace period is active at the time a transfer is approved
(whether by the server or explicitly via DomainTransferApproveFlow), the
correct behavior is to essentially "cancel" the autorenew - the losing registrar
receives a refund for the autorenew charge, and the gaining registrar's transfer
extended registration years are applied to the expiration time as it was prior
to that autorenew.  The way we implement this is that we just have the transfer
essentially "subsume" the autorenew - we deduct 1 year from the transfer's
extended registration years before extending the registration period from what
the expiration time is post-autorenew at the moment of transfer approval.

See b/19430703#comment17 for details on the policy justification; the only real
ICANN document about this is https://www.icann.org/news/advisory-2002-06-06-en,
but registrars informally document in many places that transfers will trigger
autorenew grace, e.g. see https://support.google.com/domains/answer/3251236

There are still a few parts of this bug that remain unfixed:

  1) RdeDomainImportAction repeats a lot of logic when handling imported domains
     that are in pending transfer, so it will also need to address this case in
     some way, but the policy choices there are unclear so I'm waiting until we
     know more about RDE import goals to figure out how to fix that.

  2) Behavior at the millisecond edge cases is inconsistent - specifically, for
     the case where a transfer is requested such that the automatic transfer
     time is exactly the domain's expiration time (down to the millisecond),
     the correct behavior is a little unclear and this CL for now ignores this
     issue in favor of getting a fix for 99.999% of the issue into prod.  See
     newly created b/35881941 for the gory details.

Also, there are parts of this bug that will be fixed as parts of either
b/25084229 (transfer exDate computations) or b/35110537 (disallowing transfers
with extended registration years other than 1), both of which are less pressing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149024269
2017-03-07 13:39:15 -05:00
nickfelt
f663f00251 Add test and commentary for "phantom autorenew" in pendingDelete
This documents some slightly spooky behavior around domains that have an expiration time within their pendingDelete window (meaning the whole period from DomainDeleteFlow running to the actual deletionTime, not just the 5-day pendingDelete grace period).  They will experience an autorenew in terms of expiration time and grace period status due to cloneProjectedAtTime(), but without the usual artifacts of an autorenew (billing event and poll message).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149019980
2017-03-07 13:37:40 -05:00
Ben McIlwain
fa5607c3e1 Use Immutable types for static final fields
For constant field declarations, you should use the immutable type (such as ImmutableList) instead of the general collection interface type (such as List). This communicates to your callers important semantic guarantees ([]

This change replaces constants (static final CONSTNAT_CASE) declaration type which use the general collection interface (e.g. List) with an immutable type (e.g. ImmutableList).

For more info, see: []

Cleanup change automatically generated by javacflume/refactory
Refactoring: //third_party/java_src/error_prone/project/core/src/main/java/com/google/errorprone/bugpatterns:MutableConstantField_refactoring
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149010021
2017-03-07 13:34:48 -05:00
nickfelt
2e969d6ed1 Change XsrfTokenManager to support new HMAC token format
This follows up on Brian's work to transition not just to a new format
with an empty scope value, but instead to replace the existing format
entirely with a new one that:

  1) includes a version number to support future format migrations
  2) doesn't include a field for the scope at all, since scoping the
     tokens adds no real security benefit and just makes verification
     more difficult
  3) replaces the raw SHA-256 hash with a SHA-256 HMAC instead, as a
     best practice to avoid length-extension attacks [1], even though
     in our particular case they would only be able to extend the
     timestamp and would thus be relatively innocuous

The new format will be produced by calling generateToken(), and the
scope-accepting version is renamed to generateLegacyToken() in addition
to its existing deprecation, for maximum clarity.

I changed the validateToken() logic to stop accepting a scope entirely;
when validating a legacy-style token, we'll test it against the two
existing legacy scope values ("admin" and "console") and accept it if
it matches either one.

Note that this means the xsrfScope parameter in @Action is now wholly
obsolete; I'll remove it in a follow-up to avoid bringing extra files
into this CL.

After this CL hits production, the next one will replace all calls to
generateLegacyToken() with generateToken().  Once that CL is deployed,
the last step will be removing the legacy fallback in validateToken().

[1] See https://en.wikipedia.org/wiki/Length_extension_attack

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148936805
2017-03-07 13:31:46 -05:00
nickfelt
499f1e7dbc Rewrite ServerSecret caching and accessor logic
I'm working on some changes to XsrfTokenManager (b/35388772) and ServerSecret
was crufty enough that I ended up rewriting it.  Now it uses a LoadingCache
with a transaction instead of needlessly race-condition-y static init logic.

It also now supports retrieving its value as either a UUID (the old format
used by XsrfTokenManager) or a byte[].  The latter is more flexible and can
be directly used with HMAC which the new XsrfTokenManager format will employ.

And lastly, I added tests.  In addition, I tested this code on alpha and
verified appropriate operation (XSRF tokens still work from the console and
from regtool; if you remove ServerSecret from datastore and memcache, it
persists a new one).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148931620
2017-03-07 13:30:18 -05:00
nickfelt
c56959b62b Add new DomainResource.getGracePeriodsOfType() method
This adds a new method which will be used in an upcoming CL affecting domain
transfer logic.  It also removes two older methods that are unused (they were
originally going to be used for TLD-specific logic which is now obsolete).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148928965
2017-03-07 13:28:49 -05:00
jart
13249db5cf Set generates_api=1 for Dagger component processor
Context: [] and []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148885749
2017-03-07 13:27:21 -05:00
shikhman
51a613bca6 Fix bugs in KmsConnectionImpl
A few errors emerged when doing an integration test against the actual API. I've updated the unit tests to reflect the correct behavior.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148793856
2017-03-07 13:25:51 -05:00
nickfelt
726e925b4a Refactor a few new XsrfTokenManager methods
Followup to comments on []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148792464
2017-03-07 13:24:21 -05:00
mcilwain
822cbc0494 Add missing newline to default-config.yaml
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148778586
2017-03-07 13:22:52 -05:00
shikhman
ddfe4c8a47 Add KMS parameters in configuration
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148778412
2017-03-07 13:21:23 -05:00
mountford
c7a62e9b98 Add XSRF protection to legacy authentication mechanism
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148689952
2017-03-07 13:18:04 -05:00
ctingue
a5932c0fc3 Drop FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148688861
2017-03-07 13:16:34 -05:00
ctingue
023a76349e Add ROID flag in FixDomainNameserverKeysCommand
There's a deleted domain with bad host resource keys. Since FKIs won't help us here, load a domain directly by key (i.e. by ROID) if we so choose.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148685240
2017-03-07 13:15:06 -05:00
nickfelt
5a00317dcc Exclude SYNTHETIC flag from BillingData BigQuery view
The BillingData view is intended to be the stable, clean source of data for the invoicing pipeline and detail report output.  As such, cutting over to the synthetically created OneTimes is an implementation detail and shouldn't be visible in the output (esp. since that goes straight to the detail reports that registrars receive, and it'd probably just be confusing for them).  I think it's fair to generically exclude the SYNTHETIC flag from showing up here - it will still be in BigQuery on the individual OneTimes if we want to look for it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148682719
2017-03-07 13:13:37 -05:00
mcilwain
64bfbea324 Remove duplicate version of lookupSuperordinateDomain() function
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148657943
2017-03-07 13:12:09 -05:00
mcilwain
90114858fa Perform minor refactors on premium list code
Principally, this moves a load method into DatastoreHelper that is now
only used by tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148649087
2017-02-27 11:30:45 -05:00
mcilwain
8d84397e80 Fix Nomulus build errors
This fixes Java7 generic type inference errors in premium list code
and upgrades the Truth library to v0.32, because we're now using
assert(e).hasMessageThat() which is not in prior versions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148501657
2017-02-27 11:30:45 -05:00
mcilwain
ea4e471c04 Move premium list static helper methods into their own class
It was kind of messy having all of that logic living alongside the
entities themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148498024
2017-02-27 11:30:45 -05:00
shikhman
388dd1055e Abstract KMS code with KmsConnection and create a fake KmsConnection
This simplifies the tests for KmsKeyring and KmsUpdater.

This is a followup to []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496758
2017-02-27 11:30:45 -05:00
Wolfgang Meyers
9f90597691 Handle pending transfers on domain imports from RDE deposits
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496597
2017-02-27 11:30:40 -05:00
mcilwain
3ca9bb6aeb Read from bloom filter for premium pricing checks
This also cleans up the PremiumList API so that it only has one
method for checking premium prices, which is by TLD, rather than two.

I will be refactoring a lot of the static methods currently residing in
the PremiumList class into a separate utils class, but I don't want to
include too many changes in this one CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148475345
2017-02-27 11:22:21 -05:00
mountford
3ac74fa449 Add more tests to new authentication framework
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148459400
2017-02-27 11:19:27 -05:00