Commit graph

237 commits

Author SHA1 Message Date
guyben
ab515cb352 Replace KeystoreKeyring with KmsKeystore comparison
Replace KeystoreKeyring with ComparatorKeyring between KeystoreKeyring and
KmsKeystore. In the opensource version, will replace DummyKeyring with
KmsKeyring directly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152893767
2017-04-13 10:58:57 -04:00
mountford
dea386d08a Change GenerateEscrowDepositCommand to trigger back end deposit generation
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
2017-04-13 10:57:24 -04:00
mountford
4f94464eaf Allow RdeStagingAction to be invoked manually
RdeStagingAction always processed all RDE and BRDA deposits currently outstanding, updating the cursors appropriately and kicking off the upload job. Sometimes we don't want all that. We just want to create a specific deposit by hand, without modifying the cursors or uploading. This CL adds parameters to support that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152415959
2017-04-10 13:47:51 -04:00
nickfelt
5081d780dc Add assertTldsExist(Iterable<String>) to check multiple TLDs at once
This is better than calling assertTldExists() inside a for loop because you can throw a single exception reporting all bad TLDs at once rather than only getting as far as the first failure.  And then it's also a one-liner instead of 3 lines.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152412876
2017-04-10 13:44:52 -04:00
mountford
5127aeafb5 Enable authentication/authorization checks
The code to authenticate and authorize incoming requests (including via OAuth) has been in the system. This CL actually turns it on, since we are satisfied from logging information that it is not unjustly denying access.

Auth settings are also updated on a few commands missed earlier.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152381820
2017-04-10 13:41:51 -04:00
guyben
47870f98eb Add UpdateKmsKeyringCommand to update keys in the KMS Keyring
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152253868
2017-04-10 13:26:43 -04:00
nickfelt
ef3977e837 Remove unused injected ClientScopeQualifier
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152253768
2017-04-10 13:25:11 -04:00
guyben
c3df4e26a3 Add GetKeyringSecretCommand to retrieve keyring secrets.
Cloned from CL 149476124 by 'g4 patch'.
Original change by shikhman@shikhman:registry-secrets-2:897:citc on 2017/03/07 15:37:09.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151950797
2017-04-05 10:13:45 -04:00
mountford
62c7a3935a Add nomulus tool command to delete a TLD
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151863632
2017-04-05 10:10:38 -04:00
jianglai
ff9c72097c Add Registrar Account Data view in BigQuery snapshot
This adds a new view table that contains the registrar id, the currency-specific billing account id and the corresponding currency in latest_billing dataset based on latest_snapshot dataset.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151363209
2017-04-05 10:06:04 -04:00
jianglai
190be064cb Add billingAccountId to BillingData table
Join RegistrarAccountId table to BillingData to append additional billingAccountId column

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151362466
2017-04-05 10:04:32 -04:00
nickfelt
0d32b6b7b2 Remove MakeBillingTablesCommand credit data views
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151130847
2017-03-27 13:28:33 -04:00
mmuller
5ffb424682 Add a "loadtest" command to nomulus tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151034721
2017-03-27 13:16:35 -04:00
nickfelt
aa35d74900 Remove UpdateCreditsCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151023241
2017-03-27 13:12:00 -04:00
jianglai
d7e2009ddf Add billing account map to Registrar entity
A CurrencyUnit-to-BillingAccountEntry map is persisted in the Registrar entity. It provides flexibility for billing systems that assign different account ids for accounts under different currencies of the same registrar.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151022753
2017-03-27 13:10:29 -04:00
nickfelt
ab9b7c613d Remove now-unused RegistryData SQL columns
These columns were only necessary to support the old PremiumListData and
RecurringEventData views, which were removed in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151019824
2017-03-27 13:08:55 -04:00
nickfelt
dfce7608e7 Remove billing data query Registry 1.0 vestiges
These were a few straggling special cases to deal with Registry 1.0 data;
that data was removed ages ago and I just never got around to cleaning these
up.  This removes them at long last.

There were also some TODOs for the same bug around better TLD extraction that
supports multi-part TLDs (though that's pretty unrelated to Registry 1.0/2.0),
so I fixed those since it turns out supporting multi-part TLDs is trivial.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151003181
2017-03-27 13:05:51 -04:00
jianglai
a958d0a6c3 Add a Nomulus command to set domain create restricted status
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150665065
2017-03-21 15:43:52 -04:00
nickfelt
2353bcd8c5 Cut over to generating new HMAC-based XSRF tokens
This is the second step of migrating to our new XSRF token format.  The
first step ([] made validate() start accepting new tokens
(basically, dual-read).  This step cuts over our "writing" to write the
new token format.  The third and final step will drop support for
validating the old token format (back to single-read).  We'll do that
in a subsequent push so that we don't invalidate all the current XSRF
tokens that people might have in their browsers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149790648
2017-03-13 11:22:56 -04:00
cgoldfeder
9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
mountford
1f000b94e6 Set correct auth settings for all actions
A test has been added to RequestHandlerTest, making sure that, while we merely log errors for the time being, the correct dummy AuthResult is being created.

Most actions use the default settings, which have been changed to INTERNAL / APP / IGNORED. Actions with non-default settings are:

INTERNAL/NONE/PUBLIC (non-auth public endpoints)

CheckApiAction
WhoisHttpServer
Rdap*Action

INTERNAL,API/APP/ADMIN (things currently protected by web.xml)

EppTlsAction
EppToolAction
CreateGroupsAction
CreatePremiumListAction
DeleteEntityAction
List*sAction
UpdatePremiumListAction
VerifyOteAction
WhoisServer

INTERNAL,API,LEGACY/USER/PUBLIC (registrar console)

RegistrarPaymentAction
RegistrarPaymentSetupAction
RegistrarSettingsAction
EppConsoleAction

INTERNAL,API,LEGACY/NONE/PUBLIC (registrar console main page)

ConsoleUiAction

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149761652
2017-03-13 11:22:55 -04:00
mcilwain
3b2a014524 Don't log premium list nomulus tool usage errors as SEVERE
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149661618
2017-03-13 11:22:55 -04:00
mcilwain
ad840ad4d6 Better handle syncing contacts when creating RegistrarContacts
This also adds a test that should've been there already.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149583500
2017-03-13 11:22:55 -04:00
shikhman
1dfb6afa4b Add command description to get_resource_by_key command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149561503
2017-03-13 10:41:15 -04:00
shikhman
b05afd2153 Fix nomulus tool NPEs on login/logout commands when printing usage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149535358
2017-03-13 10:35:23 -04:00
mmuller
b6f3d60e0f Refactor command running around auth exceptions
Refactor command and component code in RegistryCli so that we can handle a
LoginRequiredException from whereever we are likely to ever get one.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149329171
2017-03-07 13:59:40 -05:00
mmuller
5614760d53 Implement login/logout commands
Refactor the auth code into its own dagger module, add tests and use the new interfaces to implement the login and logout commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149108266
2017-03-07 13:45:11 -05:00
Ben McIlwain
fa5607c3e1 Use Immutable types for static final fields
For constant field declarations, you should use the immutable type (such as ImmutableList) instead of the general collection interface type (such as List). This communicates to your callers important semantic guarantees ([]

This change replaces constants (static final CONSTNAT_CASE) declaration type which use the general collection interface (e.g. List) with an immutable type (e.g. ImmutableList).

For more info, see: []

Cleanup change automatically generated by javacflume/refactory
Refactoring: //third_party/java_src/error_prone/project/core/src/main/java/com/google/errorprone/bugpatterns:MutableConstantField_refactoring
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149010021
2017-03-07 13:34:48 -05:00
nickfelt
2e969d6ed1 Change XsrfTokenManager to support new HMAC token format
This follows up on Brian's work to transition not just to a new format
with an empty scope value, but instead to replace the existing format
entirely with a new one that:

  1) includes a version number to support future format migrations
  2) doesn't include a field for the scope at all, since scoping the
     tokens adds no real security benefit and just makes verification
     more difficult
  3) replaces the raw SHA-256 hash with a SHA-256 HMAC instead, as a
     best practice to avoid length-extension attacks [1], even though
     in our particular case they would only be able to extend the
     timestamp and would thus be relatively innocuous

The new format will be produced by calling generateToken(), and the
scope-accepting version is renamed to generateLegacyToken() in addition
to its existing deprecation, for maximum clarity.

I changed the validateToken() logic to stop accepting a scope entirely;
when validating a legacy-style token, we'll test it against the two
existing legacy scope values ("admin" and "console") and accept it if
it matches either one.

Note that this means the xsrfScope parameter in @Action is now wholly
obsolete; I'll remove it in a follow-up to avoid bringing extra files
into this CL.

After this CL hits production, the next one will replace all calls to
generateLegacyToken() with generateToken().  Once that CL is deployed,
the last step will be removing the legacy fallback in validateToken().

[1] See https://en.wikipedia.org/wiki/Length_extension_attack

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148936805
2017-03-07 13:31:46 -05:00
ctingue
a5932c0fc3 Drop FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148688861
2017-03-07 13:16:34 -05:00
ctingue
023a76349e Add ROID flag in FixDomainNameserverKeysCommand
There's a deleted domain with bad host resource keys. Since FKIs won't help us here, load a domain directly by key (i.e. by ROID) if we so choose.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148685240
2017-03-07 13:15:06 -05:00
nickfelt
5a00317dcc Exclude SYNTHETIC flag from BillingData BigQuery view
The BillingData view is intended to be the stable, clean source of data for the invoicing pipeline and detail report output.  As such, cutting over to the synthetically created OneTimes is an implementation detail and shouldn't be visible in the output (esp. since that goes straight to the detail reports that registrars receive, and it'd probably just be confusing for them).  I think it's fair to generically exclude the SYNTHETIC flag from showing up here - it will still be in BigQuery on the individual OneTimes if we want to look for it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148682719
2017-03-07 13:13:37 -05:00
mcilwain
ea4e471c04 Move premium list static helper methods into their own class
It was kind of messy having all of that logic living alongside the
entities themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148498024
2017-02-27 11:30:45 -05:00
mcilwain
3ca9bb6aeb Read from bloom filter for premium pricing checks
This also cleans up the PremiumList API so that it only has one
method for checking premium prices, which is by TLD, rather than two.

I will be refactoring a lot of the static methods currently residing in
the PremiumList class into a separate utils class, but I don't want to
include too many changes in this one CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148475345
2017-02-27 11:22:21 -05:00
nickfelt
16832323d0 Make ListObjectsAction return 200 when sending JSON error
This fixes a bug in the interaction between ListObjectsAction and ListObjectsCommand/AppEngineConnection.  ListObjectsAction was returning HTTP status code 400 when it caught an IAE, but also attempting to return a JSON response payload of {"status": "error", "error": "<exception message>"}.  However, AppEngineConnection treats any HTTP error response as more like a crash on the server side - it attempts to scrape the error message out of the autogenerated HTML that AppEngine produces for uncaught exceptions, and throws an exception, killing ListObjectsCommand before it can extract the JSON which contains the nicer error (that stating the missing field, etc versus just "400 Bad Request").

The fix is just to have ListObjectsAction return a 200 and the error message so that ListObjectsCommand can correctly handle it.

I also de-scoped the catch to only catching IAE, since catching Exception was overbroad, and the only "expected" exception to be thrown is an IAE from the checkArgument() that tests if the requested fields all exist.  Any other kinds of exceptions should actually just bubble up and kill the action, and get the regular AppEngineConnection error treatment.

I also added "billingId" as an alias for "billingIdentifier", parallel to clientId/clientIdentifier, since that's why I came across this issue in the first place.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148248834
2017-02-27 11:10:37 -05:00
mmuller
acc7d19106 Put the path to client_secrets in a config option.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148214513
2017-02-22 11:49:03 -05:00
mmuller
68bac57da5 Store credentials under scope-qualified name
Store the auth credentials under a name qualified by the set of OAuth scopes
as well as the client id.  This is implemented as the base64 encoded SHA1 hash
of the concatenation of client id and sorted auth scopes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148127911
2017-02-22 11:47:37 -05:00
ctingue
b3b4bba9aa Align parameter formats for zone data commands
Seems silly that one command uses --tlds for the required parameter, while the other one doesn't.

As part of this change, create a DateParameter for commands that require only a date (i.e. a DateTime parameter restricted to midnight UTC).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148106721
2017-02-22 11:46:07 -05:00
mountford
0417f3d3a1 Daggerize XsrfTokenManager
The one-day validity period is also moved from the caller into XsrfTokenManager.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147857716
2017-02-22 11:43:10 -05:00
ctingue
4a92d97a70 Filter domains w/disallowed statuses from zone file generation MR
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147839827
2017-02-17 12:26:29 -05:00
mmuller
177bf4a5f1 Add Client-side OAuth2 to HTTP connections
Implement client-side OAuth in non-local HTTP connections.  Also add tests to
verify that the different modes of connection are set up correctly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147636222
2017-02-17 12:17:53 -05:00
mcilwain
32b236e940 Delete the unsafe DeleteEppResourceCommand
It was somewhat unsafe to run because it bypassed some of the checks
that are usually run when attempting to delete EPP resources. The
DeleteDomainCommand is a recommended replacement that uses SOY templates
to delete a domain using EPP. Similar commands to delete hosts and
contacts can be written if required.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147634146
2017-02-17 12:16:27 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
ctingue
e0cb4e465c Fix resource mismatch problem in FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147476075
2017-02-14 12:16:55 -05:00
ctingue
07625ce81b Add history entry to FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147468517
2017-02-14 12:15:29 -05:00
ctingue
335b3f0cf8 Scrap tool to fix DomainResource keys to deleted nameservers
There are about 25 active domains that have keys that point to deleted nameservers. In all cases, there are active nameservers with the same FQDN that these domains should be pointing to. Given the domains in question (pulled via BigQuery, see the bug), update the domain with the correct keys whenever a deleted host is found.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147346660
2017-02-14 12:08:16 -05:00
ctingue
3ec6c7d346 Fix a few typos
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147187320
2017-02-14 12:06:47 -05:00
ctingue
cb215adac3 Cut over to synthetic OneTime billing events
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147048115
2017-02-09 14:24:42 -05:00
mcilwain
ec55aa5361 Resolve some Guava 20 TODOs (mostly unnecessary asList() calls)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146799536
2017-02-07 13:27:40 -05:00
nickfelt
2a32f9048e Fix CommandUtilities to fail gracefully if stdin is unavailable
Right now, it just NPEs, which is harder to debug.  Also make it handle end-of-input more cleanly by assuming that means a negative response.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146674937
2017-02-06 16:45:23 -05:00