Commit graph

190 commits

Author SHA1 Message Date
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
guyben
84fdeebc2f Use specific date for testing UpdateSmdCommand
"The passage of time" caused the test to start failing because the test data
given by ICANN includes certificates that expire on 2017.

Using a fake clock to make sure the "now" date is always in the valid
certificate range solves this issue.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162987171
2017-08-01 16:57:36 -04:00
mcilwain
d3e9ebad16 Remove deprecated singular DNS writer field and update tooling
Note that even though the nomulus command line tool now supports multiple
DNS writers for all subcommands, this still won't work quite yet because
the DNS task queue format migration from [] is still in progress.
After next week's push that migration will be complete and we can remove
the final restriction against only having one DNS writer per TLD.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162490399
2017-08-01 16:50:49 -04:00
bbilbo
9688638c75 Use History Entry type for flows in VerifyOteServlet
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161855429
2017-08-01 16:24:55 -04:00
Ben McIlwain
ea4572b4b5 Add missing @Test, @Before, or @After annotations to JUnit4 test, setUp, or tearDown methods; reduce the visibility of test helper methods to prevent them from being confused for tests; and/or add @Ignore to purposely disabled tests.
This paves the way for [] which expands the set of classes Blaze will check for possible test methods that are not properly annotated.

For more details and FAQs please see: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161795590
2017-08-01 16:18:08 -04:00
mcilwain
37f33e5e7a Migrate plural DNS writers field to being the canonical one
After this point all data is migrated to use the new canonical
plural version, and subsequent code changes can be made that use
multiple writers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161673486
2017-08-01 16:12:42 -04:00
mcilwain
24587491c9 Make re-save environment entities command use batching
This makes it take a lot less time to run (roughly a 10X speedup).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161666391
2017-08-01 16:10:00 -04:00
bbilbo
39c3706321 Move EPP XML testdata to server/testdata
This will allow tests in server to reference the xml files.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161582503
2017-07-12 11:03:50 -04:00
mcilwain
4d5b6845b7 Add plural DNS writers field to Registry entity
This is the first step in a multi-step data migration to allow multiple
DNS writers per TLD. The overall process looks like this:

1. Add a plural DNS writers field with backfill (this commit).
2. Deploy it.
3. Run the ResaveEnvironmentEntitiesCommand to populate this new field
   on all entities.
4. Update the code to use the new field everywhere.
5. Deploy it.
6. Delete the now-unreferenced, old deprecated singular value field.

This process is rollback-safe.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161253436
2017-07-10 11:45:13 -04:00
mcilwain
dccc99787e Add TLDs parameter to refresh DNS action
This will allow us to migrate one TLD at a time by refreshing all zones
on the specified TLD after dual-writing is enabled.

Note that the TLDs parameter is required, which seems reasonable given
that almost all imagined use cases would be on a by-TLD basis.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160294546
2017-07-10 11:23:57 -04:00
bbilbo
bbdf9bfc38 Refactor CreateDomainCommand and add UpdateDomainCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160282921
2017-07-10 11:22:39 -04:00
jianglai
d1ef4b9c37 Allow partial update of billing account map
When doing update_registrar, it is now possible to only specify the currencies and the account ids that need updating.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159262119
2017-06-21 09:48:19 -04:00
mountford
3bbf356497 Fix bug in registrar contact nomulus command
The command was set up such that an update without any contact types specified would clear out the list, instead of leaving them unchanged, as it should.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157766429
2017-06-05 18:17:09 -04:00
mountford
6cc3ee5e15 Add option to registrar_contact command to modify the visible_in_domain_whois_as_abuse flag
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157588734
2017-06-05 18:17:09 -04:00
mmuller
d4f222cecd Fix RestoreCommitLogsCommandTest in kokoro
Convert RestoreCommitLogsCommandTest use of generics and mockito to a form
that works with the kokoro build:

- Replace ImmutableMap<String, Object> with ImmutableMap<String, ?>.
- Replace any() as a matcher for MediaType with an "eq()" matcher.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157148910
2017-06-05 18:17:09 -04:00
mmuller
b733443eab Create nomulus tool restore command
Create a nomulus tool restoreCommitLogs command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157103177
2017-06-05 18:17:09 -04:00
mountford
fb21946366 Fix parameter problem in GenerateEscrowDepositCommand
It was setting the TLD and watermark date parameters as comma-separated lists. What it really needs to do is set the parameters multiple times, once for each value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156741051
2017-05-23 17:22:49 -04:00
jianglai
35624ace08 Require currency specific billingAccountId for registrar
For the currency specific billing account ids to work properly, any REAL registrar that is allowed to register domains in a certain TLD must have a billing account id for the currency of that TLD.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864711
2017-05-17 12:09:46 -04:00
jianglai
6e6a340113 Allow multiple domain creation in nomulus tool
This helps creating test domains. Also fixed a bunch of bad test cases.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864065
2017-05-17 12:08:10 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglai
7a3852b5a5 Add visibleInWhoisAsAbuse field in RegistrarContact
Also added corresponding getters and setters for the new field. Note that
nothing has changed on the RDAP front for now, as the CL&D only concerns WHOIS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155116134
2017-05-17 11:27:24 -04:00
mountford
8bb66dd365 Add delete_host nomulus tool command
This is needed to delete an invalid host resulting from a previous bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154757425
2017-05-03 11:08:02 -04:00
mcilwain
e6249196af Fix formatting of DeleteDomainCommand
It was pretty off for some reason, so I just ran
google-java-format on it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154756158
2017-05-03 11:06:19 -04:00
mcilwain
d30f9411d8 Require that DNS writer be set on Registry entities
We ran into a bunch of prober deployment issues this past week when
attempting to spin up a new cluster because the newly created prober
TLDs had null values for the dnsWriter field. Given that VoidDnsWriter
exists, we can require that dnsWriter always be set, and have people
use that if DNS publishing is not required.

Also cleans up a bunch of related inconsistent exception messages and
tests not verifying said exception messages properly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154325830
2017-05-03 10:38:49 -04:00
mountford
a259b42b26 Remove some unnecessary uses of MockitoJUnitRunner, which is discouraged
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153711358
2017-04-26 10:45:45 -04:00
mcilwain
1751ea0b98 Replace deprecated Truth hasMessage() method with hasMessageThat()
hasMessageThat() was added in Truth 0.32 and we are already on that
version in the Nomulus release.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153095111
2017-04-26 10:35:23 -04:00
guyben
ab515cb352 Replace KeystoreKeyring with KmsKeystore comparison
Replace KeystoreKeyring with ComparatorKeyring between KeystoreKeyring and
KmsKeystore. In the opensource version, will replace DummyKeyring with
KmsKeyring directly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152893767
2017-04-13 10:58:57 -04:00
mountford
dea386d08a Change GenerateEscrowDepositCommand to trigger back end deposit generation
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
2017-04-13 10:57:24 -04:00
mcilwain
8653d2f204 Refactor out creation of server TRIDs so they can be tested
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152837185
2017-04-13 10:55:47 -04:00
nickfelt
5081d780dc Add assertTldsExist(Iterable<String>) to check multiple TLDs at once
This is better than calling assertTldExists() inside a for loop because you can throw a single exception reporting all bad TLDs at once rather than only getting as far as the first failure.  And then it's also a one-liner instead of 3 lines.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152412876
2017-04-10 13:44:52 -04:00
mountford
62c7a3935a Add nomulus tool command to delete a TLD
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151863632
2017-04-05 10:10:38 -04:00
mmuller
5ffb424682 Add a "loadtest" command to nomulus tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151034721
2017-03-27 13:16:35 -04:00
jianglai
d7e2009ddf Add billing account map to Registrar entity
A CurrencyUnit-to-BillingAccountEntry map is persisted in the Registrar entity. It provides flexibility for billing systems that assign different account ids for accounts under different currencies of the same registrar.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151022753
2017-03-27 13:10:29 -04:00
jianglai
a958d0a6c3 Add a Nomulus command to set domain create restricted status
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150665065
2017-03-21 15:43:52 -04:00
shikhman
5779cc988b Refactor RdeKeyringModule into FakeKeyringModule
A fake Keyring implementation is needed to test Keyring related tools now, so
I've refactored the fake RdeKeyring to support all Keyring API methods and
moved it to the common testing package.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149977968
2017-03-21 14:53:25 -04:00
cgoldfeder
9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
mcilwain
ad840ad4d6 Better handle syncing contacts when creating RegistrarContacts
This also adds a test that should've been there already.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149583500
2017-03-13 11:22:55 -04:00
shikhman
9eddbe2b6e Add test which verifies that all commands have descriptions
Besides being good practice, this works around a bug in JCommander 1.48, which we are stuck with for now
since newer versions of the library are Java 8 only:
https://github.com/cbeust/jcommander/issues/270

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149565357
2017-03-13 10:44:12 -04:00
mcilwain
ce4f3c0d56 Don't allow setting reserved lists with conflicting auth codes
This is an error condition that will soon throw an exception when
attempting to register the domain name, so it's good to let the registry
operator know of the error when it is first introduced.

Unfortunately there's still a backdoor that allows duplicate labels
that's harder to protect against (that this commit doesn't cover): the
case where reserved lists are already applied to a TLD, then one of the
reserved lists is updated to add another auth code, which then conflicts
with one on a different reserved list.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149443007
2017-03-13 10:27:51 -04:00
Ben McIlwain
2bba5660c2 Use Immutable types for static final fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149172966
2017-03-07 13:50:53 -05:00
mmuller
5614760d53 Implement login/logout commands
Refactor the auth code into its own dagger module, add tests and use the new interfaces to implement the login and logout commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149108266
2017-03-07 13:45:11 -05:00
Ben McIlwain
80f0910899 Use Immutable types for static final fields
Replace constants (static final CONSTANT_CASE) declaration type which use the general collection interface (e.g. List) with an immutable type (e.g. ImmutableList).

For constant field declarations, you should use the immutable type (such as ImmutableList) instead of the general collection interface type (such as List). This communicates to your callers important semantic guarantees ([]

For more info, see: []

Cleanup change automatically generated by javacflume/refactory
Refactoring: //third_party/java_src/error_prone/project/core/src/main/java/com/google/errorprone/bugpatterns:MutableConstantField_refactoring
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149061247
2017-03-07 13:43:43 -05:00
mcilwain
90114858fa Perform minor refactors on premium list code
Principally, this moves a load method into DatastoreHelper that is now
only used by tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148649087
2017-02-27 11:30:45 -05:00
mcilwain
ea4e471c04 Move premium list static helper methods into their own class
It was kind of messy having all of that logic living alongside the
entities themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148498024
2017-02-27 11:30:45 -05:00
mcilwain
3ca9bb6aeb Read from bloom filter for premium pricing checks
This also cleans up the PremiumList API so that it only has one
method for checking premium prices, which is by TLD, rather than two.

I will be refactoring a lot of the static methods currently residing in
the PremiumList class into a separate utils class, but I don't want to
include too many changes in this one CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148475345
2017-02-27 11:22:21 -05:00
nickfelt
16832323d0 Make ListObjectsAction return 200 when sending JSON error
This fixes a bug in the interaction between ListObjectsAction and ListObjectsCommand/AppEngineConnection.  ListObjectsAction was returning HTTP status code 400 when it caught an IAE, but also attempting to return a JSON response payload of {"status": "error", "error": "<exception message>"}.  However, AppEngineConnection treats any HTTP error response as more like a crash on the server side - it attempts to scrape the error message out of the autogenerated HTML that AppEngine produces for uncaught exceptions, and throws an exception, killing ListObjectsCommand before it can extract the JSON which contains the nicer error (that stating the missing field, etc versus just "400 Bad Request").

The fix is just to have ListObjectsAction return a 200 and the error message so that ListObjectsCommand can correctly handle it.

I also de-scoped the catch to only catching IAE, since catching Exception was overbroad, and the only "expected" exception to be thrown is an IAE from the checkArgument() that tests if the requested fields all exist.  Any other kinds of exceptions should actually just bubble up and kill the action, and get the regular AppEngineConnection error treatment.

I also added "billingId" as an alias for "billingIdentifier", parallel to clientId/clientIdentifier, since that's why I came across this issue in the first place.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148248834
2017-02-27 11:10:37 -05:00
mmuller
68bac57da5 Store credentials under scope-qualified name
Store the auth credentials under a name qualified by the set of OAuth scopes
as well as the client id.  This is implemented as the base64 encoded SHA1 hash
of the concatenation of client id and sorted auth scopes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148127911
2017-02-22 11:47:37 -05:00
ctingue
b3b4bba9aa Align parameter formats for zone data commands
Seems silly that one command uses --tlds for the required parameter, while the other one doesn't.

As part of this change, create a DateParameter for commands that require only a date (i.e. a DateTime parameter restricted to midnight UTC).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148106721
2017-02-22 11:46:07 -05:00
ctingue
4a92d97a70 Filter domains w/disallowed statuses from zone file generation MR
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147839827
2017-02-17 12:26:29 -05:00
mmuller
177bf4a5f1 Add Client-side OAuth2 to HTTP connections
Implement client-side OAuth in non-local HTTP connections.  Also add tests to
verify that the different modes of connection are set up correctly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147636222
2017-02-17 12:17:53 -05:00