Commit graph

4278 commits

Author SHA1 Message Date
nickfelt
6b6b8d25c7 Add type to RegistrarData billing BigQuery view
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155682529
2017-05-17 12:00:09 -04:00
guyben
c2094eda40 Make EncryptEscrowDepositCommand a RemoteApiCommand
Moving to KmsKeyring, any command that uses the Keyring must be a
RemoteApiCommand to work.

TESTED: successfully encrypted a file with no logged errors.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155673003
2017-05-17 11:58:35 -04:00
mcilwain
6e3a99a16b Tune max # of frontend instances down to 4 in sandbox and alpha
My continuing investigations into necessary resources for running
our environments seems to indicate that four instances should be
sufficient for our purposes.  If it's not, we can always revert.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155607688
2017-05-17 11:55:30 -04:00
nickfelt
2b78571c72 Replace literal 'latest_snapshot' in invoice SQL query
This should be %SOURCE_DATASET% so that it's adjusted appropriately based on the parameter value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155582718
2017-05-17 11:53:56 -04:00
mmuller
70e3271319 Add "toTime" parameter to RestoreCommitLogsAction web form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155500640
2017-05-17 11:46:18 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglai
5313ca58d6 Setup auth settings in PublishDetailReports @Action to support tool access
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155459413
2017-05-17 11:43:09 -04:00
jart
d419f6ac11 Export annotations from Guava
These annotations are part of guava's API, and they cannot safely be
optimized off the compile-time classpath.

This works around google/error-prone#615.

See []
H/T @cushon

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155433554
2017-05-17 11:40:02 -04:00
jianglai
8892656722 Make WHOIS domain query return conform to ICANN CL&D policy
ICANN's policy can be found here:
https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155375811
2017-05-17 11:38:28 -04:00
jianglai
2846f9c6b9 This CL include changes in the registrar console that makes it possible to designate an abuse contact in domain WHOIS record, per ICANN's CL&D requirement.
Frontend validation: ensures that only one WHOIS abuse contact exist per registrar. Any existing WHOIS abuse contact will be overridden when a new one is designated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155289097
2017-05-17 11:36:53 -04:00
cgoldfeder
275d6ddc10 Disable memcache completely
We've determined that getting correctness semantics right, even
in the few cases that it is possible to do so (see linked bug for
audit) is not worth the bother in terms of highly complicated code
and potential bugs. This CL turns off memcache at the Ofy level
but doesn't rip out the annotations etc. so that we can quickly
turn it back on if this turns out to have been a mistake.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155227761
2017-05-17 11:35:19 -04:00
cgoldfeder
608e121a37 Set the premium cache duration in tests to 60s not 60ms
This was causing test flakiness.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155225717
2017-05-17 11:33:44 -04:00
mmuller
3eef5095d4 Deal with gaps and forks in commit history
If there are gaps or forks in the commit log history files, raise an exception
and display the possible sets of files in the log files.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155219410
2017-05-17 11:30:36 -04:00
jianglai
7a3852b5a5 Add visibleInWhoisAsAbuse field in RegistrarContact
Also added corresponding getters and setters for the new field. Note that
nothing has changed on the RDAP front for now, as the CL&D only concerns WHOIS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155116134
2017-05-17 11:27:24 -04:00
mountford
354e1fb8b2 Change messages about failed authentication to be warnings instead of info messages
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155114145
2017-05-17 11:25:54 -04:00
guyben
93c2a1e4f0 Make JSch injection lazy
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154958750
2017-05-17 11:22:53 -04:00
guyben
5e7834b00e Wrap Kms decrypt with Retrier
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154861871
2017-05-03 11:10:53 -04:00
nickfelt
7f2821cd7e Open up RdeStagingAction @Action settings to support tool access
This change means that RdeStagingAction now accepts POST requests and supports the API auth method, which is needed for the new GenerateEscrowDepositCommand (as of [] to be able to invoke the action in manual mode.  Note that this shouldn't affect the cron setting that invokes it currently, which are GET requests with INTERNAL auth.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154764870
2017-05-03 11:09:29 -04:00
mountford
8bb66dd365 Add delete_host nomulus tool command
This is needed to delete an invalid host resulting from a previous bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154757425
2017-05-03 11:08:02 -04:00
mcilwain
e6249196af Fix formatting of DeleteDomainCommand
It was pretty off for some reason, so I just ran
google-java-format on it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154756158
2017-05-03 11:06:19 -04:00
nickfelt
f640d765e8 Log EppExceptions in EppController at INFO (vs FlowRunner at WARNING)
The logging for exceptions in FlowRunner - always at WARNING - has long been sub-optimal.  For EppExceptions it's too aggressive/spammy to log at WARNING because it's generally not actionable - EppException gets properly thrown for all kinds of ordinary reasons (trying to create a resource when one already exists with that foreign key) and/or for client misbehavior that we can't control (sending bad parameter values, etc.).  For non-EppException RuntimeExceptions, it's redundant with existing logging in EppController.

This CL resolves this by removing that logging in FlowRunner entirely in favor of the EppController logging, where we're now logging EppExceptions at INFO in parallel with the existing logging of RuntimeExceptions at SEVERE.  This has the benefit that we're now logging EppExceptions that come from FlowPicker (by way of EppExceptionInProviderException),  which previously were unlogged.

Note however that this does mean that in places where we run FlowRunner without EppController - exclusively test code as it stands today - we'd no longer be logging EppExceptions.  If that seems like a loss, we could either reinstate logging there (at INFO) and just deal with redundant messages for most EppExceptions, or we could add it manually to places where we call FlowRunner.run() in tests and avoid the redundancy that way.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154733365
2017-05-03 11:04:51 -04:00
mountford
11e7374c0f Fix bug which allowed creation of hosts with superordinate domains in pending delete state.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154716883
2017-05-03 11:03:23 -04:00
cgoldfeder
386d2bc6be Make Registrar.loadByClientId explicitly use memcache
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154712979
2017-05-03 11:01:54 -04:00
nickfelt
f9df4cf3f1 Revert unintentional no-op log handler attachment
This snuck in from [] but was just a vestige of when I was testing with the tests still in FlowRunner test and wanted to make sure they still passed with the logging moved to FlowReporter.  Now that the tests are in FlowReporterTest and FlowReporter is mocked out here, attaching the handler to that logger is a no-op.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154601472
2017-05-03 11:00:23 -04:00
mcilwain
876552f1ce Add more EPP metric lifecycle tests
This also refactors some of the existing EPP XML testdata files
to use a consistent wildcard naming scheme.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154569632
2017-05-03 10:58:51 -04:00
mountford
44546a3480 Throw error in get_history_entries if the specified ID is invalid
Currently, if the ID is invalid, parentKey is set to null, causing it to return all history entries. Note that there is still a problem that you cannot look up history entries for entities which have been soft deleted, because the foreign key lookup won't work. That is unfortunate, but at least this simple fix makes things better than they were.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154564262
2017-05-03 10:57:20 -04:00
mcilwain
2569d62b0a Reduce maximum number of service instances for sandbox and alpha
We want to lower the maximum number of service instances as much as
possible without affecting service reliability so that we can make
stronger statements about what the maximum cost of running a typical
Nomulus environment might be. This first step likely won't affect the
frontend and tools modules in practice because they aren't typically
running even this many instances, but it will clamp down on the
number of backend instances (which should be fine; it just means the
mapreduces will take longer).

Alpha is tuned down the same as sandbox for consistency reasons.
This also standardizes on the B4 size (which has 512 MiB RAM) for
all instances. Most instances were already using this, and the
deviations from it were seemingly at random. Crucially, backend,
which is likely most sensitive to this because it uses the mapreduce
library, is already on the smaller memory size.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154537995
2017-05-03 10:55:49 -04:00
larryruili
62c05c112d Make TRID field in async host/contact deletion non-optional
This cleans up a TODO introduced in the original bug, which allowed tasks to
have optional Trids in the case it was an older enqueued task.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154477042
2017-05-03 10:54:20 -04:00
cgoldfeder
d4d02f8977 Add a test for Registrar.loadByClientId not being enrolled in a transaction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154470388
2017-05-03 10:52:46 -04:00
mcilwain
8025c7547b Add flow-level EPP metric tests
This also adds an assertion helper for EPP metrics.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154458134
2017-05-03 10:51:12 -04:00
cgoldfeder
f1129ea2b1 Make DomainInfoFlow (and application info) explicitly hit memcache
TESTED=For all tests, I added @Cache to DomainBase because otherwise the tests will
    fail. We aren't ready to do this in prod yet, which is why the tests have a TODO
    in them. The new tests fail if you change line 134 in Ofy to not use memcache
    and either use the unchanged original flow code, or use the new
    inlined code and change loadWithMemcache() to load(). They pass with the new
    inlined code that calls loadWithMemcache(), as long as the @Cache is added to
    DomainResource.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154457655
2017-05-03 10:49:38 -04:00
nickfelt
e96b999a83 Fix incorrect header name in TlsCredentials javadoc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154457385
2017-05-03 10:48:01 -04:00
cgoldfeder
0267214841 Replace some unnecessary uses of "real" ofy() with Ofy.
Only Ofy itself and its two helpers (AugmentedSaver and
AugmentedDeleter) need to use the real ofy(). All other
callers should be using Ofy. Fixing this even though it
doesn't change anything because I found it baffling to
follow the code while trying to make a small change.

Update: added a presubmit to enforce this.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154456603
2017-05-03 10:46:28 -04:00
cgoldfeder
927eb43cbc Un-Ignore a test in DomainApplicationCreateFlowTest
A little injection-foo makes this test possible to run.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154442134
2017-05-03 10:44:56 -04:00
mmuller
8514f734e4 Document procedural changes to restore
- Document how to bring up an environment without starting cron.
- Fix the javascript to be copied into the console to ensure that the user
  edits the parameters.
- Link the production build step back to the procedures document.
- Fix a few textual issues.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154419128
2017-05-03 10:43:25 -04:00
nickfelt
924e5e3e3a Clean up one usage of ForeignKeyIndex.mapToFkiClass()
Followup to []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154361061
2017-05-03 10:41:53 -04:00
nickfelt
40fa9ff022 Fix some direct uses of ForeignKey(C/H/D)Index
Mostly these are calls to ForeignKeyIndex.create() (a static method) via subclasses, which is pretty misleading in this case since the type of the return value has nothing to do with the subclass you're qualifying the static method call with (the returned type depends only on the type of the EppResource parameter).

Note however though that while the style guide indeed prohibits qualifying static member references with things other than the class name, the subclassing case is apparently not considered subject to that prohibition in general:
https://groups.google.com[]d/msg/java-style/8ViX-Rh2_sc/48n2lz5nAAAJ

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154331605
2017-05-03 10:40:21 -04:00
mcilwain
d30f9411d8 Require that DNS writer be set on Registry entities
We ran into a bunch of prober deployment issues this past week when
attempting to spin up a new cluster because the newly created prober
TLDs had null values for the dnsWriter field. Given that VoidDnsWriter
exists, we can require that dnsWriter always be set, and have people
use that if DNS publishing is not required.

Also cleans up a bunch of related inconsistent exception messages and
tests not verifying said exception messages properly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154325830
2017-05-03 10:38:49 -04:00
cgoldfeder
d7da112c19 Put back accidentally deleted assertion
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154300634
2017-05-03 10:35:44 -04:00
cgoldfeder
3f9501e469 Remove obsolete comment.
init() is from pre-flattening days

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154256803
2017-04-26 11:21:35 -04:00
mountford
e00b0440b2 Roll back accidental inclusion of non-public Mockito helper methods
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154232020
2017-04-26 11:19:59 -04:00
cgoldfeder
4eba2ea07a Make Registry.get(tld) hit memcache explicitly.
TESTED=The test fails if you change line 134 in Ofy to not use memcache
    and use the unchanged original Registry.get() code. This is the
    expected behavior.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154226534
2017-04-26 11:18:02 -04:00
cgoldfeder
9e61f1d6ef Make failfastForCreate for domain and application creates explicitly hit memcache
TESTED=For all tests, I added @Cache to DomainBase because otherwise the tests will
    fail. We aren't ready to do this in prod yet, which is why the tests are still
    marked @Ignore. The new tests fail if you change line 134 in Ofy to not use memcache
    and either use the unchanged original DomainCreateFlow code, or use the new
    inlined code and change loadWithMemcache() to load(). They pass with the new
    inlined code that calls loadWithMemcache(), as long as the @Cache is added to
    DomainResource.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154224748
2017-04-26 11:16:33 -04:00
mountford
cb145e0721 Remove some unnecessary uses of MockitoJUnitRunner, which is discouraged
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154202151
2017-04-26 11:15:07 -04:00
cgoldfeder
51cfc12c1d Add MemcacheHelper to simplify memcache usage in tests
And make two callsites use it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154189438
2017-04-26 11:12:17 -04:00
cgoldfeder
4553422ade Move domainCreateFailfast before cloneAndLinkReferences
This avoids loading all contacts and hosts before the failfast runs

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154179321
2017-04-26 11:09:23 -04:00
jianglai
33fabfff04 Remove abuse contact requirement
The requirement will instead be enforced on registrar contact that is visible in WHOIS as abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154102111
2017-04-26 11:07:48 -04:00
cgoldfeder
b6e50f0467 Turn on memcache in flow tests.
No tests break because of this, but some tests that were previously
intentionally clearing memcache (for example, the test in
DomainInfoFlow that counts datastore calls) are now doing something
that is not a no-op anymore.

Also fix an incorrect comment about memcache in DatastoreHelper.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154087182
2017-04-26 11:04:56 -04:00
mcilwain
b15d715dc0 Fix two more Java 7 missing generic type errors
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154086209
2017-04-26 11:03:29 -04:00
mcilwain
e19386779c Fix Java 7 missing generic type error
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154081069
2017-04-26 11:02:02 -04:00