Commit graph

65 commits

Author SHA1 Message Date
mcilwain
43ed2cd7b3 Clean up annotation imports in BUILD files
This affects JSR305, JSR330, and Guava annotations.

The exact command run to generate this CL was:

build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
2018-06-27 15:28:53 -04:00
mcilwain
4c7bc3b18c Improve internal build system speed
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202016862
2018-06-27 15:28:52 -04:00
mcilwain
481790bc91 Always display the registrar abuse contact fields in WHOIS
ICANN's Consistent Labeling & Display policy requires that these fields be
present for all domains. We're currently tracking down the last of the data on
our registrars so that we can display these, but if we don't have the data we
should still at least be displaying the field names to make it more clear that
we do support the field, we just don't have the data for this domain yet.

The two fields in question are:

Registrar Abuse Contact Email:
Registrar Abuse Contact Phone:

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201730290
2018-06-27 15:28:52 -04:00
mcilwain
8b263baefa Delete MultiplyingCloudDnsWriter
Now that the large zone re-signing test is complete, we no longer need it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
2018-06-27 15:28:06 -04:00
jianglai
643b30d31f Redact WHOIS output instead of removing the fields altogether
See https://www.icann.org/resources/pages/gtld-registration-data-specs-en/#appendixA for details on how certain fields are redacted.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199295355
2018-06-06 15:14:38 -04:00
jianglai
70b13596e4 Migrate to Flogger (green)
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.

No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.

Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.

For more information, see []
Base CL: 197826149

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
2018-05-30 12:18:54 -04:00
jianglai
4be4fb0082 Migrate to Flogger (yellow)
This is a 'yellow' Flogger migration CL. Yellow CLs should be mostly safe
but include changes that are notable for one reason or another. Manual
intervention may be required to address small issues.

The comments in this CL indicate cases where suggested code changes
should be double checked, or even modified. There may even be cases where
files outside this CL are affected by changes to things such as logger
visibility. However if a change does not have an associated comment then
it should be safe.

For more information, see []
Base CL: 197826149

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097990
2018-05-30 12:18:54 -04:00
jianglai
fc60890136 Migrate to internal FormattingLogger in preparation of migration to Flogger
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197744904
2018-05-30 12:18:54 -04:00
jianglai
86dd6bd59e Add ability to show full WHOIS output in nomulus command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197731992
2018-05-30 12:18:54 -04:00
jianglai
7388958df7 Do not escape WHOIS output
Both WhoisAction and WhoisHttpAction set the HTTP response content type to "text/plain". There is no need to defensively escape the content. In fact, by escaping the content, it creates more problems down the line.

When used in a website, the response should be written into a DOM node by setting the textContent of the node, which automatically escapes the content.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196743398
2018-05-17 21:52:35 -04:00
larryruili
4657b8ab51 Replace 'referralUrl' with 'url' in WHOIS responses
The 'referralUrl' Datastore field is filled with mostly junk data, whereas
'url' contains real registrar web addresses. This makes the long needed fix to
display the proper url in WHOIS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194398003
2018-05-05 23:32:40 -04:00
jianglai
f289259101 Change UserPolicy to PUBLIC on WHOIS and EPP endpoints
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193407195
2018-04-23 14:59:24 -04:00
mcilwain
07d38340f3 Cache domains, contacts, and hosts in WHOIS queries
This should prevent having issues with hot key paths on entities that
experience a heavy WHOIS volume (e.g. contacts that registrars reuse on
many domains).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191506124
2018-04-10 16:24:03 -04:00
jianglai
839e4aa682 Remove some fields from WHOIS output
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191445626
2018-04-10 16:17:42 -04:00
jianglai
c40eda3235 Rename WhoisServer and WhoisHttpServer to actions
These servlets are converted to actions during daggerization. Calling them servers are misleading.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190942237
2018-04-02 16:52:14 -04:00
mcilwain
81dc2bbbc3 Rationalize logging statements across codebase
This fixes up the following problems:
1. Using string concatenation instead of the formatting variant methods.
2. Logging or swallowing exception messages without logging the exception
   itself (this swallows the stack trace).
3. Unnecessary logging on re-thrown exceptions.
4. Unnecessary use of formatting variant methods when not necessary.
5. Complicated logging statements involving significant processing not being
   wrapped inside of a logging level check.
6. Redundant logging both of an exception itself and its message (this is
   unnecessary duplication).
7. Use of the base Logger class instead of our FormattingLogger class.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182419837
2018-01-19 14:56:45 -05:00
jianglai
07622725bf Move metrics dependencies to artifacts under Maven groupId com.google.monitoring-client
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=180580386
2018-01-04 17:12:35 -05:00
guyben
8157928a35 Replace com.google.common.base.Function with java.util.function.Function
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179249159
2017-12-27 11:08:55 -05:00
jianglai
1c1f95992a Move backported JUnit file to third_party (part 2)
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
2017-12-02 11:37:46 -05:00
mcilwain
e2db3f914e Clean up some code quality issues
This removes some qualifiers that aren't necessary (e.g. public/abstract on interfaces, private on enum constructors, final on private methods, static on nested interfaces/enums), uses Java 8 lambdas and features where that's an improvement

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177182945
2017-12-01 22:14:06 -05:00
mcilwain
bbe2584da4 Refactor Guava functional methods to use lambdas
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177027488
2017-12-01 22:14:05 -05:00
mcilwain
2aa897e698 Remove unnecessary generic type arguments
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175155365
2017-11-21 18:17:31 -05:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
mcilwain
5edb7935ed Run automatic Java 8 conversion over codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171174380
2017-10-10 12:09:41 -04:00
mcilwain
1c4e79f99e Don't allow non-active registrars to create domains or applications
Specifically, this prevents suspended registrars from creating domains or applications. Pending registrars already can't perform these actions because they get an error message when attempting to log in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170481338
2017-10-04 16:16:45 -04:00
bbilbo
263aea3b2a Add logging to WhoisReader
Logging will make it easier to troubleshoot WhoisServer exceptions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167596453
2017-09-12 15:51:50 -04:00
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
guyben
e224a67eda Change @Auth to an AutoValue, and created a set of predefined Auths
We want to be safer and more explicit about the authentication needed by the many actions that exist.

As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
2017-08-01 16:33:10 -04:00
mountford
e8eabe01cf Update WHOIS output format to reflect CL&D and PDT requirements
PDT testing revealed a couple ways in which our WHOIS output was non-compliant. First, the Consistent Labeling & Display policy dictates that the contact IDs must be ROIDs. See rule 11 in https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en. Second, PDT tests expect that a WHOIS response will treat missing values either by omitting the line entirely, or by including the line with a blank value, but not both. So this is legal:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568
Fax Number Ext:

and this is legal:

Phone Number: 123-4567
Fax Number: 123-4568

but this is not:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568

In the last example, one extension line is present with a blank value, while the other extension line is omitted. We cannot do both. Therefore, we should update our code to omit lines with no value. Since we can't guarantee that we will always emit all lines that the parse might expect to see, it is safe to use the policy of omitting lines with no value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158184150
2017-06-14 10:29:55 -04:00
jianglai
3f5735ed4b Add Carriage Return in WHOIS disclaimer text
WHOIS disclaimer is read from the config yaml file directly and the line breaks
within the text does not contain carriage return, which ICANN requires. This
CL fixes the non-compliance.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156359526
2017-05-23 17:22:49 -04:00
cgoldfeder
c9d7e75946 Cache Registrars in memory
This replaces the memcache caching, which we think is overall a bad idea.
We load all registrars at once instead of caching each as needed, so that
the loadAllCached() methods can be cached as well, and therefore will
always produce results consistent with loadByClientIdCached()'s view of the
registrar's values. All of our prod registrars together total 300k of data
right now, so this is hardly worth optimizing further, and in any case this
will likely reduce latency even further since most requests will be
served out of memory.

While I was in the Registrar file I standardized the error messages for incorrect
password and clientId length to be the same format, and cleaned up a few
random things I noticed in the code.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156151828
2017-05-17 12:26:11 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglai
8892656722 Make WHOIS domain query return conform to ICANN CL&D policy
ICANN's policy can be found here:
https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155375811
2017-05-17 11:38:28 -04:00
mcilwain
07c1f58004 Fix two warnings thrown by Eclipse
One is an unnecessary import and the other is an incorrectly named
Javadoc parameter.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153095269
2017-04-26 10:36:51 -04:00
mcilwain
08009e755f Don't include "Command" suffix in WHOIS metrics command name label
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152414152
2017-04-10 13:46:21 -04:00
larryruili
7359cc13b8 Implement retry for transient errors in WHOIS server
We now attempt to retry Whois queries in the event of a short-lived error. Currently, we consider 'DatastoreTimeoutException' and 'DatastoreFailureException' as transient.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152044934
2017-04-05 10:18:23 -04:00
cgoldfeder
9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
mountford
1f000b94e6 Set correct auth settings for all actions
A test has been added to RequestHandlerTest, making sure that, while we merely log errors for the time being, the correct dummy AuthResult is being created.

Most actions use the default settings, which have been changed to INTERNAL / APP / IGNORED. Actions with non-default settings are:

INTERNAL/NONE/PUBLIC (non-auth public endpoints)

CheckApiAction
WhoisHttpServer
Rdap*Action

INTERNAL,API/APP/ADMIN (things currently protected by web.xml)

EppTlsAction
EppToolAction
CreateGroupsAction
CreatePremiumListAction
DeleteEntityAction
List*sAction
UpdatePremiumListAction
VerifyOteAction
WhoisServer

INTERNAL,API,LEGACY/USER/PUBLIC (registrar console)

RegistrarPaymentAction
RegistrarPaymentSetupAction
RegistrarSettingsAction
EppConsoleAction

INTERNAL,API,LEGACY/NONE/PUBLIC (registrar console main page)

ConsoleUiAction

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149761652
2017-03-13 11:22:55 -04:00
mcilwain
815dae2749 Refactor WhoisReader.readCommand() method to take a now param
This obviates the need for the use of @AutoFactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149570411
2017-03-13 10:45:40 -04:00
mcilwain
3fcb564251 Record metrics for WHOIS commands
Note that this does not write out metrics for invocations of the
nomulus tool.

This requires a slight refactoring of the existing WhoisResponse
interface so as to also support returning the number of results found
by the WHOIS query.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149461208
2017-03-13 10:30:53 -04:00
mcilwain
bd7db61606 Use Dagger to @Inject WhoisReader instances
This is a precursor to adding metrics to WHOIS queries (as I'd like
to be able to @Inject the metrics builders).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149418018
2017-03-07 14:02:46 -05:00
mcilwain
53785c2fc7 Add WHOIS metrics classes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149344437
2017-03-07 14:01:16 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
cgoldfeder
135b1f95f4 Fix a javadoc error
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146934558
2017-02-08 17:42:49 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mcilwain
579e4cb74e Remove TLD parameters from WHOIS command factory methods
The small efficiency increase in not having to look up the TLDs again
did not justify making the externally extensible API more complicated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145465971
2017-01-25 12:25:54 -05:00
mcilwain
124011a8ce Add Optional types to fix Java 7 WHOIS build breakage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145452302
2017-01-25 12:25:54 -05:00
Justin Graham
bb3a0c78c5 Add framework for customizable WHOIS commands
With some additional changes by Ben McIlwain.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145447136
2017-01-25 12:25:49 -05:00
mcilwain
9f142c6767 Remove the util package's dependency on the config package
This allows us to use util methods from within config, which is a useful thing
to be able to do for, e.g., being able to log errors while loading configuration.
It makes sense that the util package should be at the very base of the
class inheritance hierarchy; config seems logically higher than it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144324273
2017-01-12 14:14:51 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00