Commit graph

61 commits

Author SHA1 Message Date
guyben
f0621c4df1 Add unit tests to RequestStatusCheckerImpl
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167742899
2017-09-12 15:51:50 -04:00
guyben
fa70bd272a Add tool to check if other requests are still running
This is needed for the Lock.java enhancement where a lock will be implicitly
released if the request owning it dies.

No matter which solution we want for refactoring the Lock itself, we will need this class.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167600314
2017-09-12 15:51:50 -04:00
guyben
00f2662f33 Retry Datastore errors in CommitLogManifestReader.next()
When trying to run the MapReduce for DeleteOldCommitLogsAction, we run into a
lot of DatastoreTimeoutException during CommitLogManifestReader.next.

This causes the entire shard to fail. Since we have a lot of keys (tens of
millions), this is almost guaranteed to happen, dooming the entire MapReduce.

Here is an attempt to recover from the Timeout Exception by saving the state
before the read, then on failure restoring that state and trying again.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165172222
2017-08-29 16:06:48 -04:00
mcilwain
8869814e96 Add logging statement for # of tasks in DNS queue
This will make DNS issues easier to debug retroactively as we will be
able to determine, by looking at the logs, if the queue size was growing
unbounded.

Also adds some logging helpers to allow programmatically choosing the level
of logging.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163123783
2017-08-01 17:02:00 -04:00
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
jianglai
3a02e6fb11 Add fine level logging methods
We currently only have one method for fine level logging. Add another three so that fine level logging can be called just like other levels.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158760803
2017-06-14 10:40:49 -04:00
mountford
ddb0f5692e Add golden files for request component routing maps
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157026188
2017-06-05 18:17:09 -04:00
mcilwain
07c1f58004 Fix two warnings thrown by Eclipse
One is an unnecessary import and the other is an incorrectly named
Javadoc parameter.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153095269
2017-04-26 10:36:51 -04:00
guyben
ab515cb352 Replace KeystoreKeyring with KmsKeystore comparison
Replace KeystoreKeyring with ComparatorKeyring between KeystoreKeyring and
KmsKeystore. In the opensource version, will replace DummyKeyring with
KmsKeyring directly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152893767
2017-04-13 10:58:57 -04:00
larryruili
7359cc13b8 Implement retry for transient errors in WHOIS server
We now attempt to retry Whois queries in the event of a short-lived error. Currently, we consider 'DatastoreTimeoutException' and 'DatastoreFailureException' as transient.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152044934
2017-04-05 10:18:23 -04:00
ctingue
c6a1c3d870 Clean up minor items found during DiRT
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151125802
2017-03-27 13:25:35 -04:00
guyben
e6a5083b55 Rename .toString and .equals to be less confusing.
There will now be "resultToString" and "resultEquals" to stringify / compare
the method results.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150135475
2017-03-21 15:06:44 -04:00
Ben McIlwain
9a11f125ff Remove "@ForOverride" from .toString and .equals
Inheriting classes reimplementing these methods might want to call the super.
version of them in some cases (only change the default behavior for some
methods).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149772767
2017-03-13 11:22:56 -04:00
Ben McIlwain
f5e76868f0 Add generic implementation comparator
This is an alternative to the "manually constructed" version in []

This generator recieves two objects implementing some interface. It returns a new implementation that exactly duplicates the behavior (return values, exceptions etc.) of the first ("original") object, but also logs any differences (in either return value or thrown exception) from the second object.

Nothing in the second object's execution will affect the results - not even thrown exceptions - it's only used for logging any differences.

This is useful when switching backend architecture for the some interface (say... from keystore to datastore+KMS for the Keyring interface :)) as we can make sure everything was OK for some extended period of real data before doing the switch.

This is better than just comparing the outputs once because having a completely different backend out of our control can result in "glitches" that are rare but relevant: maybe the new backend has lower availability, or sometimes fails and gives wrong results, or... well, that's it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149706165
2017-03-13 11:22:55 -04:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
Wolfgang Meyers
6cdb3d81d3 Generate authInfo for imported domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146956307
2017-02-08 17:42:57 -05:00
mcilwain
ec55aa5361 Resolve some Guava 20 TODOs (mostly unnecessary asList() calls)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146799536
2017-02-07 13:27:40 -05:00
cgoldfeder
e2e37dc9f3 Replace Throwables.propagate and variants with Guava 20 versions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146250470
2017-02-02 17:01:23 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mcilwain
9f142c6767 Remove the util package's dependency on the config package
This allows us to use util methods from within config, which is a useful thing
to be able to do for, e.g., being able to log errors while loading configuration.
It makes sense that the util package should be at the very base of the
class inheritance hierarchy; config seems logically higher than it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144324273
2017-01-12 14:14:51 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00
mcilwain
25a8bbe890 Deprecate more fields in RegistryConfig
This primarily addresses issues with TMCH testing mode and email sending utils.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143710550
2017-01-09 11:59:04 -05:00
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
nickfelt
0405a427f1 Remove unnecessary SecureRandom from UrlFetchUtils
We're only using it for generating multiparty boundaries, and there's no real need for the random boundary values to be cryptographically secure.  The point of the randomness is just to make collisions with content in the payload sufficiently unlikely.  The app itself controls the payload contents, and while it might be derived from user-submitted content, in practice it would be nearly infeasible to get the payload to contain arbitrary boundary values even if the RNG-produced boundaries could be determined in advance.

To further insulate against this, I've increased the boundary size (from 40 bits to 192) and added an actual check that the boundary isn't present in the input data, so that in the extremely unlikely event of a collision, we fail rather than producing an invalid multipart request.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142784289
2016-12-22 16:10:44 -05:00
cgoldfeder
5b9219fbdd Add filter support to the test server
The lack of ObjectifyFilter means that in any tests using
RegistryTestServer the Objectify session cache persists
between "requests" in the same test method. This is wrong
but had not caused any failures because we didn't assert
anything that mattered. However, a CL I'm working on
asserts that there is a creationTime on a created resource,
which is set automatically on Datastore save, and therefore
is still null in the session cache's version of the resource
if you don't clear it before the next command. Fixing it here
separately from that CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141939330
2016-12-13 18:00:15 -05:00
mcilwain
2b7d580bb3 Run buildifier on codebase to format BUILD files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140362453
2016-11-28 18:15:21 -05:00
mmuller
ddacd33c5e Retry RDE report on SocketTimeoutException
It's likely that either 1) we should be doing this for more than a
SocketTimeoutException - we probably want to do this in the case of a number
of different transient failures 2) we don't want to do this at all because it
happens in a background task that will get re-run anyway.

In any case, this seems like the right fix and it addresses a problem we see
occassionally.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139935190
2016-11-28 11:31:16 -05:00
mcilwain
4d2e0941f3 Add a custom logic framework to provide pluggable extensibility
To add additional logic for flow code, write custom classes that extend the existing custom logic classes (of which DomainCreateFlowCustomLogic is the first provided example), along with a class that extends CustomLogicFactory to provide instances of the new custom logic classes. Then configure the fully qualified class name of your new custom logic factory in ConfigModule.provideCustomLogicFactoryClass().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139221577
2016-11-15 15:19:32 -05:00
cgoldfeder
d5104df453 Fix two ErrorProne warnings in TypeUtils
1) Prefer .getConstructor().newInstance() over .newInstance()
   because otherwise checked exceptions can be propagated from
   the constructor even though they aren't declared.
2) Use the type T in the parameters to instantiate().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138874730
2016-11-15 10:50:39 -05:00
ctingue
7a77819977 Add retry logic to CreateLrpTokensCommand
A transient 404 on entity save interrupts a long-running run of this command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138400654
2016-11-10 11:26:03 -05:00
jianglai
59d998954c Use correct <a> tag syntax in javadoc @see tag
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137946021
2016-11-02 15:19:34 -04:00
mcilwain
1627bd4975 Revert Guava 20 features until we get the build working properly
*** Original change description ***

Remove deprecated methods with Guava 20 release

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137945126
2016-11-02 15:19:34 -04:00
jianglai
82b0bff9b5 Remove deprecated methods with Guava 20 release
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137849843
2016-11-02 15:19:34 -04:00
jart
2e81de9954 Make essential Bazel packages publicly visible
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
2016-10-24 11:57:00 -04:00
Wolfgang Meyers
94af94ddff Refactor some RDE import logic to be generic
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136076610
2016-10-14 17:00:26 -04:00
jianglai
71d7a382f3 Change all references to Domain Registry to Nomulus
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136068582
2016-10-14 16:58:07 -04:00
shikhman
f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
mcilwain
e864fedab1 Remove trailing whitespace in files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135386752
2016-10-07 15:29:47 -04:00
ctingue
bba2fb92cc Fix TokenUtils Javadoc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135361690
2016-10-07 14:40:01 -04:00
ctingue
79387f5d1e Centralize token generation and formats
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135313726
2016-10-07 14:35:35 -04:00
nickfelt
f86c7c161c Move an internal-only tool out of opensource code directory
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134598884
2016-10-03 16:00:01 -04:00
mcilwain
aa7c05cb8b Improve pretty-print diffing of Datastore entities
This removes the countless lines of the form "[null, []]" in registry_tool diffs
that are an artifact of the way we handle nulls in Objectify.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133409440
2016-09-19 12:06:54 -04:00
Justine Tunney
7f3f03ee97 MOE strip compatible_with
This is an internal-only feature that breaks the open source build.

CL created with:

    dr-replace '(compatible_with.*)' '\1  # MOE:strip_line'

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128852873
2016-08-02 19:14:28 -04:00
Chris Povirk
5332ac4e4a Set compatible_with=appengine on GAE targets
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128475519
2016-08-02 19:09:11 -04:00
mountford
e72491e59b Replace to(Upper|Lower)Case with Ascii.to$1Case
The presubmits are warning that toUpperCase() and toLowerCase() are locale-specific, and advise using Ascii.toUpperCase() and Ascii.toLowerCase() as a local-invariant alternative.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127583677
2016-07-19 10:43:38 -04:00
mcilwain
aa2f283f7c Convert entire project to strict lexicographical import sort ordering
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127234970
2016-07-13 15:59:53 -04:00
Wolfgang Meyers
bdfa97b0ae Add XjcToContactResourceConverter
XjcToContactResourceConverter is, as it name suggests, an inverse of
ContactResourceToXjcConverter. This utility class is designed to
support the TLD data import feature.

EXTERNAL_REVIEW_URL=https://github.com/google/domain-registry/pull/19
GIT_AUTHOR=Wolfgang Meyers <wolfgang@donuts.co>
(With some minor changes by Ben McIlwain.)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125985714
2016-07-01 11:38:05 -04:00
Ben McIlwain
9eeb0c43a1 Change import to reference canonical implementation of checkArgument
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125984267
2016-06-27 16:48:41 -04:00
cgoldfeder
bb82f5bc05 Clean up the tattered shreds of SessionMetadata
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125334811
2016-06-27 16:20:48 -04:00