The main goal of this CL is to make the clock constructor injectable (so that tests do not need to use InjectRule to override the clock). The complication is that the clock is used by two static caches (ROOT_CACHE directly and CRL_CACHE indirectly). The clock is not actually used to construct the lock, but rather to verify that the root certificate is within its validity period.
For ROOT_CACHE we move the verification to its call sites. This adds a bit overhead because the validity check happens every time the cache is called, not just when the cache is built or refreshed. However this check is rather cheap. Also the resources are included in the jar and the cache is valid for 1 year. Given that we deploy every week, there's not much point making it an expiring cache rather than a static map.
For CRL_CACHE we change the key to a tuple of TmchCaMode and X509Certificate. The certificate is no longer provided from the ROOT_CACHE directly and must be verified before it is provided as a cache key. We left the CRL verification inside the cache loader because it (signature verification) is more expensive compared to simple expiration check, and we do not want to do this every time the cache is called.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218385684
When creating the various registrar objects in Sandbox for OTE, we also give access to all the registrars' data to a given google account (identified by the email)
This email has to belong to the registry's G-Suite account, just like in the registrar_contact command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217728407
This is what you want most of the time when you're running the command to
see recent domains, as this will list all the real billable domains and
exclude the prober ones.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217603275
*** Reason for rollback ***
Automated tools sometimes don't have default credentials, and can't set them up. We should redo this CL once we figure out the credential thing.
*** Original change description ***
Add metrics for the command used in the registry CLI tool
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214048616
This change completes the switch to @DefaultCredential for
all use cases in GAE.
Impacted modules:
- IcannReporting
- CreateCdnsTld command
- LoadSnapshot command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213511730
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212879670
The vast majority of the time this is the registrar client ID you want, so
there's no reason to require specifying it everything each time. These are
read-only commands anyway, so the potential negative effects are minimal.
See the existing lock/unlock_domain commands for existing occurrences of this
behavior.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=211857712
This change required several things:
- Separating out the interfaces that merely do HTTP calls to the backend from those
that require the remote API (only load the remote API for the latter). Only the
tools service provides the remote api endpoint.
- Removing the XSRF token as an authentication mechanism (with OAUTH, we no longer
need this, and trying to provide it requires initialization of the datastore
code which requires the remote API)
I can't think of a compelling unit test for this beyond what already exists.
Tested:
Verified that:
- nomulus tool commands (e.g. "list_tlds") work against the tools service as they
currently do
- The "curl" command hits endpoints on "tools" by default.
- We can use --server to specify endpoints on the default service.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=211510454
Do not allow the user to create TLDs on sandbox that aren't of the form
"*.test.". If real TLDs are created, they will block users from registering
names under that TLD for the nameserver set that we're using for sandbox.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209983482
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.
Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
16 is consistent with how we've generated codes for anchor tenants in the past.
Also gets rid of a space in the output so that it's a fully valid CSV.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=208106631
The design doc is at []
The next step will be to tie this into the domain create flow, and if the domain
name is on a reserved list, allow it to be created if the token is specified that
has the given domain name on it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207884521
Create a command to send arbitrary, authenticated HTTP requests to the backend
and remove the existing commands that are basically just wrappers around this.
Tested:
In addition to the unit tests, verified both get and post requests against
alpha.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207756509
This is left over from the transition from Guava to Java 8 Optionals.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207154260
Also adjusts the nomulus list_cursors command to output the value of this field.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206646117
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).
This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.
This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
I'm finally fed up enough with all the nameserver changes we've had to make on our
self-allocated domains to improve the command. Now you can simply run:
$ nomulus ... update_domain ... -n ns[1-4].foo.bar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205282317
Second step of RDE encoding refactoring.
Creates a single OutputStream encode RyDE files.
This replaces the 5 OutputStreams that were needed before.
Also removes all the factories that were injected. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value in each individual OutputStream.
This doesn't yet include a decoder (InputStream). And there's still a lot of overlap between the Ryde and the Ghostryde code. Both of those are left for the next CLs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204898369
Lists used as accumulators were being updated individually for each domain
without starting over from a fresh list each time, so the number of changes
would grow for each additional domain and potentially be wrong if the previous
domains were set up differently.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204526006
This means that, when writing new tests that are failing, you get much more
useful logs that show the actual XML in a more comprehensible format that is
suitable for pasting back into the golden file in the test (if the change was
intended).
This requires outputting the standalone parameter in the XML transformer, and
some minor changes to some tests as a result that were relying on it being
stripped out.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204513690
We never launched this, don't planning on launching it now anyway, and it's rotted over the past two years anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202993577
This removes the following unnecessary imports:
//third_party/java_src/gtld/java/google/registry/bigquery
//third_party/java_src/gtld/java/google/registry/config
//third_party/java_src/gtld/java/google/registry/cron
//third_party/java_src/gtld/java/google/registry/dns
//third_party/java_src/gtld/java/google/registry/gcs
//third_party/java_src/gtld/java/google/registry/mapreduce
//third_party/java_src/gtld/java/google/registry/model
//third_party/java_src/gtld/java/google/registry/module/backend
//third_party/java_src/gtld/java/google/registry/module/frontend
//third_party/java_src/gtld/java/google/registry/module/pubapi
//third_party/java_src/gtld/java/google/registry/module/tools
//third_party/java_src/gtld/java/google/registry/request
//third_party/java_src/gtld/java/google/registry/security
//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers
//third_party/java_src/gtld/java/google/registry/util
//third_party/java_src/gtld/java/google/registry/xjc
//third_party/java_src/gtld/javatests/google/registry/model
//third_party/java_src/gtld/javatests/google/registry/testing
//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java_src/gtld/java/google/registry/bigquery,//third_party/java_src/gtld/java/google/registry/config,//third_party/java_src/gtld/java/google/registry/cron,//third_party/java_src/gtld/java/google/registry/dns,//third_party/java_src/gtld/java/google/registry/gcs,//third_party/java_src/gtld/java/google/registry/mapreduce,//third_party/java_src/gtld/java/google/registry/model,//third_party/java_src/gtld/java/google/registry/module/backend,//third_party/java_src/gtld/java/google/registry/module/frontend,//third_party/java_src/gtld/java/google/registry/module/pubapi,//third_party/java_src/gtld/java/google/registry/module/tools,//third_party/java_src/gtld/java/google/registry/request,//third_party/java_src/gtld/java/google/registry/security,//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers,//third_party/java_src/gtld/java/google/registry/util,//third_party/java_src/gtld/java/google/registry/xjc,//third_party/java_src/gtld/javatests/google/registry/model,//third_party/java_src/gtld/javatests/google/registry/testing,//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202652421
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
First step of RDE encoding refactoring.
Creates a single InputStream (OutputStream) to decode (encode) Ghostryde files.
This replaces the 3 InputStreams (OutputStreams) that were needed before.
Also removes a lot of classes, and removes the "injection" of the Ghostryde
class. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value. It's just a buffer size - it doesn't actually affect much. There
are much more "important" fields that weren't configured (such as the
compression algorithm and whether or not to do integrity checks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202319102
We'll continue to use injected clocks for the rest of our tests that use signed marks files, so that they don't all fail after the current validity period. The new test TmchTestDataExpirationTest will let us know when the files are expired, so we can update them.
All updated test data files come from https://newgtlds.icann.org/en/about/trademark-clearinghouse/registries-registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202208196
The signature has expired on Mon Jun 25 23:59:59 UTC 2018.
Instead of fixing all the test files with a new signed mark, we inject the
clock used to validate the signature.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202116721
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
They're all in the same entity group anyway (the cross-TLD one), so they can be
loaded in a single call instead of individually.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201364854
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
Also explicitly state that contacts missing GAE-UserId can't access the
registrar console
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402953
This limit did not exist prior to [] which added the ability to limit
the size of the list. I didn't think that we needed to be able to query more
than 30 TLDs at any one time so I got rid of batching, but it turns out we do
need this ability for domain_watcher. So I'm re-adding batching, which is a
little bit more complicated now that we're also limiting and sorting by creation
time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199826414
All domain locks we've processed so far are as a result of the URS process, for
which the clientId is always that of the registry's registrar. So it makes sense
to default to that value, while still retaining the option to specify it if
required in case we ever support registrar-requested registry locks in the
future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199350120
Premium prices are automatically detected and set, with an informational
message displayed to the user prior to executing the command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199223541
Currently, we have two different ways to parse a "set" parameter:
key=value1&key=value2&key=value3...
and
keys=value1,value2,value3
This is error prone for several reasons:
- different parts of the code must be "synchronized" to use the same style (the
place that creates the request, and the place that parses the request)
- for the key=value1&key=value2, we often use the same key name for the single
value and the set value. This can result in subtle bugs where part of the
code will successfully read the key assuming there's only one key (and will
get the first key=value1, ignoring the rest)
Here we transition everything to the keys=value1,value2,value3 method. This one
was chosen because:
- it's shorter
- it's more intuitive for users
- the key name is plural, differentiating it from the singular key=value that
other requests might need
-----------------------------------
To make sure there are not "transition issues", we will continue to support
(with warnings) the key=value1&key=value2 parameter parsing until we're sure we
haven't forgotten to update any part of the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198810681
I'm tired of running check_domain only to then realize I needed to run
check_domain_fee instead because the domain ended up being premium. We require
the use of the fee extension to register domains on all of our TLDs anyway so we
might as well always be sending the fee extension when checking domains
too. There's no additional harm in sending along the fee extension (it only
sends back additional information that is often useful), so just make that the
default and remove the extra unnecessary command.
Note that check_domain_claims can't be merged in too because it fundamentally
works differently. It doesn't query the availability of domain names for
registration, just whether they're trademarked.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097326
We've already had the need to do this on a few occasions, so it's best to make
it easy rather than requiring hand-generated XML all the time.
Also normalizes the boolean --registrar_request parameter to not have arity=1.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198093829
You don't want to use the cache when loading them for the purposes of updating
them, but you definitely do still want to use the cache when checking the
price of individual domains.
In [] the cache clearing of premium lists on update was removed. This
is a good thing in aggregate because the cache is per-instance and thus
misleading, but it also caused us to not be able to update the same premium
list twice within an hour because the second update would hit a "PremiumList
was concurrently edited" exception, owing to first loading the stale version
from the cache for the purposes of updating it. Now we bypass the cache for
that purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197768142
