I'm finally fed up enough with all the nameserver changes we've had to make on our
self-allocated domains to improve the command. Now you can simply run:
$ nomulus ... update_domain ... -n ns[1-4].foo.bar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205282317
Lists used as accumulators were being updated individually for each domain
without starting over from a fresh list each time, so the number of changes
would grow for each additional domain and potentially be wrong if the previous
domains were set up differently.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204526006
This removes the following unnecessary imports:
//third_party/java_src/gtld/java/google/registry/bigquery
//third_party/java_src/gtld/java/google/registry/config
//third_party/java_src/gtld/java/google/registry/cron
//third_party/java_src/gtld/java/google/registry/dns
//third_party/java_src/gtld/java/google/registry/gcs
//third_party/java_src/gtld/java/google/registry/mapreduce
//third_party/java_src/gtld/java/google/registry/model
//third_party/java_src/gtld/java/google/registry/module/backend
//third_party/java_src/gtld/java/google/registry/module/frontend
//third_party/java_src/gtld/java/google/registry/module/pubapi
//third_party/java_src/gtld/java/google/registry/module/tools
//third_party/java_src/gtld/java/google/registry/request
//third_party/java_src/gtld/java/google/registry/security
//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers
//third_party/java_src/gtld/java/google/registry/util
//third_party/java_src/gtld/java/google/registry/xjc
//third_party/java_src/gtld/javatests/google/registry/model
//third_party/java_src/gtld/javatests/google/registry/testing
//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java_src/gtld/java/google/registry/bigquery,//third_party/java_src/gtld/java/google/registry/config,//third_party/java_src/gtld/java/google/registry/cron,//third_party/java_src/gtld/java/google/registry/dns,//third_party/java_src/gtld/java/google/registry/gcs,//third_party/java_src/gtld/java/google/registry/mapreduce,//third_party/java_src/gtld/java/google/registry/model,//third_party/java_src/gtld/java/google/registry/module/backend,//third_party/java_src/gtld/java/google/registry/module/frontend,//third_party/java_src/gtld/java/google/registry/module/pubapi,//third_party/java_src/gtld/java/google/registry/module/tools,//third_party/java_src/gtld/java/google/registry/request,//third_party/java_src/gtld/java/google/registry/security,//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers,//third_party/java_src/gtld/java/google/registry/util,//third_party/java_src/gtld/java/google/registry/xjc,//third_party/java_src/gtld/javatests/google/registry/model,//third_party/java_src/gtld/javatests/google/registry/testing,//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202652421
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
This limit did not exist prior to [] which added the ability to limit
the size of the list. I didn't think that we needed to be able to query more
than 30 TLDs at any one time so I got rid of batching, but it turns out we do
need this ability for domain_watcher. So I'm re-adding batching, which is a
little bit more complicated now that we're also limiting and sorting by creation
time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199826414
Premium prices are automatically detected and set, with an informational
message displayed to the user prior to executing the command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199223541
I'm tired of running check_domain only to then realize I needed to run
check_domain_fee instead because the domain ended up being premium. We require
the use of the fee extension to register domains on all of our TLDs anyway so we
might as well always be sending the fee extension when checking domains
too. There's no additional harm in sending along the fee extension (it only
sends back additional information that is often useful), so just make that the
default and remove the extra unnecessary command.
Note that check_domain_claims can't be merged in too because it fundamentally
works differently. It doesn't query the availability of domain names for
registration, just whether they're trademarked.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097326
We've already had the need to do this on a few occasions, so it's best to make
it easy rather than requiring hand-generated XML all the time.
Also normalizes the boolean --registrar_request parameter to not have arity=1.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198093829
You don't want to use the cache when loading them for the purposes of updating
them, but you definitely do still want to use the cache when checking the
price of individual domains.
In [] the cache clearing of premium lists on update was removed. This
is a good thing in aggregate because the cache is per-instance and thus
misleading, but it also caused us to not be able to update the same premium
list twice within an hour because the second update would hit a "PremiumList
was concurrently edited" exception, owing to first loading the stale version
from the cache for the purposes of updating it. Now we bypass the cache for
that purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197768142
This allows list_domains to continue working for large TLDs.
TESTED=Deploys to alpha and it works to list the most recently created domains even
on a TLD with a huge number of domains on it (much more than .app has currently).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196717389
TestDataHelper is build exactly to prevent direct reads of resources. It caches
the resources and makes sure they are in the correct directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191785004
We no longer want to check end-date sunrise for OTE verification, so we should
remove any reference to allocation (and applications, which was removed in a
previous CL)
However, this last allocate was also the command that tested secDNS. So we need
to replace it with a regular create with secDNS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191604222
Before this change the output looks like this:
registrar1 - Num actions: 93 - Reqs passed: 16/16 - Overall: PASS
registrar2 - Num actions: 47 - Reqs passed: 6/16 - Overall: FAIL
After this change the output looks like this:
registrar1 - # actions: 93 - Reqs: [----------------] 16/16 - Overall: PASS
registrar2 - # actions: 47 - Reqs: [...--.-...-...--] 6/16 - Overall: FAIL
The status of each test is displayed as a hyphen (passing) or a period (failing),
and the tests are always displayed in the same order so it's easier to get an overall
view of whether registrars are struggling with the same tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190776935
We already have verifications that a domain application was created in sunrise
- which checks for end-date sunrise. Start-date sunrise has checks that a
domain (not application) was created. There's no need to specifically check for
a signed mark, since a successful domain create during sunrise must have a
signed mark in it.
Also removed the requirement for end-date sunrise / landrush testing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190551080
The DS records consist of 4 values:
- keyTag: unsigned short (2 bytes)
- alg: unsigned byte
- digestType: unsigned byte
- digest: binary hex
NOTE: the current CL doesn't support keyData, neither as the optional field in dsData nor as a replacement for dsData
The command tool accepts DS records as a string, where the 4 values are given
as one string separated by white-spaces as follows:
<keyTag> <alg> <digestType> <digest>
e.g. something like:
60485 5 2 D4B7D520E7BB5F0F67674A0CCEB1E3E0614B93C4F9E99B8383F6A1E4469DA50A
which is how it's written in Zone files, allowing easy copy-paste from existing values.
ommas is confusing when using spaces.
The various "numbers" (keyTag, alg, digestType) are only checked that they are
positive integers - the rest is left for the server.
digest it checked to be an even-lengthed hex string.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184583068
The assertThrows/expectThrows refactoring script does not use method
references, apparently.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179089048
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.
Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
The only remaining methods on ExceptionRule after this are methods that
also exist on ExpectedException, which will allow us to, in the next CL,
swap out the one for the other and then run the automated refactoring to
turn it all into assertThrows/expectThrows.
Note that there were some assertions about root causes that couldn't
easily be turned into ExpectedException invocations, so I simply
converted them directly to usages of assertThrows/expectThrows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178623431
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
The scheme is:
- loadBytes: returns a ByteSource of the data
- loadFile: returns a string using UTF8 encoding, optionally applying
substitutions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177606406
There is a big mix of different "load Resources" from different libraries
depending on where you were and what type of resource you want. Now there is a
clear hirarchy:
ResourceUtils:
for use in actual (non-test) code, reads a file from a context directory
TestDataHelper (uses ResourceUtils):
for use in tests, reads a file from a context directory + "/testdata". Also
caches the resource so calling it multiple times with the same file will
not read the file multiple times.
Library specific helpers (e.g. ToolsTestData) (uses TestDataHelper):
for use in that library's tests only, reads from a specific testdata directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177027533
This command is used by registry operators to apply registry locks to
domain names.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176577240
This command is used by registry operators to apply registry locks to
domain names.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176549874
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
It makes sense for all mapreduces to run in backend, especially onces
that are scheduled regularly to run in cron like this one now. We don't
have many instances configured for the tools service anymore on some
of our environments, so backend is the friendliest place for a mapreduce
to run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882122
We're planning on adding a cronjob to run this mapreduce monthly, so
we may as well also project the resources being re-saved to the present
time so as to handle pending transfers, grace periods, and such. This will
make the BigQuery exports more useful.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168831056
This pattern will mainly be used for data migrations, i.e. updating all
HistoryEntries' DomainTransactionRecords to the new schema.
TESTED=Ran in alpha, the underlying data dropped non-Objectify fields as
expected.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168684356
This is the first in a series of CLs containing code from an old CL of Dai's that had never been completed, which compares zone data between Datastore and DNS. I had written a script to do this by calling two nomulus commands, but maybe it can be done directly in Java, which would be convenient.
This CL is just the plumbing to check on the status of a Mapreduce. We will need this to know that we can proceed with the next step of comparing the output to the DNS data.
Cloned from CL 134295050 by 'g4 patch'.
Original change by dxy@dxy:zoneman-reader:1939:citc on 2016/09/26 10:34:22.
Add a command for comparing zone data between DNS and datastore
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167188979
In standard DNS format, the first thing on an A, NS or DS definition line is a domain label relative to the zone, which in our case is a TLD. However, the generate_zone_files command prints out fully qualified host and domain names, resulting in a discrepancy when compared to the contents of the DNS subsystem. This CL removes the TLD suffix, which should remove one preprocessing step before file comparison.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166103705
Previously, GenerateZoneFilesAction mapreduced its way through all domains and hosts for the specified TLD(s), emitting information for each matching domain and host (subject to constraints like not being deleted and so on). This resulted in host information (aka glue records) for all hosts subordinate to domains in the specified TLD(s). This is incorrect. DNS glue records should only be present for hosts which act as nameservers for their superordinate domains.
The new version of the mapreduce iterates only over domains. When a matching domain is found, a check is made to see whether any subordinate hosts are also nameservers for the domain, in which case host information is generated.
The test was updated to reflect the new reality, and check for a couple additional nuances.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165766472
Attending to this old bug will improve our ability to perform zone comparisons between Datastore and the DNS provider. Right now, zone comparison finds some bogus differences, because the TTL we send to the DNS subsystem doesn't match the TTL we use when generating our local dump files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164635557
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).
This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
This will allow us to migrate one TLD at a time by refreshing all zones
on the specified TLD after dual-writing is enabled.
Note that the TLDs parameter is required, which seems reasonable given
that almost all imagined use cases would be on a by-TLD basis.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160294546
This is better than calling assertTldExists() inside a for loop because you can throw a single exception reporting all bad TLDs at once rather than only getting as far as the first failure. And then it's also a one-liner instead of 3 lines.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152412876
