First step of RDE encoding refactoring.
Creates a single InputStream (OutputStream) to decode (encode) Ghostryde files.
This replaces the 3 InputStreams (OutputStreams) that were needed before.
Also removes a lot of classes, and removes the "injection" of the Ghostryde
class. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value. It's just a buffer size - it doesn't actually affect much. There
are much more "important" fields that weren't configured (such as the
compression algorithm and whether or not to do integrity checks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202319102
We'll continue to use injected clocks for the rest of our tests that use signed marks files, so that they don't all fail after the current validity period. The new test TmchTestDataExpirationTest will let us know when the files are expired, so we can update them.
All updated test data files come from https://newgtlds.icann.org/en/about/trademark-clearinghouse/registries-registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202208196
The signature has expired on Mon Jun 25 23:59:59 UTC 2018.
Instead of fixing all the test files with a new signed mark, we inject the
clock used to validate the signature.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202116721
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
They're all in the same entity group anyway (the cross-TLD one), so they can be
loaded in a single call instead of individually.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201364854
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
Also explicitly state that contacts missing GAE-UserId can't access the
registrar console
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402953
This limit did not exist prior to [] which added the ability to limit
the size of the list. I didn't think that we needed to be able to query more
than 30 TLDs at any one time so I got rid of batching, but it turns out we do
need this ability for domain_watcher. So I'm re-adding batching, which is a
little bit more complicated now that we're also limiting and sorting by creation
time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199826414
All domain locks we've processed so far are as a result of the URS process, for
which the clientId is always that of the registry's registrar. So it makes sense
to default to that value, while still retaining the option to specify it if
required in case we ever support registrar-requested registry locks in the
future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199350120
Premium prices are automatically detected and set, with an informational
message displayed to the user prior to executing the command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199223541
Currently, we have two different ways to parse a "set" parameter:
key=value1&key=value2&key=value3...
and
keys=value1,value2,value3
This is error prone for several reasons:
- different parts of the code must be "synchronized" to use the same style (the
place that creates the request, and the place that parses the request)
- for the key=value1&key=value2, we often use the same key name for the single
value and the set value. This can result in subtle bugs where part of the
code will successfully read the key assuming there's only one key (and will
get the first key=value1, ignoring the rest)
Here we transition everything to the keys=value1,value2,value3 method. This one
was chosen because:
- it's shorter
- it's more intuitive for users
- the key name is plural, differentiating it from the singular key=value that
other requests might need
-----------------------------------
To make sure there are not "transition issues", we will continue to support
(with warnings) the key=value1&key=value2 parameter parsing until we're sure we
haven't forgotten to update any part of the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198810681
I'm tired of running check_domain only to then realize I needed to run
check_domain_fee instead because the domain ended up being premium. We require
the use of the fee extension to register domains on all of our TLDs anyway so we
might as well always be sending the fee extension when checking domains
too. There's no additional harm in sending along the fee extension (it only
sends back additional information that is often useful), so just make that the
default and remove the extra unnecessary command.
Note that check_domain_claims can't be merged in too because it fundamentally
works differently. It doesn't query the availability of domain names for
registration, just whether they're trademarked.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097326
We've already had the need to do this on a few occasions, so it's best to make
it easy rather than requiring hand-generated XML all the time.
Also normalizes the boolean --registrar_request parameter to not have arity=1.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198093829
You don't want to use the cache when loading them for the purposes of updating
them, but you definitely do still want to use the cache when checking the
price of individual domains.
In [] the cache clearing of premium lists on update was removed. This
is a good thing in aggregate because the cache is per-instance and thus
misleading, but it also caused us to not be able to update the same premium
list twice within an hour because the second update would hit a "PremiumList
was concurrently edited" exception, owing to first loading the stale version
from the cache for the purposes of updating it. Now we bypass the cache for
that purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197768142
This allows list_domains to continue working for large TLDs.
TESTED=Deploys to alpha and it works to list the most recently created domains even
on a TLD with a huge number of domains on it (much more than .app has currently).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196717389
Since the ConsoleReader now controls the display of the prompt, there is no
longer any way for an external program interacting with the nomulus shell to
recognize when the last command issued has been completed.
To remedy this, we introduce an "--encapsulate_output" flag, which causes
standard output and standard error to be wrapped in a class that precedes
all normal lines of output with a prefix ("out: " or "err: ", accordingly)
and allows the command processor to insert a "SUCCESS" or "FAILURE"
line (with no special prefix) after completing the processing of a
command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196702338
This efficiently counts domains without having to load them (as opposed to the
existing list_domains command which does load then).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194559095
This makes them consistent with every other command, which uses the format
verb_noun.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192301468
This is accomplished by making all non-mutating commands function with dry run set
to true, which also has the pleasurable side effect of not prompting for dry-run
mutating commands either, which also do nothing different/special on the second
run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192149150
We don't want people to accidentally run commands on prod thinking they were on
Alpha / Sandbox.
To do that - we add 2 safeguards:
1) when on prod, the shell has a strong RED "PRODUCTION" in the commandline, while on alpha/sandbox it's green.
2) if a prod shell is idle for > 1h, it exits. So don't accidentally use a prod shell from a long time ago.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191931731
After writing a flag on the shell, pressing "tab" will print out the
documentation for that flag.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191899137
TestDataHelper is build exactly to prevent direct reads of resources. It caches
the resources and makes sure they are in the correct directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191785004
We no longer want to check end-date sunrise for OTE verification, so we should
remove any reference to allocation (and applications, which was removed in a
previous CL)
However, this last allocate was also the command that tested secDNS. So we need
to replace it with a regular create with secDNS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191604222
Using the jline open-source library.
We save the history between invocations to ~/.nomulus_history
We add some simple completions:
- first argument completes to command name
- all other arguments complete to the command parameters, or filename
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191501023
JCommander doesn't seem to reset objects when it populates them with data from
an argument list during command processing, so recreate the command objects
every time we do a run().
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191332392
Before this change the output looks like this:
registrar1 - Num actions: 93 - Reqs passed: 16/16 - Overall: PASS
registrar2 - Num actions: 47 - Reqs passed: 6/16 - Overall: FAIL
After this change the output looks like this:
registrar1 - # actions: 93 - Reqs: [----------------] 16/16 - Overall: PASS
registrar2 - # actions: 47 - Reqs: [...--.-...-...--] 6/16 - Overall: FAIL
The status of each test is displayed as a hyphen (passing) or a period (failing),
and the tests are always displayed in the same order so it's easier to get an overall
view of whether registrars are struggling with the same tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190776935
We already have verifications that a domain application was created in sunrise
- which checks for end-date sunrise. Start-date sunrise has checks that a
domain (not application) was created. There's no need to specifically check for
a signed mark, since a successful domain create during sunrise must have a
signed mark in it.
Also removed the requirement for end-date sunrise / landrush testing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190551080
Add the "shell" command which lets you run multiple other command in a single
session, sparing you the initialization costs for all but the first of them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188712815
Allow specifying certificate hash other than certificate file. This makes things easier when only setting up EAP registrars. The certificate hash can be easily pulled from existing registrars (SUNRISE, GA, etc) with automation.
Also fixes a bug where we always expect the registrar name + phase string to be at least 7-character long.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188511561
Changed SUNRISE to START_SUNRISE and added a registry/registrar pair for testing EAP. The EAP period is set to 2018-03-01 to 2022-03-01 with a price of $100.
A temporary flag is added to only create EAP registry/registrar pair so that we can update existing registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187897405
It was nullable all along, but wasn't tagged as such, and thus it was
possible to misuse the method from its call sites.
Also adds an assertion about no NORDN tasks being enqueued in a failing
domain create test for a required signed mark.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187649865
This enables sharded DNS publishing on a per-TLD basis. Instead of a TLD-wide lock, the sharded scheme locks each update on the shard number, allowing parallel writes to DNS.
We allow N (the number of shards) to be 0 or 1 for no sharding, and N > 1 for an N-way sharding scheme. Unless explicitly set, all TLDs default to a numShards of 0, so we don't have to reload all registry objects explicitly.
WARNING: This will change the lock name upon deployment for the PublishDnsAction from "<TLD> Dns Updates" to "<TLD> Dns Updates shard 0". This may cause concurrency issues if the underlying DNSWriter is not parallel-write tolerant (currently all production usages are ZonemanWriter, which is parallel-tolerant, so no issues are expected).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187525655
Now that we've verified the new Beam billing pipeline works, we can delete the
old manual commands we used to use.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184707182
The DS records consist of 4 values:
- keyTag: unsigned short (2 bytes)
- alg: unsigned byte
- digestType: unsigned byte
- digest: binary hex
NOTE: the current CL doesn't support keyData, neither as the optional field in dsData nor as a replacement for dsData
The command tool accepts DS records as a string, where the 4 values are given
as one string separated by white-spaces as follows:
<keyTag> <alg> <digestType> <digest>
e.g. something like:
60485 5 2 D4B7D520E7BB5F0F67674A0CCEB1E3E0614B93C4F9E99B8383F6A1E4469DA50A
which is how it's written in Zone files, allowing easy copy-paste from existing values.
ommas is confusing when using spaces.
The various "numbers" (keyTag, alg, digestType) are only checked that they are
positive integers - the rest is left for the server.
digest it checked to be an even-lengthed hex string.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184583068
When enabled for a registrar, all EPP operations on premium domains that have
costs (e.g. creates, renews, transfers) will fail unless the EPP fee extension
is used to explicitly ack the amount of fee as part of the EPP transaction.
This ack is required regardless of whether premium fee acking is required at
the registry level. No data migration is necessary since false is the desired
default for this new attribute.
This CL also contains some slight refactoring of static utility methods used to
perform fee verification; there was short-circuiting at call-sites in two
places when what was really needed was two methods, one implementing additional
functionality on top of the other, and calling the inner method in the places
where short-circuiting had previously been necessary.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184229363
To make FOSS build compile, third_party vendoring rules for jaxb are added to package all jaxb related targets imported from maven into a uber jar, mirroring the same practice done in //third_party/java/jaxb
Cloned from CL 182666460 by 'g4 patch'.
Original change by cushon@cushon:rosie182283995-0071_Rosie:47348:citc on 2018/01/20 13:36:15.
More information:
https://docs.google.com/document/d/1htErgDIoHMEuMBfGwrtS_O4WwhTw8QOGLva-7aYYvYs/edit?usp=sharing
Tested:
TAP --sample for global presubmit queue
[] passed FOSS test
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182855173
This creates a specified number of tokens of a given schema, with a dryrun option
to not persist them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181403775
In Truth8, we can do assertThat(stream) directly. It's less verbose and clearer
in most cases.
Note that for the "finishers" (e.g. "containsExactyElementsIn") - streams are
still not allowed. So when there is:
assertThat(stream.map(someTransformation).collect(toList()))
.containsExactlyElementsIn(expecteStream.map(someTransformation).collect(toList()));
I kept the .collect in the assertThat to preserve the symmetry with the
finisher.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179697587
