The 'referralUrl' Datastore field is filled with mostly junk data, whereas
'url' contains real registrar web addresses. This makes the long needed fix to
display the proper url in WHOIS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194398003
Currently we determine fee type from the fee extension description by checking if the format string of the FeeType contains the description we received. The formatting string for EAP is "Early Access Period, fee expires: %s", so the fee description generated by a domain check command, like "Early Access Period, fee expires: 2022-03-01T00:00:00.000Z", is not recognized as EAP.
This CL adds the ability to add arbitrary extra description strings to the FeeType for a description to match against. It also changes the match to "the given description contains any of the strings from the list of format string plus extra description strings".
For EAP, we added an extra description string "Early Access Period", so any fee description that contains "Early Access Period" will be matched to EAP FeeType, including the specific description (that contains the expiry time) that we send in a domain check.
Also improved error message on multiple fee type matching.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194149162
Per EPP RFC 5730, the <clTRID> element is optional. However, we weren't handling
it not being specified in asynchronous contact/host deletions because we were
adding it directly as a parameter value on a task, which does not allow null and
thus threw a NullPointerException.
This fixes handling for nulls (the parameter isn't set at all) and adds a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194123259
This verifies anchor tenant creates by registrars are for exactly 2 years, to simplify billing down the line (anchor tenants get 2 years of free domain creates).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193931858
This adds a parameter to control invoice reporting, which defaults to false for
now (since we plan on manually adjusting the invoice next month).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193708323
(Usually the latter is because an IDE chose the repackaged name without the developer noticing.)
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193694213
This allows a registrar to check domain availability even if the fee extension is not set. To obtain the premium price or create the domain, the fee extension is still needed.
PS: I believe the previous tests are erroneous anyway. It is the presence of fee extensions in the check request that we were looking for. Checking if the extension is declared during login has no bearing on the results.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192778137
This hard-deletes all contacts and hosts owned by a specific set of registrar
client IDs, currently just "proxy".
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192325211
This makes them consistent with every other command, which uses the format
verb_noun.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192301468
This is in preparation for merging and then removing
RegistrarPremiumPriceAckAction.
This includes:
test that the data the UI sent isn't stale
---------------------------------------------
Our system is "read, modify, write". However, if between the "read" and the "write" someone else changed the registry, my write will undo their change even if I didn't touch any of their fields.
To solve that - we use the "lastUpdateTime" timestamp of the registrar. the UI reads it with the rest of the data, and sends it back on "write". We will now make sure the registrar currently in datastore has the same timestamp.
support premium-price-ack flag
---------------------------------
Add support for reading and writing this flag. We still won't be using it - that's in a followup CL, but we support it.
support changing the URL
------------------------
Add changing the URL in the UI, under the "whois" section
Will replace the Ack endpoint with this (and remove that endpoint) in a followup CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192154078
This is accomplished by making all non-mutating commands function with dry run set
to true, which also has the pleasurable side effect of not prompting for dry-run
mutating commands either, which also do nothing different/special on the second
run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192149150
We don't want people to accidentally run commands on prod thinking they were on
Alpha / Sandbox.
To do that - we add 2 safeguards:
1) when on prod, the shell has a strong RED "PRODUCTION" in the commandline, while on alpha/sandbox it's green.
2) if a prod shell is idle for > 1h, it exits. So don't accidentally use a prod shell from a long time ago.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191931731
After writing a flag on the shell, pressing "tab" will print out the
documentation for that flag.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191899137
TestDataHelper is build exactly to prevent direct reads of resources. It caches
the resources and makes sure they are in the correct directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191785004
This allows us to not obtain a certificate and encrypt it with KMS when running the proxy locally during development.
Also updated FOSS build dagger version.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191746309
The registrar security console failed because it assumed the email is a
required field for the registrar, but it isn't (at least - create_registrar
doesn't require an email, and update_registrar lets you remove the email).
Fixed by allowing it to *remain* unset if it was unset originally, but if it was set - it's required.
There are more fixes needed, but they aren't related to the email, so they will wait for the next CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191623034
We no longer want to check end-date sunrise for OTE verification, so we should
remove any reference to allocation (and applications, which was removed in a
previous CL)
However, this last allocate was also the command that tested secDNS. So we need
to replace it with a regular create with secDNS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191604222
This should prevent having issues with hot key paths on entities that
experience a heavy WHOIS volume (e.g. contacts that registrars reuse on
many domains).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191506124
Using the jline open-source library.
We save the history between invocations to ~/.nomulus_history
We add some simple completions:
- first argument completes to command name
- all other arguments complete to the command parameters, or filename
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191501023
Also increases the number of commit log buckets on alpha to 397 and correspondingly
reduces the frequency of commit log diff exporting to once every 3 minutes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191440586
JCommander doesn't seem to reset objects when it populates them with data from
an argument list during command processing, so recreate the command objects
every time we do a run().
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191332392
These servlets are converted to actions during daggerization. Calling them servers are misleading.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190942237
Before this change the output looks like this:
registrar1 - Num actions: 93 - Reqs passed: 16/16 - Overall: PASS
registrar2 - Num actions: 47 - Reqs passed: 6/16 - Overall: FAIL
After this change the output looks like this:
registrar1 - # actions: 93 - Reqs: [----------------] 16/16 - Overall: PASS
registrar2 - # actions: 47 - Reqs: [...--.-...-...--] 6/16 - Overall: FAIL
The status of each test is displayed as a hyphen (passing) or a period (failing),
and the tests are always displayed in the same order so it's easier to get an overall
view of whether registrars are struggling with the same tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190776935
Caching turns out to be an anti-pattern for the console. If we use it, changes from the user just get obliterated by the older, cached version the next time the console refreshes (and it happens to refresh after every update). Caching is also not very useful here, as the amount of database access driven by the console is very small.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190650931
This is in preparation for automatic EPP consolidation.
The assertion will now look like:
assertThatCommand("file.xml"[, substitution])
.atTime(time)
.hasResponse("file2.xml"[, substitution]);
Also, added convenience functions for often reused commands (login and logout)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190617505
We already have verifications that a domain application was created in sunrise
- which checks for end-date sunrise. Start-date sunrise has checks that a
domain (not application) was created. There's no need to specifically check for
a signed mark, since a successful domain create during sunrise must have a
signed mark in it.
Also removed the requirement for end-date sunrise / landrush testing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190551080
The parameters were optional during the transition to allow old jobs stuck in the queue to work properly. It's been 2 months now so it's time to end the transition.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190235532
Implement a checkbox in the "Resources" tab to allow registrars to toggle
their "premium price ack required" flag.
Tested:
Verfied the console functionality by hand. I've started work on an
automated test, but we can't actually test those from blaze and the
kokoro tests are way too time-consuming to be practical for development, so
we're going to have to either find a way to run those locally outside of
the normal process or make do without a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190212177
A "mark" tells us that the holder owns the trademark for a given domain name. It is signed for authentication.
If the signature's certificate is either "not yet valid" or "expired", we return explicit errors to that effect.
But in addition to the signature's certificate, the mark itself might not be valid yet or already expired. Right now if that happens - we return an error saying "the mark doesn't match the domain name".
That is wrong - as the mark can match the domain name, just be expired. Returning "the mark doesn't match the domain name" in that case is misleading.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190069976
'afterFinalFailure' is called just before rethrowing a non-retrying error from
the retrier. This can happen either because the exception shouldn't be retried,
or because we exceeded the maximum number of retries.
The same thing can be done by catching that thrown error outside of the
retrier:
retrier.callWithRetry(
callable,
new FailureReporter() {
@Override
void afterFinalFailure(Throwable thrown, int failures) {
// do something with thrown
}
},
RetriableException.class);
is (almost) the same as:
try {
retrier.callWithRetry(callable, RetriableException.class);
} catch (Throwable thrown) {
// do something with thrown
throw thrown;
}
("almost" because the retrier might wrap the Throwable in a RuntimeException,
so you might need to getCause or getRootCause. Also - there is the
"beforeRetry" I ignored for the example)
Removing "afterFinalFailure" also makes the FailureReporter in line with Java 8
functional interface - meaning we can more easily create it when we do need to
override "beforeRetry".
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189972101
TldFanoutAction fans out a given endpoint to all TLDs (either TEST, REAL, or
both).
However, it is also used to delegate a single endpoint request that we want set
in a specific queue (so we can control retries). We do that by setting the TLD
list to "runInEmpty" rather than "forEachRealTld" or "forEachTestTld".
Currently, using "runInEmpty" would still specify a TLD - but that TLD would be
the empty string. This is a bug: it sets the TLD parameter to a bad value. It
worked only because none of the endpoints called with "runInEmpty" were using
the TLD parameter.
However, this will (and does) break if either (a) the endpoint accepts an
optional TLD parameter (like deleteProberData does), or (b) the given endpoint
already has a TLD parameter in it (we want to run the endpoint with a single
TLD, but still use the "fanout" to set the right queue).
This CL fixes several things:
- if runInEmpty is given, no TLD parameter is added
- 'runInEmpty' is now mutually exclusive with 'forEach*Tld' and 'excludes'
- we do some sanity checks and added logging
- removed the buggy and unused "':tld' in path is replaced by TLD"
- in the cron.xml, removed documentation for :tld and the broken :registrar
Note that none of the endpoints that were used with runInEmpty fanout had the TLD parameter prior to deleteProberData
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189954585
<launch:create> has an optional type argument, that can take either "application" or "registration":
https://tools.ietf.org/html/rfc8334#section-3.3.1
We get that type via createExtension.get().getCreateType(), where if the type= argument isn't given, the function returns null.
In that case, we need to decide based on the TLD - application for end-date sunrise, and registration for start-date sunrise.
For now we can't do that, because FlowPicker doesn't have access to the TLD information. Until that is fixed we decide as follows:
- landrush and sunrush phases will default to APPLICATION, because there's no possible
registration for it.
- sunrise defaults to REGISTRATION because we're currenly launching start-date sunrise that uses registration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189942568
We are no longer using Eclipse internally and therefore stopped maintaining
stuff related to it. We cannot guarantee that any pertinent information remains correct
and relevant in the future.
Users are advised to use IntelliJ (Community Edition is fine) with Bazel plugin
if they want IDE support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189586127
Associate the custom metrics with the correct monitored resource type. The labels of the monitored resource are either obtained from environment variables for the container, configured in the GKE deployment file, or queried from GCE metadate server. Using the correct monitored resource can help performance and reduced out-of-order metric writes.
Also changed the metrics display name to be more descriptive.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189184411
More information: []
Tested:
TAP for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188968676
Add the "shell" command which lets you run multiple other command in a single
session, sparing you the initialization costs for all but the first of them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188712815
Allow specifying certificate hash other than certificate file. This makes things easier when only setting up EAP registrars. The certificate hash can be easily pulled from existing registrars (SUNRISE, GA, etc) with automation.
Also fixes a bug where we always expect the registrar name + phase string to be at least 7-character long.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188511561
If the proxy protocol header contains a malformatted string, such as "PROXY UNKNOWN", instead of throwing and killing the connection, use the TCP source IP as the remote IP.
Also changed how the header is read from the buffer, to avoid a potential Netty resource leak. Originally the header is read into another ByteBuf, which needs be be explicit released in order for Netty to reclaim its memory (http://netty.io/wiki/reference-counted-objects.html). Now we just read it into a byte array and let JVM GC it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188047084
