When we set the payload, it is converted to bytes using the response's
character set. Changing the MIME type later has no effect on the conversion
to bytes, even though it does change the returned MIME type. This results in
bytes that were encoded using one character set while the response reports a
different character set.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196258752
Also added a couple of more tests to make sure that we cover all edge cases.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=195872013
serves from https now
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=195473725
This should decrease the average wait time when running nomulus tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=195465469
Increase the instances on alpha to achieve parity with sandbox.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194980588
The export happens in a reducer, whois instance fields all need to be serializable. The DriveConnection is therefore installed as a class variable that can be replaced with a mock during test. Class variables are not serialized. Only fields related to a particular instance are.
Note that DriveConnection is a misnomer. It is not a connection at all. It is just a thin wrapper class around the Drive service class, which provides convenient methods to write into Drive. Regardless, it cannot be serialized.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194596695
This efficiently counts domains without having to load them (as opposed to the
existing list_domains command which does load then).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194559095
It doesn't make sense to log all 10K+ lines of a premium list every time it's
updated, and indeed that seems to hurt performance, yet that's precisely what
we were doing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194449836
The 'referralUrl' Datastore field is filled with mostly junk data, whereas
'url' contains real registrar web addresses. This makes the long needed fix to
display the proper url in WHOIS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194398003
ListObjectsCommand is a base class for a number of List commands that query the
tools server with the command input, and then show the user the server's reply.
For example, ListDomainsCommand queries ListDomainsAction with the input TLDs
from the user, and then prints the resulting domains for the user.
Currently, when the server query returns an error (because, e.g., the user of
ListDomainsCommand gave a non-existing TLD) - ListObjectsCommand just prints
the error message to stdout, and returns successfully.
That means that any automatic tool would think the command succeeded.
This CL changes the behavior of ListObjectsCommand to throw an exception with
the error message if the server returned an error. This results in the nomulus
command failing if the server encountered an error.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194393053
Currently we determine fee type from the fee extension description by checking if the format string of the FeeType contains the description we received. The formatting string for EAP is "Early Access Period, fee expires: %s", so the fee description generated by a domain check command, like "Early Access Period, fee expires: 2022-03-01T00:00:00.000Z", is not recognized as EAP.
This CL adds the ability to add arbitrary extra description strings to the FeeType for a description to match against. It also changes the match to "the given description contains any of the strings from the list of format string plus extra description strings".
For EAP, we added an extra description string "Early Access Period", so any fee description that contains "Early Access Period" will be matched to EAP FeeType, including the specific description (that contains the expiry time) that we send in a domain check.
Also improved error message on multiple fee type matching.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194149162
Per EPP RFC 5730, the <clTRID> element is optional. However, we weren't handling
it not being specified in asynchronous contact/host deletions because we were
adding it directly as a parameter value on a task, which does not allow null and
thus threw a NullPointerException.
This fixes handling for nulls (the parameter isn't set at all) and adds a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194123259
This verifies anchor tenant creates by registrars are for exactly 2 years, to simplify billing down the line (anchor tenants get 2 years of free domain creates).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193931858
This adds a parameter to control invoice reporting, which defaults to false for
now (since we plan on manually adjusting the invoice next month).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193708323
This makes it more clear than an operation is actually pending, rather
than nothing changing after pressing "Y" to run the command.
The prompt now looks like when you press "Y" and "N", respectively:
Perform this command? (y/N): Running ...
Perform this command? (y/N): Command aborted.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193521084
Locales are weird. Even if all our character individually are just 0-9a-z_,
different locales might still convert them differently to upper/lower cases...
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193512312
The autoscaling manifest doesn't really change much from environment to environment. It makes sense to move it to the service yaml file, which is not environment dependent.
Also enhanced bashrc function to update the deployment manifest when deploy the proxy to alpha
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193407184
1) Clean up alpha config to only allow alpha proxy, removing test proxy client id.
2) Add sandbox service account client id to sandbox config.
3) Add sandbox config to nomulus and proxy, remove TEST environment, which is not being used anymore. (Test now uses LOCAL.)
4) Add sandbox kubenetes config
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193400909
1) Change annealing target to watch for sandbox terraform config instead of test.
2) Delete terraform config for test project, as this project will be turned down.
3) Do not ask annealing to watch for alpha project terraform config, as we intend to change alpha regularly and manually.
4) Make terraform output display both service account email and client id.
5) Change canary node ports to 3100X, as 4000X is out of range for kubernetes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193383457
Canary proxies are not receiving real traffic but can be useful when testing Nomulus deployment (probers will probe canary proxy and compare metrics with production proxy). This CL added a separate load balancer for a canary proxy, running on the same clusters as production proxy.
The canary proxies have their own IP addresses, but are not assigned domain names. Probers will directly connect to these endpoints by IP.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=193234937
This gets around a bug in Spinnaker where the namespace, if missing in the manifest, is set to "spinnaker".
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192825895
This allows a registrar to check domain availability even if the fee extension is not set. To obtain the premium price or create the domain, the fee extension is still needed.
PS: I believe the previous tests are erroneous anyway. It is the presence of fee extensions in the check request that we were looking for. Checking if the extension is declared during login has no bearing on the results.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192778137
Five per minute just isn't working well enough on environments with lots of
entities (e.g. alpha and sandbox right now), and there doesn't seem to be a
real need to enforce such a low throttle. The mapreduce queue, for instance,
has 500/s (effectively no throttle).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192474962
The optional code has been around for a while, we can get rid of it now.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192344612
Because of the objectify cache, after the first "read, change, write" of the
first mutation - the second mutation would "read" the original value, which
would then fail to be written with the error "Entity changed since init()".
This was specifically seen in the "UpdateRegistrarCommand", but likely affected
other commands as well.
Clearing the cache before each command solves this issue.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192328213
This hard-deletes all contacts and hosts owned by a specific set of registrar
client IDs, currently just "proxy".
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192325211
This makes them consistent with every other command, which uses the format
verb_noun.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192301468
This also removes RDE tasks that shouldn't/can't run on non-production environments, like upload/reporting.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192177779
This is in preparation for merging and then removing
RegistrarPremiumPriceAckAction.
This includes:
test that the data the UI sent isn't stale
---------------------------------------------
Our system is "read, modify, write". However, if between the "read" and the "write" someone else changed the registry, my write will undo their change even if I didn't touch any of their fields.
To solve that - we use the "lastUpdateTime" timestamp of the registrar. the UI reads it with the rest of the data, and sends it back on "write". We will now make sure the registrar currently in datastore has the same timestamp.
support premium-price-ack flag
---------------------------------
Add support for reading and writing this flag. We still won't be using it - that's in a followup CL, but we support it.
support changing the URL
------------------------
Add changing the URL in the UI, under the "whois" section
Will replace the Ack endpoint with this (and remove that endpoint) in a followup CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192154078
This is accomplished by making all non-mutating commands function with dry run set
to true, which also has the pleasurable side effect of not prompting for dry-run
mutating commands either, which also do nothing different/special on the second
run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192149150
We don't want people to accidentally run commands on prod thinking they were on
Alpha / Sandbox.
To do that - we add 2 safeguards:
1) when on prod, the shell has a strong RED "PRODUCTION" in the commandline, while on alpha/sandbox it's green.
2) if a prod shell is idle for > 1h, it exits. So don't accidentally use a prod shell from a long time ago.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191931731
Currently the example whitelist IP is 1.1.1.1/24, which is illegal. Changed to
1.1.1.0/24, which is legal
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191900036
After writing a flag on the shell, pressing "tab" will print out the
documentation for that flag.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191899137
This allows us to not ship the proxy with certificates/private keys. The secret is still encrypted by KMS. Reading the secret only happens once when the first EPP request comes in, which should not incur any tangible performance penalty.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191771680
This allows us to not obtain a certificate and encrypt it with KMS when running the proxy locally during development.
Also updated FOSS build dagger version.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191746309
The registrar security console failed because it assumed the email is a
required field for the registrar, but it isn't (at least - create_registrar
doesn't require an email, and update_registrar lets you remove the email).
Fixed by allowing it to *remain* unset if it was unset originally, but if it was set - it's required.
There are more fixes needed, but they aren't related to the email, so they will wait for the next CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191623034
This also reduces the interval of the commitLogCheckpoint cron job to once
every three minutes, as this job needs to load all commit log bucket entities.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191613858
