Commit graph

997 commits

Author SHA1 Message Date
mountford
2b7f78db98 Change registrar console login code in preparation for removing requireLogin
We are going to remove the requireLogin attribute from the action attribute, because it is specific to the UserService API. This is used by four actions:

ConsoleUIAction
RegistrarSettingsAction
RegistrarPaymentSetupAction
RegistrarPaymentAction

Instead, these four actions will now check the login status directly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159562335
2017-06-21 09:56:10 -04:00
nickfelt
17697388b8 Fix stale naming in KmsModule binding for KmsConnection
And also change it to use @Binds.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159506945
2017-06-21 09:54:36 -04:00
nickfelt
c41cf9142a Remove FixDomainTransferDataCommand
Scrap tool is no longer needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159500743
2017-06-21 09:53:03 -04:00
Ben McIlwain
fd1051f90a Remove temporary @OnLoad used for setting period in DomainApplication
Now that the registration period has been added to DomainApplication, we
can remove this @OnLoad that was populating it for objects that were
missing the period.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159464438
2017-06-21 09:51:28 -04:00
nickfelt
fc78fd7d4a Update transfer data fixing command to fix one more domain
I happened to notice that one domain affected by b/33289763 had existing transfer data, so in that case the data got overwritten to look like a server cancellation (a year after the transfer request, so obviously wrong) rather than creating invalid transfer data, and hence wasn't picked up in the previous fixes.  See b/33289763#comment32 for more details.

This CL renames the RemoveDomainTransferDataCommand used to fix the errors to FixDomainTransferDataCommand and changes its behavior to instead reset the TransferData to a server-approval result.  It also changes it to load domains by ROID because loadByForeignKey() fails for deleted domains.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159273225
2017-06-21 09:49:53 -04:00
jianglai
d1ef4b9c37 Allow partial update of billing account map
When doing update_registrar, it is now possible to only specify the currencies and the account ids that need updating.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159262119
2017-06-21 09:48:19 -04:00
mountford
7d2f53a6fe Remove xsrfScope and xsrfProtection authentication attributes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159121132
2017-06-21 09:46:40 -04:00
Ben McIlwain
580c41f2d6 Make the superuser flag bypass TLD access checks
The --superuser command in the nomulus command-line tool should be
bypassing checks on whether the passed-in registrar client ID has access
to the TLD in question, but currently it is not.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158974462
2017-06-14 10:43:50 -04:00
jianglai
3a02e6fb11 Add fine level logging methods
We currently only have one method for fine level logging. Add another three so that fine level logging can be called just like other levels.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158760803
2017-06-14 10:40:49 -04:00
tbreisacher
f3f5f7652b Put goog.forwardDeclare()s in alphabetical order
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158710116
2017-06-14 10:39:17 -04:00
mcilwain
f5f383dc38 Temporarily add null handling for autorenew billing events
These shouldn't ever be null, but we have some bad data in production
for prober TLDs left over from the Registry 2.0 transition. Ignoring
null values here is required to finish cleanup for this old data, which
currently cannot even be deleted because it's throwing an NPE when
trying to update these values.

This commit will be reverted after the bad data is cleaned up, likely
sometime next week.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158546840
2017-06-14 10:36:13 -04:00
mcilwain
7002026da5 Finish async task metrics data migration
Now that we've been live with the new code, all of the old tasks
have been processed, and the fallback code to handle the situation
of those fields not being specified is not necessary.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158530336
2017-06-14 10:34:41 -04:00
mcilwain
0fdde1cc9e Change FibonacciFitter to just be a utility method
It turns out that StackdriverWriter uses reflection on the class of
the DistributionFitter instance, so rather than giving it custom
handling for FibonacciFitters, it's easier to turn FibonacciFitter
into a single utility method that takes advantage of existing support
for CustomFitters.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158190147
2017-06-14 10:31:29 -04:00
mountford
e8eabe01cf Update WHOIS output format to reflect CL&D and PDT requirements
PDT testing revealed a couple ways in which our WHOIS output was non-compliant. First, the Consistent Labeling & Display policy dictates that the contact IDs must be ROIDs. See rule 11 in https://www.icann.org/resources/pages/rdds-labeling-policy-2017-02-01-en. Second, PDT tests expect that a WHOIS response will treat missing values either by omitting the line entirely, or by including the line with a blank value, but not both. So this is legal:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568
Fax Number Ext:

and this is legal:

Phone Number: 123-4567
Fax Number: 123-4568

but this is not:

Phone Number: 123-4567
Phone Number Ext:
Fax Number: 123-4568

In the last example, one extension line is present with a blank value, while the other extension line is omitted. We cannot do both. Therefore, we should update our code to omit lines with no value. Since we can't guarantee that we will always emit all lines that the parse might expect to see, it is safe to use the policy of omitting lines with no value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158184150
2017-06-14 10:29:55 -04:00
cgoldfeder
ae039aa0d8 Remove all vestiges of memcache
Memcache is already off but now it's not in the code anymore.

This includes removing domain creation failfast, since that is actually
slower now than just running the flow - all you gain is a non-transactional
read over a transactional read, but the cost is that you always pay that
read, which is going to drive up latency.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158183506
2017-06-14 10:28:24 -04:00
guyben
75bb1aacdc Update cloudkms_java with the non-beta version
Changed [] to use v1 instead of v1beta1, and replaced v1beta1 with v1 in all the java files.

If there is special build rules for open-source etc. that also need to be updated, or non "TAP-able" tests that need to be run, please check and see if they are OK.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157895888
2017-06-05 18:17:09 -04:00
mountford
3bbf356497 Fix bug in registrar contact nomulus command
The command was set up such that an update without any contact types specified would clear out the list, instead of leaving them unchanged, as it should.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157766429
2017-06-05 18:17:09 -04:00
mountford
6cc3ee5e15 Add option to registrar_contact command to modify the visible_in_domain_whois_as_abuse flag
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157588734
2017-06-05 18:17:09 -04:00
mountford
009522d078 Change auth settings on RdeStagingAction
[] opened up the settings on RdeStaging, in order to make it usable by the nomulus tool. But in retrospect, we think that all we needed to do was support the POST method, not loosen the auth settings, since nomulus invokes RdeStaging via a task queue. Removing the looser auth settings will bring this action into line with other backend actions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157262595
2017-06-05 18:17:09 -04:00
mountford
bc63d80a5e Change auth settings on some tools service actions
This brings the affected actions into line with the settings on other similar actions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157259842
2017-06-05 18:17:09 -04:00
mountford
e46937beac Add JS deprecation error suppression
Our build got broken by the deprecation of goog.structs.Map in [] This is a quick fix.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157236744
2017-06-05 18:17:09 -04:00
mmuller
b733443eab Create nomulus tool restore command
Create a nomulus tool restoreCommitLogs command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157103177
2017-06-05 18:17:09 -04:00
mountford
ddb0f5692e Add golden files for request component routing maps
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157026188
2017-06-05 18:17:09 -04:00
mcilwain
bb67841884 Add metrics for async batch operation processing
We want to know how long it's actually taking to process asynchronous
contact/host deletions and DNS refreshes on host renames. This adds
instrumentation. Five metrics are recorded as follows:

* An incrementable metric for each async task processed (split out by
  type of task and result).
* Two event metrics for processing time between when a task is enqueued
  and when it is processed -- tracked separately for contact/host
  deletion and DNS refresh on host rename.
* Two event metrics for batch size every time the two mapreduces are
  run (this is usually 0). Tracked separately for contact/host deletion
  and DNS refresh on host rename.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157001310
2017-06-05 18:17:09 -04:00
mcilwain
1adeb57fea Add a Fibonacci fitter for metrics bucketing
A Fibonacci fitter is useful in situations where you want more precision on the
low end than an ExponentialFitter with exponent base 2 provides without the
hassle of dealing with non-integer boundaries, such as would be created by an
exponential fitter with a base of less than 2. Fibonacci fitters are ideal for
integer metrics that are bounded across a certain range, e.g. integers between 1
and 1,000.

This also cleans up some unit test comments.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156773367
2017-05-23 17:22:49 -04:00
mmuller
e6af34301d Move restore from backend to tools
Move the "restoreCommitLogs" command from the backend module to the tools
module so it's easier to access with nomulus.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156768389
2017-05-23 17:22:49 -04:00
nickfelt
9ee9229456 Fix message for SuperordinateDomainInPendingDeleteException
The StatusProhibitsOperationException superclass constructor just wants a raw message to return to the client, so we should pass it an explanation rather than just the superordinate domain name.  In fact, I think the superordinate domain name isn't really necessary since it should be obvious from the hostname in the create/update request what the superordinate domain would be.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156747185
2017-05-23 17:22:49 -04:00
mountford
fb21946366 Fix parameter problem in GenerateEscrowDepositCommand
It was setting the TLD and watermark date parameters as comma-separated lists. What it really needs to do is set the parameters multiple times, once for each value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156741051
2017-05-23 17:22:49 -04:00
mcilwain
42513f41ca Remove migration @OnLoad for EppResource StatusValue LINKED
This was needed to correct bad data (LINKED status values on EppResources). The code has been fixed to no longer persist LINKED on any resources and I ran a resave all action yesterday to remove all persisted LINKED status values, so the migration @OnLoad can be safely removed now.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156580334
2017-05-23 17:22:49 -04:00
shikhman
2b1ab579dc Move metrics shutdown behavior to a ShutdownHook
HttpServlet#destroy() is not called on instance shutdown, so the metrics code was never shut down correctly before.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156360866
2017-05-23 17:22:49 -04:00
jianglai
3f5735ed4b Add Carriage Return in WHOIS disclaimer text
WHOIS disclaimer is read from the config yaml file directly and the line breaks
within the text does not contain carriage return, which ICANN requires. This
CL fixes the non-compliance.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156359526
2017-05-23 17:22:49 -04:00
shikhman
413a634533 Automated g4 rollback of changelist 156106183.
*** Reason for rollback ***

This change is crashing alpha

*** Original change description ***

Change metrics ThreadFactory to produce daemon threads

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156336467
2017-05-17 15:19:10 -04:00
mountford
7d5a67f5a2 Localize a variable
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156187775
2017-05-17 12:27:43 -04:00
cgoldfeder
c9d7e75946 Cache Registrars in memory
This replaces the memcache caching, which we think is overall a bad idea.
We load all registrars at once instead of caching each as needed, so that
the loadAllCached() methods can be cached as well, and therefore will
always produce results consistent with loadByClientIdCached()'s view of the
registrar's values. All of our prod registrars together total 300k of data
right now, so this is hardly worth optimizing further, and in any case this
will likely reduce latency even further since most requests will be
served out of memory.

While I was in the Registrar file I standardized the error messages for incorrect
password and clientId length to be the same format, and cleaned up a few
random things I noticed in the code.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156151828
2017-05-17 12:26:11 -04:00
blickly
9a48aae107 Add a few missing type annotations to braintree externs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156148567
2017-05-17 12:24:26 -04:00
mountford
74a0defef3 Add the ability to generate RDE deposits in lenient mode
We will not want to run this under normal circumstances, but in cases (such as PDT testing in sandbox) where it's desirable to generate an escrow deposit even if it isn't technically valid XML, this gives us that option. Manual-mode RDE generation is also changed so that, if no watermark date is specified, it defaults to the previous midnight, to better support running of RDE in sandbox to catch data problems.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156108295
2017-05-17 12:22:49 -04:00
shikhman
e1f4df86bd Change metrics ThreadFactory to produce daemon threads
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156106183
2017-05-17 12:21:12 -04:00
mmuller
5ff56ac7a0 Make "throws" clause use IOException
Now that we've updated the oauth java6 package in the open source build, we
can safely replace "Exception" with "IOException" in the throws clause of the
method that calls AuthorizationCodeInstalledApp.authorize().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156057190
2017-05-17 12:17:40 -04:00
mmuller
fc5546ad58 Add latest oauth/{java6,jetty} and fix repo procedure
Upgrade to the latest versions of omit_com_google_oauth_client_java6 and
omit_com_google_oauth_client_jetty and fix the broken stuff in our local
mirroring procedures.  Specifically:

- Document that users need to add a GCS url.
- Document how to run the check_bazel_deps script.
- Document that we don't use the in-piper repo.
- Remove the broken ibiblio link for the "truth" package.
- Delete sync_bazel_deps (which doesn't currently work, isn't python-style
  compliant and is no longer relevant since we don't use the in-piper repo)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155898485
2017-05-17 12:12:56 -04:00
jianglai
35624ace08 Require currency specific billingAccountId for registrar
For the currency specific billing account ids to work properly, any REAL registrar that is allowed to register domains in a certain TLD must have a billing account id for the currency of that TLD.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864711
2017-05-17 12:09:46 -04:00
jianglai
6e6a340113 Allow multiple domain creation in nomulus tool
This helps creating test domains. Also fixed a bunch of bad test cases.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155864065
2017-05-17 12:08:10 -04:00
larryruili
b08e317505 Add scrap tool for populating required registrar fields
This tool finds all registrars that have a null name, or a null localizedAddress field, and populates them as follows:

- The registrar's name is updated to its clientId
- The registrar's empty localizedAddress fields are replaced with fake data

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155739946
2017-05-17 12:01:49 -04:00
nickfelt
6b6b8d25c7 Add type to RegistrarData billing BigQuery view
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155682529
2017-05-17 12:00:09 -04:00
guyben
c2094eda40 Make EncryptEscrowDepositCommand a RemoteApiCommand
Moving to KmsKeyring, any command that uses the Keyring must be a
RemoteApiCommand to work.

TESTED: successfully encrypted a file with no logged errors.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155673003
2017-05-17 11:58:35 -04:00
mcilwain
6e3a99a16b Tune max # of frontend instances down to 4 in sandbox and alpha
My continuing investigations into necessary resources for running
our environments seems to indicate that four instances should be
sufficient for our purposes.  If it's not, we can always revert.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155607688
2017-05-17 11:55:30 -04:00
nickfelt
2b78571c72 Replace literal 'latest_snapshot' in invoice SQL query
This should be %SOURCE_DATASET% so that it's adjusted appropriately based on the parameter value.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155582718
2017-05-17 11:53:56 -04:00
mmuller
70e3271319 Add "toTime" parameter to RestoreCommitLogsAction web form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155500640
2017-05-17 11:46:18 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglai
5313ca58d6 Setup auth settings in PublishDetailReports @Action to support tool access
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155459413
2017-05-17 11:43:09 -04:00
jart
d419f6ac11 Export annotations from Guava
These annotations are part of guava's API, and they cannot safely be
optimized off the compile-time classpath.

This works around google/error-prone#615.

See []
H/T @cushon

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155433554
2017-05-17 11:40:02 -04:00