Commit graph

1966 commits

Author SHA1 Message Date
mcilwain
3e34f61b8c Smear out DNS updates over time when updating DNS for all domains
These updates can generally ran slowly in the background (i.e. they're low
priority), and shouldn't clog up the system to the point where they slow down
live incoming customer-requested updates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250732171
2019-06-06 10:44:27 -04:00
gbrodman
ba23e3efaa Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
gbrodman
5c127b8020 Modify DeleteAllocationTokensCommand to have the same input structure as UpdateATC
It's dangerous to have a blank prefix delete all tokens and this allows for some code unification.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250493453
2019-05-30 12:52:21 -04:00
guyben
4d4926bac7 Fix bazel breakage caused by []
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)

Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
2019-05-30 12:52:21 -04:00
guyben
a457b57621 Conform to RDAP Response Profile 15feb19
This is only about the Response Profile, not the Technical Implementation guide.

The Response Profile can be found at https://www.icann.org/en/system/files/files/rdap-response-profile-15feb19-en.pdf

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250277559
2019-05-30 12:52:21 -04:00
guyben
65b5a9d063 Increase delay between attempts of rde staging
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.

Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.

8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
2019-05-30 12:52:21 -04:00
jianglai
6b4bdf8d9a Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-30 12:52:21 -04:00
mmuller
e43fffe4b2 Increase NS record default TTLs to 3 hours
Per further discussion on b/130533696, we've decided to increase these so
tenants can have greater resilience to an outage in our DNS servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248546853
2019-05-21 11:42:39 -04:00
gbrodman
fa8ac2cf09 Add a small prompt to the UpdateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248523681
2019-05-21 11:37:22 -04:00
jianglai
f6bac5297b Add Google Analytics to registrar console
To support the open source community, which may want to use different analytics services, we implement a soy template for analytics services that:

1) Does not require users to implement Google Analytics
2) Allows users to add their own analytics code to `Analytics.soy`
3) Gives users the flexibility to pass as much or as little static configuration to their custom analytics code as needed.
4) Ensures that users can merge upstream Nomulus code in the future without having to delete their custom analytics implementations
5) Does not allow code to be injected from configuration, which Soy as a framework actively discourages.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248340081
2019-05-21 11:35:34 -04:00
gbrodman
f635047748 Add a bit more verification text when unrenewing domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248150595
2019-05-21 11:33:47 -04:00
guyben
52c453813e Save the RDAP request time globally instead of passing it around
Also removed the rdapWhoisServer value, as it's just null and will likely stay that way (it isn't mentioned in the RDAP response profile)

If it'll ever become required, we can add it back.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247643981
2019-05-13 12:44:55 -04:00
gbrodman
d7b443d284 Remove the scrap AllocationToken command and OnLoad method
I have resaved all the tokens and verified in BigQuery that there are no
remaining null tokenTypes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247425132
2019-05-13 12:39:47 -04:00
guyben
d29d43a9bb Inject RdapAuthorization
We currently create it from injected arguments, and pass it to every function.

Instead, we just create a provider for it and inject it where needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247072517
2019-05-13 12:34:53 -04:00
guyben
7202a38f8b Reimplement the RDAP Json creation using Jsonables
Currently we try to reimplemnet the same behavior of the existing code as much
as possible.

We only fix issues that go against the RFC7483, but we don't yet update the
code to follow the latest (15feb19) RDAP Response Profile. That will require a
much bigger change especially for the test files, so it'll wait for a followup
CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246948018
2019-05-13 12:31:32 -04:00
gbrodman
85f04799db Add a command to update allocation tokens
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824258
2019-05-06 16:56:33 -04:00
gbrodman
74b5b407fa Validate on-load that an AllocationToken can be used
Check the timing (that is, whether or not we're in a promotion), the allowed registrar client IDs, and the allowed TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824080
2019-05-06 16:54:55 -04:00
gbrodman
f7e51549d6 Add promotional fields in GenerateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246579125
2019-05-06 16:47:25 -04:00
gbrodman
b9f9083ea0 Add a scrap command to re-save all allocation token entities
This will populate the token type to be SINGLE_USE for existing tokens that
don't have a type.

Modeled after the ResaveEntitiesCommand, but modified so that we load all AllocationTokens (rather than having to provide a list of entity IDs on the command line)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246572756
2019-05-06 16:45:46 -04:00
weiminyu
b211d8e1ba Update to Truth 0.44 in Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246548162
2019-05-06 16:40:54 -04:00
guyben
219f59e3b0 Fix Kokoro failure
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246411007
2019-05-06 16:34:27 -04:00
guyben
336a34b95a Add Jsonable and AbstractJsonableObject for easier RDAP object building
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246345611
2019-05-06 16:27:59 -04:00
jianglai
abc508712d Restore RemoteApiOptions method accessibility after use
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246161567
2019-05-06 16:24:44 -04:00
guyben
9f017edc2e Simplify some of the RDAP Action classes
Overriding getter methods to change values is a bit overkill when these values
are static (don't change based on internal state).

Just setting them in the base class' constructor is simpler.

Also, we can read the PATH of an Action based on the Annotation instead
returning it manually for each Action.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246135754
2019-05-06 16:23:05 -04:00
mmuller
d2a67d029e Change default TTLs on all records to 1 hour.
We need this as a precursor to running refreshDnsForAllDomains to change our
TTLs from 3 minutes to an hour (see the associated bug for details).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246011747
2019-05-06 16:19:48 -04:00
gbrodman
8af2afc616 Use a potential discount in the AllocationToken when determining domain create price
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245458027
2019-04-27 00:08:57 -04:00
gbrodman
29b9f04aaa Fix up a comment based on investigation into billing events
Please see the linked bug for more context--I believe this is correct based on what I've observed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245423464
2019-04-27 00:05:49 -04:00
gbrodman
4895f455d8 Add additional tests for the AllocationToken status transitions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292932
2019-04-26 23:59:32 -04:00
gbrodman
6ee34a79b1 Store a reference to an allocation token in the OneTime billing event
We will need to be able to find all redemptions associated with a particular token and this will allow us to do that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292865
2019-04-26 23:57:56 -04:00
gbrodman
2b12ca42f5 Use the registrar client ID and abuse email address in Spec11
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244899045
2019-04-26 23:45:07 -04:00
gbrodman
9f979790e4 Use stream-collect rather than transformValues in Spec11 collection
As it turns out, using Maps::transformValues does not allow us to change the
resulting map--calling Map::put throws an UnsupportedOperationException. As a
result, we have to do this roundabout stream-collect to do a group-by.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244852373
2019-04-26 23:41:55 -04:00
jianglai
619c8a984e Update proxy release pipeline following Spinnaker 1.13 release
The image name in the manifest needs to be the same as the name that Spinnaker trigger catches. With the new release, Spinnaker now correctly recognizes gcr.io/${PROJECT_ID}/proxy as the image name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244845037
2019-04-23 10:44:44 -04:00
mcilwain
b46a6b6d55 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
jakubvrana
18d1654dbf Remove references to |blessStringAsTrustedResourceUrlForLegacy in <link href>.
This directive will be deleted in the future, this change prepares for it.

More information: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243847668
2019-04-16 17:24:35 -04:00
mcilwain
68488f0833 Only send registrar update notification emails to primary contacts
The test changes are perhaps a little bit more involved than expected, because I
had to add a second RegistrarContact which had knock-on effects on other tests.
This does make the other tests better though, in that we're now testing registrars
with multiple contacts (we weren't much before).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243827803
2019-04-16 14:11:06 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
gbrodman
74971144d2 Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
jianglai
77eb7610e8 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
cfee7e7fd5 Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
gbrodman
565c4bcf50 Use a multimap to index the Spec11 threat matches
Collecting by key leads to exceptions if there are multiple client IDs with the
same email address (if we group by client ID in the pipeline). Using
Multimaps::index means that if we're grouping by email, all matches with the
same email get concatenated together

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242858112
2019-04-11 14:58:08 -04:00
guyben
896cc252e7 Add abuse entity to registrar entities
From to the RDAP response profile:

2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members

Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
2019-04-11 14:54:28 -04:00
mcilwain
d5b540dfe7 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
shicong
5e310f8fcd Fix Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242472456
2019-04-11 14:47:31 -04:00
guyben
15e54f2803 Show only the last of each event type in RDAP domain response
We also ignore events that happened before the domain was created (for example, in a previous incarnation of the same domain name) and we set the last changed event to be the later of the last EPP change and any other event that happened before "now".

From RDAP response profile
2.3.2 The domain object in the RDAP response MAY contain the following events:
2.3.2.3 An event of *eventAction* type *transfer*, with the last date and time that the
domain was transferred. The event of *eventAction* type *transfer* MUST be omitted if
the domain name has not been transferred since it was created.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242461310
2019-04-11 14:45:46 -04:00
shicong
72b9ca6894 Add required dependency for RemoteApiInstaller
We got 500 Internal Server Error when calling GAE RemoteApi in Nomulus
tool. The root cause is we missed the dependency described in this
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242166362
2019-04-11 14:43:46 -04:00
mmuller
c8c7a043bd Change text of registrar change notification e-mail
Since we're now sending these to actual registrars, change the text to make it
a little bit clearer what's going on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241992717
2019-04-05 11:54:45 -04:00
shicong
4240583af7 Improve exception message when TMDB not available
We saw a few IOExceptions recently which indicated that TMDB may
be down at that time. However, these IOExceptions didn't have
any message to clearly describe that. This CL is to improve
the exception message so next time we don't have to look at the
source code to figure out what happened.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241984023
2019-04-05 11:53:08 -04:00
gbrodman
31c2ea1b3d Clarify that we are actually using the client ID in Spec11
The Registrar object's @ID is on its clientId field, so that's what we're actually using here, not the name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241936005
2019-04-05 11:51:28 -04:00
shicong
387042bf3a Enable flow documentation in external build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241934689
2019-04-05 11:49:49 -04:00
mcilwain
9b80b31917 Make RDE report generation correctly handle DISABLED registrars
This is a follow-up to [] We can't set registrars as DISABLED until
this is deployed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241767990
2019-04-05 11:46:29 -04:00