Commit graph

1973 commits

Author SHA1 Message Date
guyben
4110cae814 Link the Registrar's RDAP server from RDAP domain replies
To do this we add a field of "rdapServers" in the Registrar object. Currently, we can only set this field manually, but a subsequent CL will add a cron-job to read these values from the ICANN servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252438618
2019-06-12 13:04:40 -04:00
weiminyu
7c64992c73 Cloned from CL 251456914 by 'g4 patch'.
Original change by cpovirk@cpovirk:rosie251284456-0055_Rosie:31511:citc on 2019/06/04 09:48:27.

Update to Truth 0.45, and address deprecations.

Renames may include:
- containsAllOf => containsAtLeast
- containsAllIn => containsAtLeastElementsIn
- isSameAs => isSameInstanceAs
- isOrdered => isInOrder
- isStrictlyOrdered => isInStrictOrder

The other major change is to change custom subjects to extend raw Subject instead of supplying type parameters. The type parameters are being removed from Subject. This CL will temporarily produce rawtypes warnings, which will go away when I remove the type parameters (as soon as this batch of CLs is submitted).

Some CLs in this batch also migrate calls away from actualAsString(). Its literal replacement is `"<" + actual + ">"` (unless an object overrides actualCustomStringRepresentation()), but usually I've made a larger change, such as switching from an old-style "Not true that..." failure message to one generated with the Fact API. In that case, the new code usually contains a direct reference to this.actual (a field that I occasionally had to create). Another larger change I sometimes made is to switch from a manual check-and-fail approach to instead use check(...). And sometimes I just remove a withMessage() call that's no longer necessary now that the code uses check(...), or I introduce a check(...) call. (An assertion made with check(...) automatically includes the actual value from the original subject, so there's no need to set it again with withMessage().)

Finally, there's one CL in this batch in which I migrate a Correspondence subclass to instead use Correspondence.from.

END_PUBLIC

If this is too many changes at once, let me know, and I can split it up and/or explain further. In the past, I've erred on the side of sending separate CLs for each change, which has required some owners to manually reapply each one, so now I'm trying this way.

Thanks again for your patience. There is an outside chance that this will be the last CL I send you before Truth 1.0 -- but certainly no promises :)

More information:
  Renames:
    []
    []
    []
  Removing type parameters: []
  Migration from old fail*(...) methods to new ones and to check(...): []
  Changes that replace assert*(...) with introduce check(...): []
  Correspondence subclass to Correspondence.from: []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251921007
2019-06-07 11:46:44 -04:00
guyben
ca1d525e28 Conform to RDAP Technical Implementation Guide
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251864499
2019-06-07 11:43:09 -04:00
mcilwain
55dc735ba0 Export PollMessage entities to BigQuery
It would be nice to be able to query them using SQL in order to debug
some potential issues registrars might be facing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251863662
2019-06-07 11:41:22 -04:00
mcilwain
ff6d327183 Allow reserved domains to be created during quiet periods
We'll use this for LRP. This is safe because we must specifically reserve a
domain by including it in a reserved list, create an associated allocation
token, and distribute that token, before a create would succeed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250901144
2019-06-06 10:53:27 -04:00
gbrodman
79bcb227be De-duplicate flow documentation error reasons on a per-code basis
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250894847
2019-06-06 10:51:43 -04:00
guyben
e489539df4 Enable more DNS update metrics on production
They are useful for viewing and diagnosing problems, and are not that
expensive.

We aren't enabling all the metrics though - the whole "per batch / per domain"
performance indicators are still disabled

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250876554
2019-06-06 10:49:55 -04:00
mcilwain
987a648296 Smear out DNS updates over time when updating DNS for all domains
These updates can generally ran slowly in the background (i.e. they're low
priority), and shouldn't clog up the system to the point where they slow down
live incoming customer-requested updates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250732171
2019-06-06 10:44:27 -04:00
gbrodman
d2f955a488 Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
gbrodman
6a272bc8c6 Modify DeleteAllocationTokensCommand to have the same input structure as UpdateATC
It's dangerous to have a blank prefix delete all tokens and this allows for some code unification.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250493453
2019-05-30 12:52:21 -04:00
guyben
44ccd45439 Fix bazel breakage caused by []
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)

Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
2019-05-30 12:52:21 -04:00
guyben
c79e0ea670 Conform to RDAP Response Profile 15feb19
This is only about the Response Profile, not the Technical Implementation guide.

The Response Profile can be found at https://www.icann.org/en/system/files/files/rdap-response-profile-15feb19-en.pdf

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250277559
2019-05-30 12:52:21 -04:00
guyben
7cc5c16ab9 Increase delay between attempts of rde staging
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.

Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.

8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
2019-05-30 12:52:21 -04:00
jianglai
243fce84ce Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-30 12:52:21 -04:00
mmuller
f18f389b03 Increase NS record default TTLs to 3 hours
Per further discussion on b/130533696, we've decided to increase these so
tenants can have greater resilience to an outage in our DNS servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248546853
2019-05-21 11:42:39 -04:00
gbrodman
8f9f384406 Add a small prompt to the UpdateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248523681
2019-05-21 11:37:22 -04:00
jianglai
b12a462f5e Add Google Analytics to registrar console
To support the open source community, which may want to use different analytics services, we implement a soy template for analytics services that:

1) Does not require users to implement Google Analytics
2) Allows users to add their own analytics code to `Analytics.soy`
3) Gives users the flexibility to pass as much or as little static configuration to their custom analytics code as needed.
4) Ensures that users can merge upstream Nomulus code in the future without having to delete their custom analytics implementations
5) Does not allow code to be injected from configuration, which Soy as a framework actively discourages.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248340081
2019-05-21 11:35:34 -04:00
gbrodman
7f69ebc5d9 Add a bit more verification text when unrenewing domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248150595
2019-05-21 11:33:47 -04:00
guyben
a0040c5eda Save the RDAP request time globally instead of passing it around
Also removed the rdapWhoisServer value, as it's just null and will likely stay that way (it isn't mentioned in the RDAP response profile)

If it'll ever become required, we can add it back.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247643981
2019-05-13 12:44:55 -04:00
gbrodman
aa83472393 Remove the scrap AllocationToken command and OnLoad method
I have resaved all the tokens and verified in BigQuery that there are no
remaining null tokenTypes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247425132
2019-05-13 12:39:47 -04:00
guyben
3b8a8892bb Inject RdapAuthorization
We currently create it from injected arguments, and pass it to every function.

Instead, we just create a provider for it and inject it where needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247072517
2019-05-13 12:34:53 -04:00
guyben
bdc41edd34 Reimplement the RDAP Json creation using Jsonables
Currently we try to reimplemnet the same behavior of the existing code as much
as possible.

We only fix issues that go against the RFC7483, but we don't yet update the
code to follow the latest (15feb19) RDAP Response Profile. That will require a
much bigger change especially for the test files, so it'll wait for a followup
CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246948018
2019-05-13 12:31:32 -04:00
gbrodman
3483f348dc Add a command to update allocation tokens
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824258
2019-05-06 16:56:33 -04:00
gbrodman
df7e9a1225 Validate on-load that an AllocationToken can be used
Check the timing (that is, whether or not we're in a promotion), the allowed registrar client IDs, and the allowed TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824080
2019-05-06 16:54:55 -04:00
gbrodman
e9d220e6f3 Add promotional fields in GenerateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246579125
2019-05-06 16:47:25 -04:00
gbrodman
c9ee5c3fb3 Add a scrap command to re-save all allocation token entities
This will populate the token type to be SINGLE_USE for existing tokens that
don't have a type.

Modeled after the ResaveEntitiesCommand, but modified so that we load all AllocationTokens (rather than having to provide a list of entity IDs on the command line)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246572756
2019-05-06 16:45:46 -04:00
weiminyu
e6e6303b9e Update to Truth 0.44 in Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246548162
2019-05-06 16:40:54 -04:00
guyben
5962c9c561 Fix Kokoro failure
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246411007
2019-05-06 16:34:27 -04:00
guyben
e1926dbc57 Add Jsonable and AbstractJsonableObject for easier RDAP object building
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246345611
2019-05-06 16:27:59 -04:00
jianglai
c45d3b843c Restore RemoteApiOptions method accessibility after use
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246161567
2019-05-06 16:24:44 -04:00
guyben
90c53152bf Simplify some of the RDAP Action classes
Overriding getter methods to change values is a bit overkill when these values
are static (don't change based on internal state).

Just setting them in the base class' constructor is simpler.

Also, we can read the PATH of an Action based on the Annotation instead
returning it manually for each Action.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246135754
2019-05-06 16:23:05 -04:00
mmuller
bbdf3db63e Change default TTLs on all records to 1 hour.
We need this as a precursor to running refreshDnsForAllDomains to change our
TTLs from 3 minutes to an hour (see the associated bug for details).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246011747
2019-05-06 16:19:48 -04:00
gbrodman
70c7e6c224 Use a potential discount in the AllocationToken when determining domain create price
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245458027
2019-04-27 00:08:57 -04:00
gbrodman
274b331522 Fix up a comment based on investigation into billing events
Please see the linked bug for more context--I believe this is correct based on what I've observed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245423464
2019-04-27 00:05:49 -04:00
gbrodman
312c1a46ab Add additional tests for the AllocationToken status transitions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292932
2019-04-26 23:59:32 -04:00
gbrodman
a0f9e3b3dd Store a reference to an allocation token in the OneTime billing event
We will need to be able to find all redemptions associated with a particular token and this will allow us to do that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292865
2019-04-26 23:57:56 -04:00
gbrodman
53c1557d2a Use the registrar client ID and abuse email address in Spec11
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244899045
2019-04-26 23:45:07 -04:00
gbrodman
0644fab76e Use stream-collect rather than transformValues in Spec11 collection
As it turns out, using Maps::transformValues does not allow us to change the
resulting map--calling Map::put throws an UnsupportedOperationException. As a
result, we have to do this roundabout stream-collect to do a group-by.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244852373
2019-04-26 23:41:55 -04:00
jianglai
143ae1f93e Update proxy release pipeline following Spinnaker 1.13 release
The image name in the manifest needs to be the same as the name that Spinnaker trigger catches. With the new release, Spinnaker now correctly recognizes gcr.io/${PROJECT_ID}/proxy as the image name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244845037
2019-04-23 10:44:44 -04:00
mcilwain
24bb78bd16 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
jakubvrana
212bd54b55 Remove references to |blessStringAsTrustedResourceUrlForLegacy in <link href>.
This directive will be deleted in the future, this change prepares for it.

More information: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243847668
2019-04-16 17:24:35 -04:00
mcilwain
3f974902df Only send registrar update notification emails to primary contacts
The test changes are perhaps a little bit more involved than expected, because I
had to add a second RegistrarContact which had knock-on effects on other tests.
This does make the other tests better though, in that we're now testing registrars
with multiple contacts (we weren't much before).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243827803
2019-04-16 14:11:06 -04:00
mcilwain
55f8e0efad Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
gbrodman
d1e3194fce Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
jianglai
416a39b003 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
63807aa9be Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
gbrodman
6b6e2ed930 Use a multimap to index the Spec11 threat matches
Collecting by key leads to exceptions if there are multiple client IDs with the
same email address (if we group by client ID in the pipeline). Using
Multimaps::index means that if we're grouping by email, all matches with the
same email get concatenated together

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242858112
2019-04-11 14:58:08 -04:00
guyben
d1f833b9bc Add abuse entity to registrar entities
From to the RDAP response profile:

2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members

Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
2019-04-11 14:54:28 -04:00
mcilwain
1c33f38555 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
shicong
1897b9d442 Fix Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242472456
2019-04-11 14:47:31 -04:00