Commit graph

2590 commits

Author SHA1 Message Date
jianglai
648fb0e46d Update proxy deployment pipeline
The pipeline is broken into two. The first one is to be triggered when the public repo is tagged. It then tags the private repo, builds and upload the builder and base images, and push a new commit to the release (merged repo). This pipeline also does text manipulation on several files in the release repo to ensure that the images uploaded in this pipeline is always used to reproducibly build the release repo at the same commit.

The second pipeline is then triggered by commit into the release repo, which builds, signs and uploads the proxy image.

Also updated the dependency lock files to use the latest plugins dependencies, which are uploaded to the GCS repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244666211
2019-04-22 13:02:39 -04:00
jianglai
89298ea4cb Do not track the repos folder in git
It is only used when uploading dependencies to GCS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244654178
2019-04-22 12:59:27 -04:00
cpovirk
a46227b201 Use new Truth failWithActual() method
Migrate Truth subjects from the old fail(String, Object) to the new failWithActual(String, Object), tweaking verbs for the new grammar.

Before:
  fail("has foo", expected);

After:
  failWithActual("expected to have foo", expected);

Open-source note: The fail*() methods used by this CL were added in Truth 0.41.

More information:
  []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244418884
2019-04-22 12:57:49 -04:00
mcilwain
24bb78bd16 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
jianglai
9f360587ff Make repo override messages more consistent
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244042884
2019-04-22 12:52:55 -04:00
mcilwain
8b71a3cafb Add missing type parameters
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243877573
2019-04-16 17:30:58 -04:00
jianglai
3c34d23e87 Automated g4 rollback of changelist 243838016.
*** Reason for rollback ***

Looks like Travis has the same problem. We may need to look into how to get a credential on Travis, or use Cloud Build as CI for our internal build.

Rolling back this CL for now.

*** Original change description ***

Use GCS Maven and plugins repo in travis

This make it easier to detect problems earlier when we forget to update the GCS
repo.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243862581
2019-04-16 17:27:49 -04:00
jakubvrana
212bd54b55 Remove references to |blessStringAsTrustedResourceUrlForLegacy in <link href>.
This directive will be deleted in the future, this change prepares for it.

More information: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243847668
2019-04-16 17:24:35 -04:00
jianglai
6ee462247c Use GCS Maven and plugins repo in travis
This make it easier to detect problems earlier when we forget to update the GCS
repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243838016
2019-04-16 14:12:42 -04:00
mcilwain
3f974902df Only send registrar update notification emails to primary contacts
The test changes are perhaps a little bit more involved than expected, because I
had to add a second RegistrarContact which had knock-on effects on other tests.
This does make the other tests better though, in that we're now testing registrars
with multiple contacts (we weren't much before).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243827803
2019-04-16 14:11:06 -04:00
mcilwain
55f8e0efad Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
jianglai
3b1d38e1e6 update dependency lock file
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243320475
2019-04-15 14:58:38 -04:00
gbrodman
d1e3194fce Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
jianglai
416a39b003 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
63807aa9be Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
jianglai
3b87d4de64 Build the builder image in a script
This makes it so that only one extra layer is added in the builder image, improving
performance (may no longer relevant for newer versions of docker).

See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242936360
2019-04-11 15:01:22 -04:00
mmuller
0c64d1bffc Sign an attestation for the proxy build
This signs an attestation that the proxy image was built by cloud build,
allowing the image to be used on a bin-auth restricted kubernetes cluster.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242917524
2019-04-11 14:59:46 -04:00
gbrodman
6b6e2ed930 Use a multimap to index the Spec11 threat matches
Collecting by key leads to exceptions if there are multiple client IDs with the
same email address (if we group by client ID in the pipeline). Using
Multimaps::index means that if we're grouping by email, all matches with the
same email get concatenated together

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242858112
2019-04-11 14:58:08 -04:00
jianglai
641c3f6bc3 Set encoding for java tests
The java compiler in the builder with new base image defaults to ASCII encoding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242726695
2019-04-11 14:56:13 -04:00
guyben
d1f833b9bc Add abuse entity to registrar entities
From to the RDAP response profile:

2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members

Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
2019-04-11 14:54:28 -04:00
gbrodman
bf0d83585a Change the email address for the second test registrar
NewRegistrar will still have the email address "new.registrar@example.com" and TheRegistrar will now have the email address "the.registrar@example.com".

I noticed this when looking at the Spec11 testing code and this will make it easier to test that code later when we retrieve email addresses from the registrar datastore objects themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242676543
2019-04-11 14:52:43 -04:00
jianglai
ababada79a Update builder base image
This image seems to be more publicly advertised compared to the one currently used, even though the latter is also from Google.

See: https://cloud.google.com/container-registry/docs/managed-base-images#available_images

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242666510
2019-04-11 14:50:55 -04:00
mcilwain
1c33f38555 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
shicong
1897b9d442 Fix Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242472456
2019-04-11 14:47:31 -04:00
guyben
3569f89c40 Show only the last of each event type in RDAP domain response
We also ignore events that happened before the domain was created (for example, in a previous incarnation of the same domain name) and we set the last changed event to be the later of the last EPP change and any other event that happened before "now".

From RDAP response profile
2.3.2 The domain object in the RDAP response MAY contain the following events:
2.3.2.3 An event of *eventAction* type *transfer*, with the last date and time that the
domain was transferred. The event of *eventAction* type *transfer* MUST be omitted if
the domain name has not been transferred since it was created.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242461310
2019-04-11 14:45:46 -04:00
shicong
9bcaaec7fd Add required dependency for RemoteApiInstaller
We got 500 Internal Server Error when calling GAE RemoteApi in Nomulus
tool. The root cause is we missed the dependency described in this
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242166362
2019-04-11 14:43:46 -04:00
jianglai
e51c193558 Upgrade to Gradle 5.3.1
Also modified the deployment pipeline to upload artifacts for crash.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242032822
2019-04-05 11:58:03 -04:00
shicong
d4f780607e Fix Kokoro build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242021652
2019-04-05 11:56:24 -04:00
mmuller
2808364381 Change text of registrar change notification e-mail
Since we're now sending these to actual registrars, change the text to make it
a little bit clearer what's going on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241992717
2019-04-05 11:54:45 -04:00
shicong
baed9dc6cc Improve exception message when TMDB not available
We saw a few IOExceptions recently which indicated that TMDB may
be down at that time. However, these IOExceptions didn't have
any message to clearly describe that. This CL is to improve
the exception message so next time we don't have to look at the
source code to figure out what happened.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241984023
2019-04-05 11:53:08 -04:00
gbrodman
d217939894 Clarify that we are actually using the client ID in Spec11
The Registrar object's @ID is on its clientId field, so that's what we're actually using here, not the name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241936005
2019-04-05 11:51:28 -04:00
shicong
33cdc3ddda Enable flow documentation in external build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241934689
2019-04-05 11:49:49 -04:00
mcilwain
1346f7ab70 Make RDE report generation correctly handle DISABLED registrars
This is a follow-up to [] We can't set registrars as DISABLED until
this is deployed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241767990
2019-04-05 11:46:29 -04:00
gbrodman
3ad0d091f5 Continue Spec11 emails even if one fails
One failure should not stop the rest from sending.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241760092
2019-04-05 11:44:49 -04:00
mmuller
bf70f83359 Make cloud build of nomulus use the GCS repo.
Also change comments to cover a few ambiguous points.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241731091
2019-04-05 11:39:53 -04:00
shicong
ac74ad3088 Use Docker Java API to manage container for WebDriver
Previously we had a few customized Gradle build task to manage
the Docker container for provisioning browser and ChromeDriverService
used by WebDriver tests. This CL changed to use a java library
from testcontainers.org to achieve the same purpose. The main
benefit of it is that we can expect to run the WebDriver tests
from IDE going forward.

Also, this CL refactored the structure of WebDriver related classes
to have JUnit rule to manage the lifecycle of WebDriver instance,
this is also compatible with the API from testcontainers library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241539861
2019-04-05 11:38:16 -04:00
mmuller
5470414e48 Convert update_dependency procedure to use gradle proxy
Collect the set of dependencies using the gradle proxy and push to GCS using
gcs_sync.

TESTED: Verified that the script works against both unupdated and up-to-date
dependency sets, verified that the proxy server is destroyed after completion.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241529357
2019-04-05 11:36:41 -04:00
guyben
c0f5710c31 Update the rdapConformance and notices fields
We're changing rdapConformance from rdap_level_0 to icann_rdap_response_profile_0, and changing the EPP status notices title from "EPP Status Codes" to "Status Codes" to conform with the changing spec.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241527211
2019-04-05 11:35:08 -04:00
gbrodman
9c1739d562 Include the registrar name in the Spec11 threats report
Because we make use of previous reports, it's easier to add this in and then use it later as two steps, rather than trying to add this and use it in one change. This way we don't really need to deal with backcompat issues.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241378853
2019-04-05 11:33:33 -04:00
gbrodman
25f1d58969 Automated g4 rollback of changelist 240574585.
*** Reason for rollback ***

The inconsistent class loading is breaking the tests

*** Original change description ***

Validate provided email addresses when creating a Registrar

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241014945
2019-03-29 16:23:59 -04:00
gbrodman
315be3eab0 Sort the Spec11 pipeline test output by string length
The JSON object toString isn't necessarily deterministic in its ordering of the keys, so we can only rely on length

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241003060
2019-03-29 16:22:23 -04:00
gbrodman
d5fbdea37d Skip emailing registrars with a null or empty email address
Obviously this is a bad thing and would fail if it ever happened. If this does occur, we will send a warning email.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240977242
2019-03-29 16:20:47 -04:00
shicong
8af9090fb0 Use reflection to inject the attempt number
This CL is to address the public static field in RepeatableRunner
for caller to get the current attempt number. We tried to have
a JUnit TestRule to achieve the purpose but it ended up with having
a RuleChain in each class where we already have multiple rules and
need to add the retry rule. This is because we have to make sure
the retry rule is the last one to wrap the test statement so that
the actual retry can include the actions defined in other rules.
Having a rule chain is not scalable and confuses engineer so we
gave it up.

Instead, we decided to expand the current RepeatableRunner to
use reflection to inject the attempt number to the test class.
Doing it this way can reduce the burden from the caller and it also
gets rid of the global state from the previous public static field.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240789045
2019-03-29 16:17:35 -04:00
gbrodman
174d8d69ea Validate provided email addresses when creating a Registrar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240574585
2019-03-29 16:14:22 -04:00
mcilwain
26eda15d47 Check registrar existence prior to verifying access
This way the error messages are more sensible when a registrar doesn't exist
(which realistically shouldn't happen in the typical case anyway).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240376239
2019-03-29 16:11:10 -04:00
mcilwain
e1abef7b3e Explain why permission check occurs before existence check
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240355450
2019-03-29 16:09:32 -04:00
gbrodman
648656e002 Coalesce null to the empty string in the Spec11 pipeline
We'll have a separate change to make sure we're not actually trying to email these folks, but this will make it so that the entire pipeline doesn't crash. The test makes sure that we can run the pipeline properly with these empty strings.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240346954
2019-03-29 16:07:54 -04:00
guyben
1f3c419e20 Consolidate domain_info_fee xml test files
This removes a LOT of "almost duplicate" test files, and moves some of the info into the java test itself making it clearer what's actually different between tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240230982
2019-03-29 16:06:09 -04:00
mcilwain
4fb0021b14 Display created domain name in get_allocation_token command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240212479
2019-03-29 16:02:48 -04:00
jianglai
64edf25fbe Add crash canary environment for proxy
This makes it possible to set up canary testing on crash.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240210405
2019-03-29 16:01:07 -04:00