Commit graph

3796 commits

Author SHA1 Message Date
jianglai
784a7581e2 Update .gitignore file
When a release repo is first created by combining the public and the internal repos, we need to make sure to include files that are currently excluded in the public repo's .gitignore but have been tracked before the exclusion rules are added (because the rules do not apply retroactively: if a file is already tracked by git, even if it later matches a rule in .gitignore, it will not be excluded).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244916005
2019-04-26 23:48:22 -04:00
jianglai
5421609cfd Exclude files under repos/ from being uploaded to Cloud Build
When submitting a Cloud Build job from the command line, we do not want to include files under repos/, which can amount to almost a Gig.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244906528
2019-04-26 23:46:44 -04:00
gbrodman
2b12ca42f5 Use the registrar client ID and abuse email address in Spec11
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244899045
2019-04-26 23:45:07 -04:00
gbrodman
9f979790e4 Use stream-collect rather than transformValues in Spec11 collection
As it turns out, using Maps::transformValues does not allow us to change the
resulting map--calling Map::put throws an UnsupportedOperationException. As a
result, we have to do this roundabout stream-collect to do a group-by.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244852373
2019-04-26 23:41:55 -04:00
jianglai
619c8a984e Update proxy release pipeline following Spinnaker 1.13 release
The image name in the manifest needs to be the same as the name that Spinnaker trigger catches. With the new release, Spinnaker now correctly recognizes gcr.io/${PROJECT_ID}/proxy as the image name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244845037
2019-04-23 10:44:44 -04:00
jianglai
78edc62ab3 Upgrade to Gradle 5.4
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244686718
2019-04-23 10:43:09 -04:00
jianglai
926e68e806 Update proxy deployment pipeline
The pipeline is broken into two. The first one is to be triggered when the public repo is tagged. It then tags the private repo, builds and upload the builder and base images, and push a new commit to the release (merged repo). This pipeline also does text manipulation on several files in the release repo to ensure that the images uploaded in this pipeline is always used to reproducibly build the release repo at the same commit.

The second pipeline is then triggered by commit into the release repo, which builds, signs and uploads the proxy image.

Also updated the dependency lock files to use the latest plugins dependencies, which are uploaded to the GCS repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244666211
2019-04-22 13:02:39 -04:00
jianglai
7a006df6c3 Do not track the repos folder in git
It is only used when uploading dependencies to GCS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244654178
2019-04-22 12:59:27 -04:00
cpovirk
8798cfb4a9 Use new Truth failWithActual() method
Migrate Truth subjects from the old fail(String, Object) to the new failWithActual(String, Object), tweaking verbs for the new grammar.

Before:
  fail("has foo", expected);

After:
  failWithActual("expected to have foo", expected);

Open-source note: The fail*() methods used by this CL were added in Truth 0.41.

More information:
  []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244418884
2019-04-22 12:57:49 -04:00
mcilwain
b46a6b6d55 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
jianglai
2c049a65b0 Make repo override messages more consistent
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244042884
2019-04-22 12:52:55 -04:00
mcilwain
e3bc794151 Add missing type parameters
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243877573
2019-04-16 17:30:58 -04:00
jianglai
81959e2588 Automated g4 rollback of changelist 243838016.
*** Reason for rollback ***

Looks like Travis has the same problem. We may need to look into how to get a credential on Travis, or use Cloud Build as CI for our internal build.

Rolling back this CL for now.

*** Original change description ***

Use GCS Maven and plugins repo in travis

This make it easier to detect problems earlier when we forget to update the GCS
repo.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243862581
2019-04-16 17:27:49 -04:00
jakubvrana
18d1654dbf Remove references to |blessStringAsTrustedResourceUrlForLegacy in <link href>.
This directive will be deleted in the future, this change prepares for it.

More information: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243847668
2019-04-16 17:24:35 -04:00
jianglai
c9271a2f5e Use GCS Maven and plugins repo in travis
This make it easier to detect problems earlier when we forget to update the GCS
repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243838016
2019-04-16 14:12:42 -04:00
mcilwain
68488f0833 Only send registrar update notification emails to primary contacts
The test changes are perhaps a little bit more involved than expected, because I
had to add a second RegistrarContact which had knock-on effects on other tests.
This does make the other tests better though, in that we're now testing registrars
with multiple contacts (we weren't much before).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243827803
2019-04-16 14:11:06 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
jianglai
f6dbc4156a update dependency lock file
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243320475
2019-04-15 14:58:38 -04:00
gbrodman
74971144d2 Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
jianglai
77eb7610e8 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
cfee7e7fd5 Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
jianglai
314daff8a1 Build the builder image in a script
This makes it so that only one extra layer is added in the builder image, improving
performance (may no longer relevant for newer versions of docker).

See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242936360
2019-04-11 15:01:22 -04:00
mmuller
b95987230e Sign an attestation for the proxy build
This signs an attestation that the proxy image was built by cloud build,
allowing the image to be used on a bin-auth restricted kubernetes cluster.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242917524
2019-04-11 14:59:46 -04:00
gbrodman
565c4bcf50 Use a multimap to index the Spec11 threat matches
Collecting by key leads to exceptions if there are multiple client IDs with the
same email address (if we group by client ID in the pipeline). Using
Multimaps::index means that if we're grouping by email, all matches with the
same email get concatenated together

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242858112
2019-04-11 14:58:08 -04:00
jianglai
ec9e4e6475 Set encoding for java tests
The java compiler in the builder with new base image defaults to ASCII encoding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242726695
2019-04-11 14:56:13 -04:00
guyben
896cc252e7 Add abuse entity to registrar entities
From to the RDAP response profile:

2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members

Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
2019-04-11 14:54:28 -04:00
gbrodman
15faefef25 Change the email address for the second test registrar
NewRegistrar will still have the email address "new.registrar@example.com" and TheRegistrar will now have the email address "the.registrar@example.com".

I noticed this when looking at the Spec11 testing code and this will make it easier to test that code later when we retrieve email addresses from the registrar datastore objects themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242676543
2019-04-11 14:52:43 -04:00
jianglai
d54e97dc18 Update builder base image
This image seems to be more publicly advertised compared to the one currently used, even though the latter is also from Google.

See: https://cloud.google.com/container-registry/docs/managed-base-images#available_images

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242666510
2019-04-11 14:50:55 -04:00
mcilwain
d5b540dfe7 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
shicong
5e310f8fcd Fix Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242472456
2019-04-11 14:47:31 -04:00
guyben
15e54f2803 Show only the last of each event type in RDAP domain response
We also ignore events that happened before the domain was created (for example, in a previous incarnation of the same domain name) and we set the last changed event to be the later of the last EPP change and any other event that happened before "now".

From RDAP response profile
2.3.2 The domain object in the RDAP response MAY contain the following events:
2.3.2.3 An event of *eventAction* type *transfer*, with the last date and time that the
domain was transferred. The event of *eventAction* type *transfer* MUST be omitted if
the domain name has not been transferred since it was created.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242461310
2019-04-11 14:45:46 -04:00
shicong
72b9ca6894 Add required dependency for RemoteApiInstaller
We got 500 Internal Server Error when calling GAE RemoteApi in Nomulus
tool. The root cause is we missed the dependency described in this
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242166362
2019-04-11 14:43:46 -04:00
jianglai
931388213e Upgrade to Gradle 5.3.1
Also modified the deployment pipeline to upload artifacts for crash.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242032822
2019-04-05 11:58:03 -04:00
shicong
0c57f768e0 Fix Kokoro build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242021652
2019-04-05 11:56:24 -04:00
mmuller
c8c7a043bd Change text of registrar change notification e-mail
Since we're now sending these to actual registrars, change the text to make it
a little bit clearer what's going on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241992717
2019-04-05 11:54:45 -04:00
shicong
4240583af7 Improve exception message when TMDB not available
We saw a few IOExceptions recently which indicated that TMDB may
be down at that time. However, these IOExceptions didn't have
any message to clearly describe that. This CL is to improve
the exception message so next time we don't have to look at the
source code to figure out what happened.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241984023
2019-04-05 11:53:08 -04:00
gbrodman
31c2ea1b3d Clarify that we are actually using the client ID in Spec11
The Registrar object's @ID is on its clientId field, so that's what we're actually using here, not the name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241936005
2019-04-05 11:51:28 -04:00
shicong
387042bf3a Enable flow documentation in external build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241934689
2019-04-05 11:49:49 -04:00
mcilwain
9b80b31917 Make RDE report generation correctly handle DISABLED registrars
This is a follow-up to [] We can't set registrars as DISABLED until
this is deployed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241767990
2019-04-05 11:46:29 -04:00
gbrodman
d16f3ae590 Continue Spec11 emails even if one fails
One failure should not stop the rest from sending.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241760092
2019-04-05 11:44:49 -04:00
mmuller
4eb1cd592d Make cloud build of nomulus use the GCS repo.
Also change comments to cover a few ambiguous points.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241731091
2019-04-05 11:39:53 -04:00
shicong
aae1e42da8 Use Docker Java API to manage container for WebDriver
Previously we had a few customized Gradle build task to manage
the Docker container for provisioning browser and ChromeDriverService
used by WebDriver tests. This CL changed to use a java library
from testcontainers.org to achieve the same purpose. The main
benefit of it is that we can expect to run the WebDriver tests
from IDE going forward.

Also, this CL refactored the structure of WebDriver related classes
to have JUnit rule to manage the lifecycle of WebDriver instance,
this is also compatible with the API from testcontainers library.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241539861
2019-04-05 11:38:16 -04:00
mmuller
f7dbaf1f81 Convert update_dependency procedure to use gradle proxy
Collect the set of dependencies using the gradle proxy and push to GCS using
gcs_sync.

TESTED: Verified that the script works against both unupdated and up-to-date
dependency sets, verified that the proxy server is destroyed after completion.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241529357
2019-04-05 11:36:41 -04:00
guyben
55673ceb2d Update the rdapConformance and notices fields
We're changing rdapConformance from rdap_level_0 to icann_rdap_response_profile_0, and changing the EPP status notices title from "EPP Status Codes" to "Status Codes" to conform with the changing spec.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241527211
2019-04-05 11:35:08 -04:00
gbrodman
822c1d129f Include the registrar name in the Spec11 threats report
Because we make use of previous reports, it's easier to add this in and then use it later as two steps, rather than trying to add this and use it in one change. This way we don't really need to deal with backcompat issues.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241378853
2019-04-05 11:33:33 -04:00
gbrodman
397f222521 Automated g4 rollback of changelist 240574585.
*** Reason for rollback ***

The inconsistent class loading is breaking the tests

*** Original change description ***

Validate provided email addresses when creating a Registrar

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241014945
2019-03-29 16:23:59 -04:00
gbrodman
78e87e4b93 Sort the Spec11 pipeline test output by string length
The JSON object toString isn't necessarily deterministic in its ordering of the keys, so we can only rely on length

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241003060
2019-03-29 16:22:23 -04:00
gbrodman
c174c86437 Skip emailing registrars with a null or empty email address
Obviously this is a bad thing and would fail if it ever happened. If this does occur, we will send a warning email.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240977242
2019-03-29 16:20:47 -04:00
shicong
c8aa6005f2 Use reflection to inject the attempt number
This CL is to address the public static field in RepeatableRunner
for caller to get the current attempt number. We tried to have
a JUnit TestRule to achieve the purpose but it ended up with having
a RuleChain in each class where we already have multiple rules and
need to add the retry rule. This is because we have to make sure
the retry rule is the last one to wrap the test statement so that
the actual retry can include the actions defined in other rules.
Having a rule chain is not scalable and confuses engineer so we
gave it up.

Instead, we decided to expand the current RepeatableRunner to
use reflection to inject the attempt number to the test class.
Doing it this way can reduce the burden from the caller and it also
gets rid of the global state from the previous public static field.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240789045
2019-03-29 16:17:35 -04:00
gbrodman
308d5eb76b Validate provided email addresses when creating a Registrar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=240574585
2019-03-29 16:14:22 -04:00