Commit graph

2605 commits

Author SHA1 Message Date
jianglai
04f5901b2b Upload gradle binary to GCS when preparing the release
This makes it possible to pull the gradle binary from a trusted source when building the release artifacts.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245450562
2019-04-27 00:07:23 -04:00
gbrodman
29b9f04aaa Fix up a comment based on investigation into billing events
Please see the linked bug for more context--I believe this is correct based on what I've observed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245423464
2019-04-27 00:05:49 -04:00
jianglai
94aa5cd6ec Update Nomulus release pipeline
Refactor out the build and package logic to a reusable script. Also removed the gradle task flag to skip lint check, as failing lint check is no longer a fatal error.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245296563
2019-04-27 00:01:06 -04:00
gbrodman
4895f455d8 Add additional tests for the AllocationToken status transitions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292932
2019-04-26 23:59:32 -04:00
gbrodman
6ee34a79b1 Store a reference to an allocation token in the OneTime billing event
We will need to be able to find all redemptions associated with a particular token and this will allow us to do that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292865
2019-04-26 23:57:56 -04:00
jianglai
aadefd9595 Use the GCS repository when pulling dependencies in CI
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245292305
2019-04-26 23:56:20 -04:00
jianglai
52bfd74bd8 Use GCS HTTPS endpoints when pulling dependencies
This alleviates the needs to have Google credentials even when downloading from
a public bucket.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245290867
2019-04-26 23:54:43 -04:00
mmuller
a84e7c9fba Pin to a tagged selenium docker image
The latest version of the selenium/standalone-chrome docker image appears to
be out of sync with the API that we're using and there is no more recent
version of the API available in maven central.

This change pins us to the earlier version and fixes our CI build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245286532
2019-04-26 23:53:06 -04:00
jianglai
94dfa39722 LSC: Mark Python binaries with explicit python_version.
Having PY2 global version default is deprecated and change in 2019Q3.  This LSC ([] is a prerequisite to that Blaze change to prevent unintended breakages when we flip the default.

This CL is no-op to “freeze” the world. If it conflicts with your plans to move to PY3, feel free to change forward to PY3 as you see fit (eg. remove python_version when you move to py2and3_test).

BEGIN_PUBLIC
n/a
END_PUBLIC

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245028632
2019-04-26 23:51:31 -04:00
jianglai
784a7581e2 Update .gitignore file
When a release repo is first created by combining the public and the internal repos, we need to make sure to include files that are currently excluded in the public repo's .gitignore but have been tracked before the exclusion rules are added (because the rules do not apply retroactively: if a file is already tracked by git, even if it later matches a rule in .gitignore, it will not be excluded).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244916005
2019-04-26 23:48:22 -04:00
jianglai
5421609cfd Exclude files under repos/ from being uploaded to Cloud Build
When submitting a Cloud Build job from the command line, we do not want to include files under repos/, which can amount to almost a Gig.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244906528
2019-04-26 23:46:44 -04:00
gbrodman
2b12ca42f5 Use the registrar client ID and abuse email address in Spec11
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244899045
2019-04-26 23:45:07 -04:00
gbrodman
9f979790e4 Use stream-collect rather than transformValues in Spec11 collection
As it turns out, using Maps::transformValues does not allow us to change the
resulting map--calling Map::put throws an UnsupportedOperationException. As a
result, we have to do this roundabout stream-collect to do a group-by.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244852373
2019-04-26 23:41:55 -04:00
jianglai
619c8a984e Update proxy release pipeline following Spinnaker 1.13 release
The image name in the manifest needs to be the same as the name that Spinnaker trigger catches. With the new release, Spinnaker now correctly recognizes gcr.io/${PROJECT_ID}/proxy as the image name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244845037
2019-04-23 10:44:44 -04:00
jianglai
78edc62ab3 Upgrade to Gradle 5.4
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244686718
2019-04-23 10:43:09 -04:00
jianglai
926e68e806 Update proxy deployment pipeline
The pipeline is broken into two. The first one is to be triggered when the public repo is tagged. It then tags the private repo, builds and upload the builder and base images, and push a new commit to the release (merged repo). This pipeline also does text manipulation on several files in the release repo to ensure that the images uploaded in this pipeline is always used to reproducibly build the release repo at the same commit.

The second pipeline is then triggered by commit into the release repo, which builds, signs and uploads the proxy image.

Also updated the dependency lock files to use the latest plugins dependencies, which are uploaded to the GCS repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244666211
2019-04-22 13:02:39 -04:00
jianglai
7a006df6c3 Do not track the repos folder in git
It is only used when uploading dependencies to GCS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244654178
2019-04-22 12:59:27 -04:00
cpovirk
8798cfb4a9 Use new Truth failWithActual() method
Migrate Truth subjects from the old fail(String, Object) to the new failWithActual(String, Object), tweaking verbs for the new grammar.

Before:
  fail("has foo", expected);

After:
  failWithActual("expected to have foo", expected);

Open-source note: The fail*() methods used by this CL were added in Truth 0.41.

More information:
  []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244418884
2019-04-22 12:57:49 -04:00
mcilwain
b46a6b6d55 Fix some statically detected code issues
This includes: unnecessary semicolons, suppress warnings, switch statements, final/private qualifiers, Optional wrapping, conditionals, both inline and non-inline variables, ternaries, Collection putAll() calls, StringBuilders, and throws declarations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244182539
2019-04-22 12:54:34 -04:00
jianglai
2c049a65b0 Make repo override messages more consistent
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=244042884
2019-04-22 12:52:55 -04:00
mcilwain
e3bc794151 Add missing type parameters
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243877573
2019-04-16 17:30:58 -04:00
jianglai
81959e2588 Automated g4 rollback of changelist 243838016.
*** Reason for rollback ***

Looks like Travis has the same problem. We may need to look into how to get a credential on Travis, or use Cloud Build as CI for our internal build.

Rolling back this CL for now.

*** Original change description ***

Use GCS Maven and plugins repo in travis

This make it easier to detect problems earlier when we forget to update the GCS
repo.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243862581
2019-04-16 17:27:49 -04:00
jakubvrana
18d1654dbf Remove references to |blessStringAsTrustedResourceUrlForLegacy in <link href>.
This directive will be deleted in the future, this change prepares for it.

More information: []

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243847668
2019-04-16 17:24:35 -04:00
jianglai
c9271a2f5e Use GCS Maven and plugins repo in travis
This make it easier to detect problems earlier when we forget to update the GCS
repo.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243838016
2019-04-16 14:12:42 -04:00
mcilwain
68488f0833 Only send registrar update notification emails to primary contacts
The test changes are perhaps a little bit more involved than expected, because I
had to add a second RegistrarContact which had knock-on effects on other tests.
This does make the other tests better though, in that we're now testing registrars
with multiple contacts (we weren't much before).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243827803
2019-04-16 14:11:06 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
jianglai
f6dbc4156a update dependency lock file
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243320475
2019-04-15 14:58:38 -04:00
gbrodman
74971144d2 Don't delete or "redeem" unlimited use AllocationTokens
We haven't started dealing with timing or discounts yet, but unlimited use tokens should actually be unlimited use

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243318266
2019-04-15 14:57:02 -04:00
jianglai
77eb7610e8 Remove @OnLoad on setDefaultNumDnsPublishLocks
All entities are updated.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243274828
2019-04-15 14:55:23 -04:00
gbrodman
cfee7e7fd5 Add necessary fields to the AllocationToken schema
See https://docs.google.com/document/d/1SSWrILRpx0Mtr4sdvlYwz9I8wJp5Gu_o4qlml3iJDKI

This is just the base for now--we don't actually do anything with it.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243265164
2019-04-15 14:53:45 -04:00
jianglai
314daff8a1 Build the builder image in a script
This makes it so that only one extra layer is added in the builder image, improving
performance (may no longer relevant for newer versions of docker).

See: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/#minimize-the-number-of-layers

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242936360
2019-04-11 15:01:22 -04:00
mmuller
b95987230e Sign an attestation for the proxy build
This signs an attestation that the proxy image was built by cloud build,
allowing the image to be used on a bin-auth restricted kubernetes cluster.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242917524
2019-04-11 14:59:46 -04:00
gbrodman
565c4bcf50 Use a multimap to index the Spec11 threat matches
Collecting by key leads to exceptions if there are multiple client IDs with the
same email address (if we group by client ID in the pipeline). Using
Multimaps::index means that if we're grouping by email, all matches with the
same email get concatenated together

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242858112
2019-04-11 14:58:08 -04:00
jianglai
ec9e4e6475 Set encoding for java tests
The java compiler in the builder with new base image defaults to ASCII encoding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242726695
2019-04-11 14:56:13 -04:00
guyben
896cc252e7 Add abuse entity to registrar entities
From to the RDAP response profile:

2.4.5. Abuse Contact (email, phone) - an RDAP server MUST include an *entity*
with the *abuse* role within the registrar *entity* which MUST include *tel*
and *email*, and MAY include other members

Even though this is a MUST - this field will only be shown if the registrar has a *visible* abuse contact.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242684303
2019-04-11 14:54:28 -04:00
gbrodman
15faefef25 Change the email address for the second test registrar
NewRegistrar will still have the email address "new.registrar@example.com" and TheRegistrar will now have the email address "the.registrar@example.com".

I noticed this when looking at the Spec11 testing code and this will make it easier to test that code later when we retrieve email addresses from the registrar datastore objects themselves.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242676543
2019-04-11 14:52:43 -04:00
jianglai
d54e97dc18 Update builder base image
This image seems to be more publicly advertised compared to the one currently used, even though the latter is also from Google.

See: https://cloud.google.com/container-registry/docs/managed-base-images#available_images

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242666510
2019-04-11 14:50:55 -04:00
mcilwain
d5b540dfe7 Delete unused description field from premium/reserved list entities
It isn't used now, and as far as I can tell it was never used for anything, so
get rid of it.  We won't be using it in our new Cloud SQL schema anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242501877
2019-04-11 14:49:14 -04:00
shicong
5e310f8fcd Fix Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242472456
2019-04-11 14:47:31 -04:00
guyben
15e54f2803 Show only the last of each event type in RDAP domain response
We also ignore events that happened before the domain was created (for example, in a previous incarnation of the same domain name) and we set the last changed event to be the later of the last EPP change and any other event that happened before "now".

From RDAP response profile
2.3.2 The domain object in the RDAP response MAY contain the following events:
2.3.2.3 An event of *eventAction* type *transfer*, with the last date and time that the
domain was transferred. The event of *eventAction* type *transfer* MUST be omitted if
the domain name has not been transferred since it was created.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242461310
2019-04-11 14:45:46 -04:00
shicong
72b9ca6894 Add required dependency for RemoteApiInstaller
We got 500 Internal Server Error when calling GAE RemoteApi in Nomulus
tool. The root cause is we missed the dependency described in this
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242166362
2019-04-11 14:43:46 -04:00
jianglai
931388213e Upgrade to Gradle 5.3.1
Also modified the deployment pipeline to upload artifacts for crash.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242032822
2019-04-05 11:58:03 -04:00
shicong
0c57f768e0 Fix Kokoro build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242021652
2019-04-05 11:56:24 -04:00
mmuller
c8c7a043bd Change text of registrar change notification e-mail
Since we're now sending these to actual registrars, change the text to make it
a little bit clearer what's going on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241992717
2019-04-05 11:54:45 -04:00
shicong
4240583af7 Improve exception message when TMDB not available
We saw a few IOExceptions recently which indicated that TMDB may
be down at that time. However, these IOExceptions didn't have
any message to clearly describe that. This CL is to improve
the exception message so next time we don't have to look at the
source code to figure out what happened.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241984023
2019-04-05 11:53:08 -04:00
gbrodman
31c2ea1b3d Clarify that we are actually using the client ID in Spec11
The Registrar object's @ID is on its clientId field, so that's what we're actually using here, not the name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241936005
2019-04-05 11:51:28 -04:00
shicong
387042bf3a Enable flow documentation in external build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241934689
2019-04-05 11:49:49 -04:00
mcilwain
9b80b31917 Make RDE report generation correctly handle DISABLED registrars
This is a follow-up to [] We can't set registrars as DISABLED until
this is deployed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241767990
2019-04-05 11:46:29 -04:00
gbrodman
d16f3ae590 Continue Spec11 emails even if one fails
One failure should not stop the rest from sending.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241760092
2019-04-05 11:44:49 -04:00
mmuller
4eb1cd592d Make cloud build of nomulus use the GCS repo.
Also change comments to cover a few ambiguous points.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=241731091
2019-04-05 11:39:53 -04:00