Commit graph

2830 commits

Author SHA1 Message Date
gbrodman
bf29d159f9
Fix a few deprecations (#186) 2019-07-22 14:12:55 -04:00
Lai Jiang
e17cb52bf7
Fail gracefully when copying detailed reports (#181)
* Fail gracefully when copying detailed reports

When the detailed reports are copied from GCS to registrars' Drive
folders, do not fail the entire copy operation when a single registrar
fails. Instead, send an alert email about the failure, and continue to copy the
rest of the reports.

Also, instead of creating duplicates, overwrite the existing files on
Drive.

BUG=127690361
2019-07-22 14:09:49 -04:00
Lai Jiang
7352f9b4a6
Remove unused local variable (#185) 2019-07-22 10:04:16 -04:00
Lai Jiang
5da48184f9
Merge beam and GAE configs deployment to one GCB job (#182)
* Merge beam and GAE configs deployment to one GCB job

Deployment of GAE configs requires that the credential used by gcloud to
have GAE admin role of the project to be managed. We do not want to
grant the GCB service account that role, because it would all *any* GCB
job to deploy anything to GAE. Instead we use a dedicated credential
originally created to deploy beam pipelines. This credential is
encrypted by KMS and stored on GCS. Since the beam pipeline deployment
GCB job already does the decryption, it make sense to add the config
deployment step there as well. The beam deployment steps are tweaked to
use the nomulus tool docker image instead of the jar file.

Also moved the content of deploy_configs_to_env.sh to the GCB yaml file
itself because the shell script is not uploaded to GC Bat the same time
as the yaml file when the job is triggered by Spinnaker.

Lastly, due to b/137891685, using GCB to deploy cron jobs does not work
as we cannot use service account credential to deploy to projects under
google.com.
2019-07-19 16:54:56 -04:00
gbrodman
5bd2ccd210
Add a Cloud Build task to update YAML configs (#177)
* Add a Cloud Build task to update YAML configs

* CR responses

* Move config deployment to a script

* Pin builder version

* Create different beam and deploy-config files per environment

* Update comments and make a for loop
2019-07-18 12:15:15 -04:00
Lai Jiang
8fd5ab2bec
Build proxy image in Gradle (#179) 2019-07-17 20:38:03 -04:00
Lai Jiang
30f6113b05
Upgrade to Gradle 5.5.1 (#178)
Also make the default wrapper type "all" instead of "binary". This is
helpful for IDEs to understand the gradle script.
2019-07-17 17:37:44 -04:00
Michael Muller
bd48041961 Build docker image of nomulus tool (#142)
* Build docker image of nomulus tool

In the course of "gradle build", build a docker image of nomulus tool so that
users can run this to allow us to bundle the java version with the image.
2019-07-16 20:18:44 -04:00
gbrodman
39ceda628c
Don't extend expiration times for deleted domains (#160)
* Don't extend expiration times for deleted domains

* Flip order and add a comment

* oops forgot a period

* Use END_OF_TIME

* Add tests for expiration times of domains with pending transfers

* Add test for transfer during autorenew and clean up other tests

* Clarify tests

* Add domain expiration check in EppLifecycleDomainTest

* Add a comment and format test files
2019-07-16 18:34:21 -04:00
Aman Sanger
0e9b75e5e9
Created Prober subproject and setup basic ActionHandler and its unit tests (#133)
* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Merge remote-tracking branch 'upstream/master'

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues
2019-07-16 10:35:14 -04:00
Lai Jiang
c1207464d8
Fix a typo (#174) 2019-07-15 17:49:22 -04:00
Weimin Yu
62eab98921
Unused deps check (#171)
* Save for later

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.
2019-07-15 16:19:34 -04:00
Shicong Huang
633dd887f4
Fix permission issue in Beam pipeline deployment (#170) 2019-07-15 16:13:42 -04:00
Lai Jiang
650f1fdd52
Bump lodash from 4.17.11 to 4.17.14 (#173) 2019-07-15 11:26:12 -04:00
gbrodman
b0f6a498fd
Bump the version of the AppEngine Gradle plugin (#169)
* Bump the version of the AppEngine Gradle plugin

* deploy should depend on deployAll
2019-07-15 11:25:23 -04:00
gbrodman
77590dcd8e
Add a metric for EPP processing time regardless of ID/TLD (#163)
* Add a metric for EPP processing time regardless of ID/TLD

* Change name to request_time

* Record EPP processing time by traffic type

* grammar

* request type

* semicolon
2019-07-11 14:28:37 -04:00
Weimin Yu
29e4d8de05
Check dependencies' open-source licenses (#165)
* Check dependencies' open-source licenses

Use jk1/Gradle-License-Report to verify that all
dependencies have open-source licenses.

Note that the following dependencies are not checked:

Dependencies of official Gradle plugins
Dependencies declared in buildscript block
Dependencies of jk1/Gradle-License-Report itself
2019-07-11 12:08:08 -04:00
Weimin Yu
242a560f20
Remove the maybeRuntime configuration (#164)
* Remove the maybeRuntime configuration

It contains dependencies present in the bazel
build but not needed for compile. We now know
they are not needed in runtime either.
2019-07-10 16:56:53 -04:00
gbrodman
3903abd9de
Move and/or delete a bunch of random resources (#152) 2019-07-09 17:38:24 -04:00
Shicong Huang
8371cb838c
Add a 30s timeout for all webdriver tests (#161)
Sometimes, the webdriver tests get stuck forever for no reason. It could
be some issue in the test container but it is hard to root cause it. So,
adding a 30s timeout can either trigger the retry earlier or let the
test just fail.
2019-07-09 14:42:32 -04:00
Shicong Huang
8dd6797614
Read golden images from src directly (#159)
This PR prevents Gradle from copying the golden images
to build/resources/test, so the screenshot test would
read golden images from src/test/resources directly and
display the path in test log if the test fails. Because
the path pointing to the actual file in src/ folder,
the engineer can easily find it.
2019-07-09 10:49:20 -04:00
Lai Jiang
730f108e13
Upgrade to Gradle 5.5 (#158) 2019-07-08 12:56:43 -04:00
gbrodman
e5bafddd2f
Move JS and CSS files to a Javascript source dir (#156) 2019-07-05 12:01:16 -04:00
Lai Jiang
82f51accbd
Set beam deployment environment in GCB trigger (#157) 2019-07-03 16:28:28 -04:00
Shicong Huang
6536631857
Remove injected credentials from invoice pipeline (#155)
We got non-serialization object error when deploying the invoicing
pipeline. It turns out that Beam requires every field in the pipeline
object is serilizable. However, it is non-trivial to make
GoogleCredentialsBundle serilizable because almost all of its
dependency are not serilizable and not contraled by us. Also,
it is non-necessary to inject the credential as the spec11
pipeline also writes output to GCS without having injected
credential. So, removing the injected variable can solve the
problem.

TESTED=First reproduced the problem locally by deploying the invoicing pipeline with the previous code; applied this change and successfully deploy the pipeline without having any issue.
2019-07-03 15:12:48 -04:00
gbrodman
1be92968bf
Attempt login to MosAPI via all available TLDs (#141)
* Attempt login to MosAPI via all available TLDs

There's no reason why we should need a TLD as input here because it
doesn't actually matter which one we use (they all have the same
password).

* Refactor the TLD loop and change cron jobs

* Re-throw the last exception if one exists

* Fix tests and exception

* Remove alpha cron job
2019-07-03 14:25:39 -04:00
Ben McIlwain
0564b207f2
Prevent accidentally using full Drive URL as folder ID (#144)
* Prevent accidentally using full Drive URL as folder ID
2019-07-03 14:22:15 -04:00
gbrodman
8afc4f4d3d
Enforce that source files end in a newline (#153) 2019-07-03 12:16:11 -04:00
guyben13
888bc158fe Add explanation on how to use the MoSAPI endpoint (#137)
* Add explanation on how to use the MoSAPI endpoint

* Add a $ before each command so that it's clearer
2019-07-02 18:20:53 -04:00
Lai Jiang
25ee92b5d8
Re-add hello.xml file (#147)
* Re-add hello.xml file

Apparently it was lost during the great refactoring.

* Remove hello.xml from test resources
2019-07-02 17:21:40 -04:00
gbrodman
cf507dad6d
Move test resource files into src/test/resources (#143)
* Move test resource files into src/test/resources

* fix a test

* Remove references to javatests/ in Java files

* fix import order

* fix semantic merge conflict
2019-07-02 16:54:49 -04:00
gbrodman
32d5940be3
Remove old Bazel files (#151) 2019-07-02 16:24:01 -04:00
Shicong Huang
82fa3d7349
Change button color to blue style (#150) 2019-07-02 15:29:17 -04:00
Ben McIlwain
07239710ef
Throw a more useful error message on attempted domain restore reports (#145)
* Throw a more useful error message on attempted domain restore reports

Per DomainRestoreRequestFlow's Javadoc, we automatically approve and instantly
enact all domain restore requests, thus we don't use or support restore
reports. This improves the registrar-visible error message to help make this
more clear.
2019-07-02 14:11:37 -04:00
gbrodman
0e8724a48f
Require the license in Gradle files (#149) 2019-07-02 11:47:35 -04:00
gbrodman
8332664af9
Move golden test files to src/test/resources (#146) 2019-07-02 11:34:41 -04:00
gbrodman
39abee6279
Update checkstyle.xml with many presubmit checks (#130)
* Update checkstyle.xml with many presubmit checks

* Remove a comment
2019-07-02 10:35:32 -04:00
Shicong Huang
6daf72a54e
Replace deprecated GoogleCredential with new auth lib (#129)
Replace deprecated GoogleCredential with new lib

This PR also introduced a CredentialsBundle class to carry
HttpTransport and JsonFactory object which are needed by
most of the GCP library to instantiate client.
2019-07-02 10:29:51 -04:00
gbrodman
7ce37147c0
Refactor the license check to be a generic presubmit check (#131)
* Refactor the license check to be a generic presubmit check

This includes all of the old presubmits that were in our METADATA file
that are still valid.

* Responses to CR

Added docstrings, refactored the file finder, changed variable names,
and formatted the file

* More respones to CR
2019-06-27 10:08:27 -04:00
Shicong Huang
661e348fcb
Stop provisioning RegistryEnvironment from Dagger (#140)
We found that some webdriver tests failed because RegistryEnvironment
was set to 'production' by other test and was carried over to the
webdriver test, and it was nontrivial to fix them because the instance
of RegistryEnvironment was injected into the testing web server.

Also, to eliminate this problem from potential victims, we stopped
provisioning RegistryEnvironment from Dagger. Instead, we just use
RegistryEnvironment.get() function to get the instance, which indeed
retrives the value from system property every time. If any test case
needs to run the test with other environment, it can just simply use
the existing SystemPropertyRule and RegistryEnvironment.${ENV}.setup
to change it.
2019-06-26 16:57:19 -04:00
Lai Jiang
33a7808e9e
Disable Javadoc and tests in Java 12 (#135)
The API is deprecated and removed in Java 12. Do not build these
classes when using Java 12, till we migrate to the new API.
2019-06-25 22:19:36 -04:00
gbrodman
16b0c420a2
Use strict versions in NPM versioning (#132)
It'll reduce surprise version changes
2019-06-25 16:10:04 -04:00
Lai Jiang
b7f3154f4e
Update GCB beam deployment pipeline (#134)
* Update GCB beam deployment pipeline

Some of the texts are not really secerts because they are per-project.
Also changed the location of the credential file to `secerts` so that in
the future we may add more secerts in that folder.

The encrypted file is base64 encoded, consistent with how the proxy
certificates are encoded. Also made some changes to the other pipelines
to facilitate automation with Spinnaker
2019-06-24 14:36:56 -04:00
Lai Jiang
56105105b5
Exclude Visual Studio Code project files (#136)
* Exclude Visual Studio Code project files
2019-06-24 11:39:48 -04:00
Lai Jiang
368c264e3c
Update IAM binding to restrict proxy service account's access to GCS (#125)
Per
https://cloud.google.com/container-registry/docs/access-control#granting_users_and_other_projects_access_to_a_registry,
for a service account to access GCR, it does not need reader access to *all*
buckets in a project, but just the specific one.

This is duped from cl/254092941.
2019-06-21 15:59:01 -04:00
Lai Jiang
21be4b8227
Update README.md (#128) 2019-06-21 15:24:17 -04:00
gbrodman
3a485a03a6
Add a license-checking Python script (#123)
* Add a license-checking Python script

* Responses to CR
2019-06-21 12:25:35 -04:00
gbrodman
00147c1bc4
Install Chrome in the Cloud Build process (#124)
* Install Chrome in the Cloud Build process

It's not in the built-in Linux apt repos so we need to manually wget it

* Update package lock file
2019-06-20 18:19:23 -04:00
Lai Jiang
520145f641
Move terraform and kubernetes folder to be under proxy (#127)
* Move terraform and kubernetes folder to be under proxy

There is no reason for them to be under proxy/src/... any more now that
we have a Gradle-idiomatic folder structure.
2019-06-20 14:28:32 -04:00
Lai Jiang
9cd7c9402f
Always clone the internal repo to nomulus-internal
Also updated .gcloudignore to not pull in unnecessary files when running
`gcloud builds submit`.
2019-06-20 14:26:36 -04:00