Commit graph

3154 commits

Author SHA1 Message Date
Shicong Huang
a68b1a12fd
Implement TransactionManager for datastore (#207)
This PR created the new interface named TransactionManager which defines
methods to manage transaction. Also, the access to all transaction related
methods of Ofy.java are restricted to package private, and they will be exposed
by DatastoreTransactionManager which is the datastore implementation of
TransactionManager.
2019-08-07 12:59:08 -04:00
Weimin Yu
ce791e8b97
Generate code coverage report (#216)
* Generate code coverage report

Enable jacoco, the official Gradle code coverage plugin.

The 'build' task will write a code coverage report to
build/reports/jacoco for each subproject that has tests.
We should consider publish periodical reports to a well known
location.

This change also defines a minimum coverage verification task.
The task is for experiment only, and is not added to the build
process yet.
2019-08-07 11:21:04 -04:00
gbrodman
cf3f9603f7
Clean up RDAP action logs and request plain text (#213)
* Remove unused log argument

* Use the right accept-encoding

By default we request gzip and theoretically we'd decode it
automatically on our end but for some reason that's not working. I
tested this on Alpha and it worked
2019-08-06 14:00:09 -04:00
Lai Jiang
4fd32c8b5c
Add a Gradle task to deploy the proxy (#214)
Also refactored some common code regarding the mapping between
environments and projects.

The script to update proxy deployments and kill all pods are lifted from
https://cs.corp.google.com/piper///depot/google3/domain/registry/tools/bashrc?l=163
2019-08-06 11:45:34 -04:00
gbrodman
3488f95990
Clean up Gradle stylings and fix issues IDed in Prober (#212) 2019-08-05 15:54:20 -04:00
Shicong Huang
8e00f85f60
Use OpenJDK8 for Gradle build (#210) 2019-08-02 17:00:35 -04:00
Shicong Huang
3cc8d83396
Hard code webdriver docker version (#208) 2019-08-01 14:02:57 -04:00
gbrodman
0a779612f6
Remove old EPP processing time metrics (#206) 2019-07-31 19:18:40 -04:00
gbrodman
545a03618b
Add an underline when hovering/focusing on <a> tags (#193) 2019-07-31 17:36:53 -04:00
Ben McIlwain
bcdacc88d3
Remove vestigial path for streaming EPP metrics (#184)
* Remove vestigial path for streaming EPP metrics

The relevant action was deleted last year here: google@218c451

This removes the final hanging piece.
2019-07-29 14:37:04 -04:00
gbrodman
56b10ea136
Create a Gradle task to run the test server (#192)
* Create a Gradle task to run the test server

As an artifact of the old build system, the test server relies on having
the built registrar_(bin|dbg)*(\.css)?.js in place (see ConsoleUiAction
among others). As a result, we create a Gradle task that puts those
files into the correct, readable, location before running the test
server.

* Depend on assemble rather than build

* refactor gitignores
2019-07-29 11:03:29 -04:00
Ben McIlwain
9479e1e8b9
Rename Spec11 reply-to email and also use it as sending address (#201)
* Rename Spec11 reply-to email and also use it as sending address
2019-07-26 15:30:46 -04:00
Shicong Huang
110bd9c057
Fix deploying to alpha from workstation (#198)
appengineDeployAll requires appengine.deploy.version to be set
otherwiese the deployment would fail.
2019-07-26 14:37:49 -04:00
Lai Jiang
2ae5aba7ff
Rename dependency_license.gradle file (#196)
The file name confused the licensee app
(https://github.com/licensee/licensee) that GitHub uses to detect
license files. It thinks this file is also a license file and was not
able to determine is type.
2019-07-26 14:19:08 -04:00
Lai Jiang
b878e5acc1
Remove port to protocol map (#200)
There's no need for it. We can get the port from the protocol.
2019-07-25 22:05:12 -04:00
gbrodman
4d0409c924
Add HTML lang tags where possible (#199) 2019-07-25 21:09:32 -04:00
gbrodman
0292887cb9
Store only interesting failures, not login failures (#188)
Login failures will happen any time that we aren't coming from a
whitelisted IP for that particular TLD. Since whitelists are out of date
(and we don't whitelist IPs for every TLD anyway) those failures aren't
interesting. Store and fully-log the interesting failures if one
happened.
2019-07-24 10:55:55 -04:00
Lai Jiang
464f6ba90a
Add a missing space (#197) 2019-07-24 10:50:27 -04:00
Lai Jiang
0ab0a8c2e6
Use positive booleans to improve readability (#190)
This allows us to use lgtm.com to scan for vulnerabilities.
2019-07-23 22:30:46 -04:00
Lai Jiang
2cc4e5fa94
Add a badge for LGTM analysis results (#194) 2019-07-23 21:42:19 -04:00
gbrodman
47a890253e
Revert "Use parallel Gradle builds by default (#189)" (#195)
This reverts commit 5dfd96d26d.
2019-07-23 18:12:04 -04:00
Lai Jiang
b452b0628d
Add customized .lgtm.yml (#191)
* Add customized .lgtm.yml
2019-07-23 17:09:05 -04:00
gbrodman
5dfd96d26d
Use parallel Gradle builds by default (#189) 2019-07-23 15:27:30 -04:00
Shicong Huang
e2a673d914
Bring back the old GoogleCredential for Drive API (#187)
Using the new GoogleCredentials to access Drive API caused 403 forbidden
exception. So, this PR brought back the old GoogleCredential to
temporarily resolve the production issue while we are figuring out the
long term fix.

TESTED=Deployed to alpha and verified exportPremiumTerms succeeded, see
https://paste.googleplex.com/6153215760400384.
2019-07-23 11:31:35 -04:00
gbrodman
bf29d159f9
Fix a few deprecations (#186) 2019-07-22 14:12:55 -04:00
Lai Jiang
e17cb52bf7
Fail gracefully when copying detailed reports (#181)
* Fail gracefully when copying detailed reports

When the detailed reports are copied from GCS to registrars' Drive
folders, do not fail the entire copy operation when a single registrar
fails. Instead, send an alert email about the failure, and continue to copy the
rest of the reports.

Also, instead of creating duplicates, overwrite the existing files on
Drive.

BUG=127690361
2019-07-22 14:09:49 -04:00
Lai Jiang
7352f9b4a6
Remove unused local variable (#185) 2019-07-22 10:04:16 -04:00
Lai Jiang
5da48184f9
Merge beam and GAE configs deployment to one GCB job (#182)
* Merge beam and GAE configs deployment to one GCB job

Deployment of GAE configs requires that the credential used by gcloud to
have GAE admin role of the project to be managed. We do not want to
grant the GCB service account that role, because it would all *any* GCB
job to deploy anything to GAE. Instead we use a dedicated credential
originally created to deploy beam pipelines. This credential is
encrypted by KMS and stored on GCS. Since the beam pipeline deployment
GCB job already does the decryption, it make sense to add the config
deployment step there as well. The beam deployment steps are tweaked to
use the nomulus tool docker image instead of the jar file.

Also moved the content of deploy_configs_to_env.sh to the GCB yaml file
itself because the shell script is not uploaded to GC Bat the same time
as the yaml file when the job is triggered by Spinnaker.

Lastly, due to b/137891685, using GCB to deploy cron jobs does not work
as we cannot use service account credential to deploy to projects under
google.com.
2019-07-19 16:54:56 -04:00
gbrodman
5bd2ccd210
Add a Cloud Build task to update YAML configs (#177)
* Add a Cloud Build task to update YAML configs

* CR responses

* Move config deployment to a script

* Pin builder version

* Create different beam and deploy-config files per environment

* Update comments and make a for loop
2019-07-18 12:15:15 -04:00
Lai Jiang
8fd5ab2bec
Build proxy image in Gradle (#179) 2019-07-17 20:38:03 -04:00
Lai Jiang
30f6113b05
Upgrade to Gradle 5.5.1 (#178)
Also make the default wrapper type "all" instead of "binary". This is
helpful for IDEs to understand the gradle script.
2019-07-17 17:37:44 -04:00
Michael Muller
bd48041961 Build docker image of nomulus tool (#142)
* Build docker image of nomulus tool

In the course of "gradle build", build a docker image of nomulus tool so that
users can run this to allow us to bundle the java version with the image.
2019-07-16 20:18:44 -04:00
gbrodman
39ceda628c
Don't extend expiration times for deleted domains (#160)
* Don't extend expiration times for deleted domains

* Flip order and add a comment

* oops forgot a period

* Use END_OF_TIME

* Add tests for expiration times of domains with pending transfers

* Add test for transfer during autorenew and clean up other tests

* Clarify tests

* Add domain expiration check in EppLifecycleDomainTest

* Add a comment and format test files
2019-07-16 18:34:21 -04:00
Aman Sanger
0e9b75e5e9
Created Prober subproject and setup basic ActionHandler and its unit tests (#133)
* Initial Commit.

* Deleted unfinished features. Added ActionHandler and its Unit Tests.

* Included prober subproject in settings.gradle

* Added Protocol Class and its Basic Unit Tests

* Added Changes Suggested by jianglai

* Fixed Gitignore to take out AutoValue generated code

* Removed AutoValue java files

* Added gitignore within prober

* Removed all generated java

* Final Changes in .gitignore

* Added Ssl and WebWhois Action Handlers and their unit tests in addition to the ProbingAction class

* Fixed build.gradle changes requested

* Removed Files irrelevant to current pull request

* Minor fixes to ActionHandler, as responded in comments, removed package-info, and updated settings.gradle

* Fully Updated ActionHandler (missing updated JavaDoc)

* Added changed Protocol and both Inbound and Outbound Markers

* Removed AutoVaue ignore clause from .gitignore

* removed unneccessary dependencies in build.gradle

* Fixed Javadoc and comments for ActionHandler

* Fixed comments and JavaDoc on other files

* EOL added

* Removed Unnecessary Files

* fixed .gradle files styles

* Merge remote-tracking branch 'upstream/master'

* Removed outbound message from ActionHandler's fields and renamed Marker Interfaces

* Fixed javadoc for Marker Interfaced

* Modified Comments on ActionHandler

* Removed LocalAddress from Protocol

* Fixed Travis Build Issues
2019-07-16 10:35:14 -04:00
Lai Jiang
c1207464d8
Fix a typo (#174) 2019-07-15 17:49:22 -04:00
Weimin Yu
62eab98921
Unused deps check (#171)
* Save for later

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.

* Check for unused dependencies

Add a task to check for unused dependencies in
dependencies.gradle. This file should only contain
dependencies explicitly added to a configuration
in root project or some subprojects.
2019-07-15 16:19:34 -04:00
Shicong Huang
633dd887f4
Fix permission issue in Beam pipeline deployment (#170) 2019-07-15 16:13:42 -04:00
Lai Jiang
650f1fdd52
Bump lodash from 4.17.11 to 4.17.14 (#173) 2019-07-15 11:26:12 -04:00
gbrodman
b0f6a498fd
Bump the version of the AppEngine Gradle plugin (#169)
* Bump the version of the AppEngine Gradle plugin

* deploy should depend on deployAll
2019-07-15 11:25:23 -04:00
gbrodman
77590dcd8e
Add a metric for EPP processing time regardless of ID/TLD (#163)
* Add a metric for EPP processing time regardless of ID/TLD

* Change name to request_time

* Record EPP processing time by traffic type

* grammar

* request type

* semicolon
2019-07-11 14:28:37 -04:00
Weimin Yu
29e4d8de05
Check dependencies' open-source licenses (#165)
* Check dependencies' open-source licenses

Use jk1/Gradle-License-Report to verify that all
dependencies have open-source licenses.

Note that the following dependencies are not checked:

Dependencies of official Gradle plugins
Dependencies declared in buildscript block
Dependencies of jk1/Gradle-License-Report itself
2019-07-11 12:08:08 -04:00
Weimin Yu
242a560f20
Remove the maybeRuntime configuration (#164)
* Remove the maybeRuntime configuration

It contains dependencies present in the bazel
build but not needed for compile. We now know
they are not needed in runtime either.
2019-07-10 16:56:53 -04:00
gbrodman
3903abd9de
Move and/or delete a bunch of random resources (#152) 2019-07-09 17:38:24 -04:00
Shicong Huang
8371cb838c
Add a 30s timeout for all webdriver tests (#161)
Sometimes, the webdriver tests get stuck forever for no reason. It could
be some issue in the test container but it is hard to root cause it. So,
adding a 30s timeout can either trigger the retry earlier or let the
test just fail.
2019-07-09 14:42:32 -04:00
Shicong Huang
8dd6797614
Read golden images from src directly (#159)
This PR prevents Gradle from copying the golden images
to build/resources/test, so the screenshot test would
read golden images from src/test/resources directly and
display the path in test log if the test fails. Because
the path pointing to the actual file in src/ folder,
the engineer can easily find it.
2019-07-09 10:49:20 -04:00
Lai Jiang
730f108e13
Upgrade to Gradle 5.5 (#158) 2019-07-08 12:56:43 -04:00
gbrodman
e5bafddd2f
Move JS and CSS files to a Javascript source dir (#156) 2019-07-05 12:01:16 -04:00
Lai Jiang
82f51accbd
Set beam deployment environment in GCB trigger (#157) 2019-07-03 16:28:28 -04:00
Shicong Huang
6536631857
Remove injected credentials from invoice pipeline (#155)
We got non-serialization object error when deploying the invoicing
pipeline. It turns out that Beam requires every field in the pipeline
object is serilizable. However, it is non-trivial to make
GoogleCredentialsBundle serilizable because almost all of its
dependency are not serilizable and not contraled by us. Also,
it is non-necessary to inject the credential as the spec11
pipeline also writes output to GCS without having injected
credential. So, removing the injected variable can solve the
problem.

TESTED=First reproduced the problem locally by deploying the invoicing pipeline with the previous code; applied this change and successfully deploy the pipeline without having any issue.
2019-07-03 15:12:48 -04:00
gbrodman
1be92968bf
Attempt login to MosAPI via all available TLDs (#141)
* Attempt login to MosAPI via all available TLDs

There's no reason why we should need a TLD as input here because it
doesn't actually matter which one we use (they all have the same
password).

* Refactor the TLD loop and change cron jobs

* Re-throw the last exception if one exists

* Fix tests and exception

* Remove alpha cron job
2019-07-03 14:25:39 -04:00