Commit graph

105 commits

Author SHA1 Message Date
Ben McIlwain
23310bd688 Rename whitelist -> allow list (#635)
* Rename whitelist -> allow list

* Merge branch 'master' into allowlist-denylist
2020-06-18 18:36:05 -04:00
Lai Jiang
e76da2c66d Update Netty to the latest version (#620)
* Upgrade to the latest version of Netty

* Update lock files
2020-06-10 16:08:11 -04:00
Lai Jiang
02e43ab134 Add a task to compile javadoc across all packages (#597)
Also fixes various issues that prevent javadoc compliation.
2020-05-27 10:33:46 -04:00
Weimin Yu
5face3b2fe Add Test suite support for JUnit 5 classes (#549)
* Add Test suite support for JUnit 5 classes

Added Gradle dependencies and updated lockfiles.

Updated SqlInegrationTestSuite to use new annotations.

Migrated one member class in SqlIntegrationTestSuite (CursorDaoTest)
to JUnit 5, and verified that the new Suite runner can handle a
mixture of JUnit 4 and 5 tests in one suite.

Note that Gradle tests that run TestSuites must choose JUnit 4.
Updated core/build.gradle and integration/build.gradle.
2020-04-07 21:06:49 -04:00
Shicong Huang
79a3605e9e Supress exccesive logging message from Cloud SQL (#540)
* Supress exccesive logging message from Cloud SQL

* Upgrade package versions that were downgraded before
2020-03-31 17:57:18 -04:00
Weimin Yu
dda9b90587 Fix broken builds when Maven Central is used (#509)
* Fix broken builds when Maven Central is used

Gradle 6.2.1 apparently introduces a behavior change wrt boolean
expression: empty string used to eval to false, but now evals to
true.

Pre Gradle 6.2.1, root project's Gradle properties apparently were
not set to buildSrc. Now they are passed on to buildSrc -- mavenUrl
in buildSrc changes from null to "".

Both changes break the project when mavenUrl and/or pluginsUrl are
not set on command line.

Also added junit.jupiter-api as testCompile dependencies to projects.
This is a directly used dependency, whose absence causes a Lint
warning.
2020-03-10 11:21:03 -04:00
Lai Jiang
456eefd95f Upgrade to Gradle 6.2.1 (#501) 2020-03-05 18:47:25 -05:00
Ben McIlwain
3139a2ffee Start using JUnit 5 (#488)
* Start using JUnit 5

This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration

In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.

* Delete no longer needed lockfiles

* Merge branch 'master' into first-junit5
2020-02-19 18:29:59 -05:00
Weimin Yu
b9c63da753 Use CA cert in tests (#475)
* Use CA cert in tests

Our tests were using a self-signed cert created by a netty utility.
That cert is not CA, and cannot be used with newer JDK versions
(starting from u242 with Ubuntu openjdk).

Now we generate a unsafe cert for tests. The new cert,
SelfSignedCaCertificate, needs to be made test-only.
To do so we need to refactor both prober and proxy.
2020-02-07 10:23:24 -05:00
Lai Jiang
cfd5307cef Use monitoring client v1.0.7 (#445)
This version of the monitoring client no longer depends on backported
JUnit classes.
2020-01-14 23:11:54 -05:00
Weimin Yu
74c614d002 Upgradle JUnit to 4.13 (#442)
* Upgradle JUnit to 4.13

Removed third_party/junit folder and all usage of the
JunitBackPort class. As a result, third_party is no
longer a Gradle subproject.

Minor code changes were needed to work around an
error-prone pattern: multiple statement in assertThrows'
runnable lambda.

Also third_party/activation and third_party/jsch. These
dependencies are loaded from remote maven repo. The local
copies are not in use.
2020-01-14 10:54:09 -05:00
Lai Jiang
9df3792fa4 Use the correct certificate provider type (#427)
TESTED: tested in alpha. Previous I only  tested locally and missed this
typo.
2020-01-06 16:19:50 -05:00
Lai Jiang
69ec237f19 Consolidate certificate supplier module (#410)
* Consolidate certificate supplier module

Both the proxy and the proxy needs certificate suppliers. The PR
consolidates the module that providings those bindings to a shared
module and switched the proxy to use that module. The prober currently
uses P12 file to store its certificates. I am debating keeping that
supplier ro converting them to PEM files for simplicity.

* Rename mode enum values to be more descriptive

* Update annotation names to be more descriptive
2019-12-23 13:09:47 -05:00
Weimin Yu
1a1ad54a17 Allow project dependency to use runtimeClasspath (#395)
* Allow project dependency to use runtimeClasspath

Project dependency should use runtimeClasspath. However, if
left unspecified, it uses 'default', which is the same as
the legacy 'runtime' configuration. (runtimeOnly dependencies
are left out).

Since runtimeClasspath cannot be referenced directly, we use
a custom config (deploy_jar) as a proxy.

By excluding testjars (leaked into 'compile' by third-party
dependencies) from runtimeClasspath, we prevent them from
getting into release artifacts.

Two meaningful changes in appengine_war.gradle and java_common.gradle

TESTED=Diffed contents of services/{module}/build/exploded-*
       Only three jars are removed: hamcrest-core, junit, and
       mockito-core.
2019-12-02 16:10:13 -05:00
Lai Jiang
1c1ccee75e Respect certificate validity period (#391)
Client SSL handler already performs the necessary validation. Only tests are
added.

Server SSL handler does not currently check for the validity period of
the client certificate as the insecure trust manager is used. This PR
added the check but does not actually terminate the connection yet. It
will log the expired certificates so that we can contact the registrars
to update them.

Once we are certain that all certificates are updated, we can turn off
dryrun mode.

We should also consider checking if the certificate has too long a
validity period as it defeats the purpose of using regularly updated
certificates to deprecate insecure cipher suites.
2019-11-27 16:08:38 -05:00
Lai Jiang
bb88ef621f Remove SSL initializer from the prober (#378)
The prober now uses the common SSL initializer in the networking
subproject.

Also changed both initializers to take an ImmutableList of certificates
other than an array of those, for better immutability.

I have no idea where these lockfile changes are coming from. They seem
to be pure noise as far as code review is concerned.
2019-11-22 17:46:06 -05:00
Lai Jiang
de02996f00 No-op: Use nicer HCL2 syntax. (#384)
Generated with perl -pi -e 's/\"\$\{([a-zA-Z0-9._-]*)\}\"/$1/g' $(find ./ -name '*.tf')

Copied from cl/282012376.
2019-11-22 16:08:56 -05:00
Weimin Yu
9359f40665 Break circular dependency between core and util (#379)
* Break circular dependency between core and util

Created a new :common project and moved a minimum
number of classes to break the circular dependency
between the two projects. This gets rid of the
gradle lint dependency warnings.

Also separated api classes and testing helpers into
separate source sets in :common so that testing
classes may be restricted to test configurations.
2019-11-21 15:36:55 -05:00
Lai Jiang
4c11e92ca8 Refactor common code used by the proxy and the prober (#375) 2019-11-20 12:42:44 -05:00
Lai Jiang
d2ebb591a2 Upgrade to Gradle 6.0 (#364) 2019-11-13 13:17:29 -05:00
Lai Jiang
c3d93ab173 Make Joda Money embeddable in entities (#340) 2019-11-07 17:03:00 -05:00
Ben McIlwain
6a2a5b4dbd Add a converter for CurrencyUnits stored in the database (#334)
* Add a converter for CurrencyUnits stored in the database

This uses the well-known String representation for currency units. It also
provides a base class for other converters that will be persisting the
toString() representation.

* Add DB and formatting changes

* Add tests, make minor fixes
2019-10-31 15:26:40 -04:00
Shicong Huang
7099652159 Upgrade org.apache.beam related packages to latest version (#333) 2019-10-30 11:44:26 -04:00
Lai Jiang
0667994b77 Instrument proxy frontend latency metric (#320)
* Instrument proxy frontend latency metric

Backend latency only captures the time spent waiting for the GAE backend
to respond to a request, which is not representitive of what clients
experience. For instance it does not take into account the time the
proxy spends on processing the requests and the time it takes to send a
response to the client.

This PR adds a metric for frontend latency, which is the time spent on
answering a client request. This should serve as a better proxy for the
latency clients observe.

* Rename AbstractMetrics to BaseMetrics
2019-10-29 16:02:47 -04:00
Weimin Yu
3763cc285d Add a test task for all schema-dependent tests (#325)
Such tests are used in server-schema compatibility tests.

Also added a test that verifies that all eligible tests
are included.
2019-10-25 13:25:01 -04:00
Lai Jiang
a6aa1ca9fe Protect KMS-secured data against destruction in upcoming google provider update. (#284)
Export of cl/270900150.

To refer to a KMS key or key ring, we should use the stable `.self_link`. Using `.id` instead provides an unstable identifier which may change (and it will change in the upcoming update of the google provider to 2.9.1). A change in the identifier will cause Terraform to destroy and recreate the key. Destroying the key means all data associated with it is lost; the key cannot be recreated.

This CL replaces `.id` with `.self_link`, so all of those problems will not happen. In addition, `prevent_destroy` protects the key against delete-and-recreate in general.
2019-09-27 12:12:58 -04:00
Weimin Yu
7db99e3308 Upgrade to Truth 1.0 (#281)
* Upgrade to Truth 1.0

Refactored fail(...) to assertWithMessage().fail().

Upgraded com.google.monitoring-client family of dependencies to 1.0.6

Also fixed bad use of io.StringIO (on binary buffer) recently introduced to
google-java-format-diff.py.
2019-09-24 10:23:58 -04:00
Shicong Huang
a85fcb6237 Make project core depend on db at test runtime (#278)
This PR also upgraded the version of Guava to 28.1-jre to
resolve missing stream package issue.
2019-09-23 14:12:24 -04:00
Weimin Yu
89404dc43d Fix dependency-locking config (#276)
* Fix dependency-locking config

Reenable dependency locking after a bug errorneouly turned it off.

Removed the guava-related workaround that forcefully resolve to
the -jre distribution.

Enabled locking for buildSrc by updating its property file.

Updated all lock files.
2019-09-19 11:37:20 -04:00
Lai Jiang
1779c63882 Update proxy deployment script description (#245) 2019-08-28 17:35:43 -04:00
Lai Jiang
43dc3d7c47 Fix expired access token bug in the proxy (#217)
https://github.com/google/nomulus/pull/129 migrated `GoogleCredential`
to `GoogleCredentialsBundle` and introduced a subtle bug. I don't fully
understand why but there are times when the access token is null but
`credentials.refresh()` is not called, resulting in NullPointerException
when `credentials.getAccessToken().getTokenValue()` is called.

Since the new GoogleCredentials class supports `shouldRefresh()`, we now
just rely on it to make sure that we always get a value access token.
2019-08-08 14:44:33 -04:00
Lai Jiang
8240dee691 Add a Gradle task to deploy the proxy (#214)
Also refactored some common code regarding the mapping between
environments and projects.

The script to update proxy deployments and kill all pods are lifted from
https://cs.corp.google.com/piper///depot/google3/domain/registry/tools/bashrc?l=163
2019-08-06 11:45:34 -04:00
gbrodman
2a381b7071 Clean up Gradle stylings and fix issues IDed in Prober (#212) 2019-08-05 15:54:20 -04:00
Lai Jiang
5fad1a6408 Remove port to protocol map (#200)
There's no need for it. We can get the port from the protocol.
2019-07-25 22:05:12 -04:00
Lai Jiang
91ceae6aeb Build proxy image in Gradle (#179) 2019-07-17 20:38:03 -04:00
Lai Jiang
3181ff0d1a Re-add hello.xml file (#147)
* Re-add hello.xml file

Apparently it was lost during the great refactoring.

* Remove hello.xml from test resources
2019-07-02 17:21:40 -04:00
gbrodman
b2199387f6 Move test resource files into src/test/resources (#143)
* Move test resource files into src/test/resources

* fix a test

* Remove references to javatests/ in Java files

* fix import order

* fix semantic merge conflict
2019-07-02 16:54:49 -04:00
gbrodman
3a81ec5480 Require the license in Gradle files (#149) 2019-07-02 11:47:35 -04:00
Shicong Huang
34a28e871e Replace deprecated GoogleCredential with new auth lib (#129)
Replace deprecated GoogleCredential with new lib

This PR also introduced a CredentialsBundle class to carry
HttpTransport and JsonFactory object which are needed by
most of the GCP library to instantiate client.
2019-07-02 10:29:51 -04:00
Lai Jiang
3202665660 Update IAM binding to restrict proxy service account's access to GCS (#125)
Per
https://cloud.google.com/container-registry/docs/access-control#granting_users_and_other_projects_access_to_a_registry,
for a service account to access GCR, it does not need reader access to *all*
buckets in a project, but just the specific one.

This is duped from cl/254092941.
2019-06-21 15:59:01 -04:00
Lai Jiang
684bb119db Move terraform and kubernetes folder to be under proxy (#127)
* Move terraform and kubernetes folder to be under proxy

There is no reason for them to be under proxy/src/... any more now that
we have a Gradle-idiomatic folder structure.
2019-06-20 14:28:32 -04:00
Lai Jiang
a1d3f7be8b Add tasks used in release as dependencies of the build task
Our CI (Travis & Kokoro) runs "gradle build", so we need to make
sure that all tasks used in the release process are called during
the build so that breakage can be caught earlier.

In order to stage the GAE folder we need gcloud to be present.
Therefore the Travis config is changed to install gcloud.

See: https://gist.github.com/mjackson/5887963e7d8b8fb0615416c510ae8857
2019-06-19 18:46:46 -04:00
gbrodman
29d7cf7b9a Add Spotless to the Gradle build (#88)
* Add Spotless to the Gradle build

* spotless apply

* one dot instead of two
2019-06-19 16:24:46 -04:00
Weimin Yu
f3ceeb33af Merge pull request #121 from weiminyu/deprecate
Fix or suppress some deprecation warnings
2019-06-19 11:30:02 -04:00
Shicong Huang
0a419f1a30 Upgrade google-auth-library-java to latest version
This is a preparation for replacing the deprecated GoogleCredential
library.
2019-06-18 17:27:21 -04:00
Weimin Yu
cdd92a0a75 Fix or suppress some deprecation warnings
Fix or suppress deprecation warnings except those about GoogleCredential,
which is being handled separately.

The @SuppressWarnings("deprecation") annotation does not cover imports
even when it is at the class level. We removed imports of deprecated
classes and use their fully qualified names in class body.
2019-06-18 17:23:45 -04:00
gbrodman
2aab7ce519 Merge branch 'master' into gb/upgradeJCommander 2019-06-13 18:18:55 -04:00
Lai Jiang
e17986dfa4 Remove Bazel build (#94)
* Remove bazel build

Also updated the build badge to reflect internal build status.
2019-06-13 18:15:33 -04:00
Gus Brodman
65dfebf99d Upgrade jcommander dep 2019-06-13 13:04:48 -04:00
Gus Brodman
9a59cd587e More moving of hello.xml files -- they're all the same 2019-06-13 11:52:04 -04:00