Commit graph

72 commits

Author SHA1 Message Date
Weimin Yu
31c16e8369 Add a Secret Manager client for Nomulus (#872)
* Add a Secret Manager client for Nomulus
2020-11-12 17:12:52 -05:00
Weimin Yu
9ddde4799c Upgrade error-prone to 3.3.4 (#848)
* Upgrade error-prone to 3.3.4

This would fix the failure with openjdk 11.0.9 in
3.3.3.

Fixed new antipatterns raised by the new version:
- Replaced unnecessary lambdas with methods.
- Switched wait/sleep calls to equivalent methods using java.time types
- Types inheriting Object.toString() should not be assigned to string
parameter in logging statements.
2020-10-23 11:17:57 -04:00
sarahcaseybot
35ebe371ba Add a CertificateChecker class (#793)
* CertificateChecker with checks for expiration and key length

* Add validity length check

* Get rid of hard-coded constants and DSA checks

* add files that for some reason weren't included in last commit

* Rename violations and other fixes

* Add displayMessage to CertificateViolation enum

* Switch violations from an enum to a class

* small changes

* Get rid of ECDSA checks

* add checks for old validity length

* Change error message for validity length
2020-10-06 15:47:42 -04:00
Lai Jiang
6a4ebaed62 Revert "Request 101m CPU in sandbox proxy (#813)" (#818)
This reverts commit e30c0f9a11.

The proposed solution didn't work.
2020-09-25 11:55:46 -04:00
Lai Jiang
4f4d5330a5 Request 101m CPU in sandbox proxy (#813)
This is suggested as a mitigation to allow us to deploy to sandbox. The default
value is 100m.

See: https://b.corp.google.com/issues/167295064#comment36.
2020-09-21 21:22:41 -04:00
Lai Jiang
b6ed1982c3 Use the save API version for the HPA controller and the deployment (#807)
This supposedly can fix our deployment problem. Tested on alpha.

Also updated the deployment script to replace the service object as
well.
2020-09-16 09:15:35 -04:00
Lai Jiang
6a5dbf8817 Update dependencies to get rid of some illegal reflection warnings (#768)
For some inexplicable reasons I have to move the javax.mail package one
spot up to avoid its classes being shadowed by those provided in the
appengine package...
2020-08-14 16:53:22 -04:00
Lai Jiang
f6749ad663 Enable Java 11 features (#737)
* Enable Java 11 features

As of this commit Java 11 must be used to build. The generated bytecode
is still at Java 8 due to App Engine task queue limit.

Also fixed a bug where the included google-java-format jar file is not
used, requiring the user to install it separately.

See: https://cloud.google.com/appengine/docs/standard/java/taskqueue/push
2020-08-03 16:08:29 -04:00
Ben McIlwain
d290b82566 Upgrade prober & proxy packages from JUnit 4 to JUnit 5 (#734)
* Upgrade prober & proxy packages from JUnit 4 to JUnit 5
2020-08-01 11:34:58 -04:00
Ben McIlwain
0d857f8291 Get rid of all remaining JUnit 4 usages except in prober & proxy (#731)
* Get rid of all remaining JUnit 4 usages except in prober & proxy subprojects

Caveat: Test suites aren't yet implemented in JUnit 5 so we still use the ones
from JUnit 5 in the core subproject.

* Fix some build errors
2020-07-30 20:29:00 -04:00
Lai Jiang
043c0aaeb8 Clean up generated sources configuration (#724)
1. The Gradle apt plugin is no longer needed to process annotations.
2. Without the apt plugin, Gralde puts the source files generated by
   annotation processors in build/generated/sources/annotationProcessor.
3. Change the location of custom generated files to be consistent.
4. Fix a javadoc formatting error.
2020-07-30 15:43:42 -04:00
Ben McIlwain
56be3f37ef Upgrade command test classes from JUnit 4 to JUnit 5 (#700)
* Convert first batch of command tests to JUnit 5

* Upgrade rest of command tests to JUnit 5

* Migrate the last few test classes
2020-07-20 20:45:52 -04:00
Lai Jiang
b4a4e23d5e Increase the maximum number of nodes in a nood pool to 15 (#672) 2020-07-10 21:54:18 -04:00
Ben McIlwain
10b23e590e Convert RDE tests (and some test rules) from JUnit 4 to JUnit 5 (#677)
* Add JUnit Params and start using it

* Convert rest of RDE tests

* Don't check headers for generated tests

* Expand visibility to fix build breakage

* Bump JUnit versions to 5.6.2
2020-07-10 21:32:36 -04:00
Lai Jiang
b7b4658d0c Patch terraform changes made internally (#651)
There were several LSC that made some formatting changes to our .tf
files. Export these changes externally for consistency.
2020-06-25 13:59:37 -04:00
Weimin Yu
b295287fc9 Fix flaky tests that depends on order of rules
SystemPropertyRule in some cases should be applied last:
when multiple rules exist and and modified property is checked
in cleanups.

ConsoleOteSetupActionTest and ConsoleRegistrarCreatorActionTest
are two such classes, and can be flaky in JUnit 4. This PR
migrates them to JUnit5 and applies ordering to extensions in
them.

Added a mockito dependency, and upgraded mockito-core to 3.3.3.

Meaningful changes: SystemPropertyRule.java and
ConsoleOteSetupActionTest.java, and
ConsoleRegistrarCreatorActionTest.java
2020-06-24 23:14:47 -04:00
Weimin Yu
3ac5f06991 Add testcontainers' Junit5 support dependency (#640)
* Add testcontainers' Junit5 support dependency

Also updated guava, dagger, hibernate, postgresql, and cloud socket factory
to latest version.

Migrated PersistenceModuleTest as an example.

Real changes:
- dependencies.gradle
- core/build.gradle
- PersistenceModuleTest.java
2020-06-22 13:12:58 -04:00
Ben McIlwain
23310bd688 Rename whitelist -> allow list (#635)
* Rename whitelist -> allow list

* Merge branch 'master' into allowlist-denylist
2020-06-18 18:36:05 -04:00
Lai Jiang
e76da2c66d Update Netty to the latest version (#620)
* Upgrade to the latest version of Netty

* Update lock files
2020-06-10 16:08:11 -04:00
Lai Jiang
02e43ab134 Add a task to compile javadoc across all packages (#597)
Also fixes various issues that prevent javadoc compliation.
2020-05-27 10:33:46 -04:00
Weimin Yu
5face3b2fe Add Test suite support for JUnit 5 classes (#549)
* Add Test suite support for JUnit 5 classes

Added Gradle dependencies and updated lockfiles.

Updated SqlInegrationTestSuite to use new annotations.

Migrated one member class in SqlIntegrationTestSuite (CursorDaoTest)
to JUnit 5, and verified that the new Suite runner can handle a
mixture of JUnit 4 and 5 tests in one suite.

Note that Gradle tests that run TestSuites must choose JUnit 4.
Updated core/build.gradle and integration/build.gradle.
2020-04-07 21:06:49 -04:00
Shicong Huang
79a3605e9e Supress exccesive logging message from Cloud SQL (#540)
* Supress exccesive logging message from Cloud SQL

* Upgrade package versions that were downgraded before
2020-03-31 17:57:18 -04:00
Weimin Yu
dda9b90587 Fix broken builds when Maven Central is used (#509)
* Fix broken builds when Maven Central is used

Gradle 6.2.1 apparently introduces a behavior change wrt boolean
expression: empty string used to eval to false, but now evals to
true.

Pre Gradle 6.2.1, root project's Gradle properties apparently were
not set to buildSrc. Now they are passed on to buildSrc -- mavenUrl
in buildSrc changes from null to "".

Both changes break the project when mavenUrl and/or pluginsUrl are
not set on command line.

Also added junit.jupiter-api as testCompile dependencies to projects.
This is a directly used dependency, whose absence causes a Lint
warning.
2020-03-10 11:21:03 -04:00
Lai Jiang
456eefd95f Upgrade to Gradle 6.2.1 (#501) 2020-03-05 18:47:25 -05:00
Ben McIlwain
3139a2ffee Start using JUnit 5 (#488)
* Start using JUnit 5

This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration

In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.

* Delete no longer needed lockfiles

* Merge branch 'master' into first-junit5
2020-02-19 18:29:59 -05:00
Weimin Yu
b9c63da753 Use CA cert in tests (#475)
* Use CA cert in tests

Our tests were using a self-signed cert created by a netty utility.
That cert is not CA, and cannot be used with newer JDK versions
(starting from u242 with Ubuntu openjdk).

Now we generate a unsafe cert for tests. The new cert,
SelfSignedCaCertificate, needs to be made test-only.
To do so we need to refactor both prober and proxy.
2020-02-07 10:23:24 -05:00
Lai Jiang
cfd5307cef Use monitoring client v1.0.7 (#445)
This version of the monitoring client no longer depends on backported
JUnit classes.
2020-01-14 23:11:54 -05:00
Weimin Yu
74c614d002 Upgradle JUnit to 4.13 (#442)
* Upgradle JUnit to 4.13

Removed third_party/junit folder and all usage of the
JunitBackPort class. As a result, third_party is no
longer a Gradle subproject.

Minor code changes were needed to work around an
error-prone pattern: multiple statement in assertThrows'
runnable lambda.

Also third_party/activation and third_party/jsch. These
dependencies are loaded from remote maven repo. The local
copies are not in use.
2020-01-14 10:54:09 -05:00
Lai Jiang
9df3792fa4 Use the correct certificate provider type (#427)
TESTED: tested in alpha. Previous I only  tested locally and missed this
typo.
2020-01-06 16:19:50 -05:00
Lai Jiang
69ec237f19 Consolidate certificate supplier module (#410)
* Consolidate certificate supplier module

Both the proxy and the proxy needs certificate suppliers. The PR
consolidates the module that providings those bindings to a shared
module and switched the proxy to use that module. The prober currently
uses P12 file to store its certificates. I am debating keeping that
supplier ro converting them to PEM files for simplicity.

* Rename mode enum values to be more descriptive

* Update annotation names to be more descriptive
2019-12-23 13:09:47 -05:00
Weimin Yu
1a1ad54a17 Allow project dependency to use runtimeClasspath (#395)
* Allow project dependency to use runtimeClasspath

Project dependency should use runtimeClasspath. However, if
left unspecified, it uses 'default', which is the same as
the legacy 'runtime' configuration. (runtimeOnly dependencies
are left out).

Since runtimeClasspath cannot be referenced directly, we use
a custom config (deploy_jar) as a proxy.

By excluding testjars (leaked into 'compile' by third-party
dependencies) from runtimeClasspath, we prevent them from
getting into release artifacts.

Two meaningful changes in appengine_war.gradle and java_common.gradle

TESTED=Diffed contents of services/{module}/build/exploded-*
       Only three jars are removed: hamcrest-core, junit, and
       mockito-core.
2019-12-02 16:10:13 -05:00
Lai Jiang
1c1ccee75e Respect certificate validity period (#391)
Client SSL handler already performs the necessary validation. Only tests are
added.

Server SSL handler does not currently check for the validity period of
the client certificate as the insecure trust manager is used. This PR
added the check but does not actually terminate the connection yet. It
will log the expired certificates so that we can contact the registrars
to update them.

Once we are certain that all certificates are updated, we can turn off
dryrun mode.

We should also consider checking if the certificate has too long a
validity period as it defeats the purpose of using regularly updated
certificates to deprecate insecure cipher suites.
2019-11-27 16:08:38 -05:00
Lai Jiang
bb88ef621f Remove SSL initializer from the prober (#378)
The prober now uses the common SSL initializer in the networking
subproject.

Also changed both initializers to take an ImmutableList of certificates
other than an array of those, for better immutability.

I have no idea where these lockfile changes are coming from. They seem
to be pure noise as far as code review is concerned.
2019-11-22 17:46:06 -05:00
Lai Jiang
de02996f00 No-op: Use nicer HCL2 syntax. (#384)
Generated with perl -pi -e 's/\"\$\{([a-zA-Z0-9._-]*)\}\"/$1/g' $(find ./ -name '*.tf')

Copied from cl/282012376.
2019-11-22 16:08:56 -05:00
Weimin Yu
9359f40665 Break circular dependency between core and util (#379)
* Break circular dependency between core and util

Created a new :common project and moved a minimum
number of classes to break the circular dependency
between the two projects. This gets rid of the
gradle lint dependency warnings.

Also separated api classes and testing helpers into
separate source sets in :common so that testing
classes may be restricted to test configurations.
2019-11-21 15:36:55 -05:00
Lai Jiang
4c11e92ca8 Refactor common code used by the proxy and the prober (#375) 2019-11-20 12:42:44 -05:00
Lai Jiang
d2ebb591a2 Upgrade to Gradle 6.0 (#364) 2019-11-13 13:17:29 -05:00
Lai Jiang
c3d93ab173 Make Joda Money embeddable in entities (#340) 2019-11-07 17:03:00 -05:00
Ben McIlwain
6a2a5b4dbd Add a converter for CurrencyUnits stored in the database (#334)
* Add a converter for CurrencyUnits stored in the database

This uses the well-known String representation for currency units. It also
provides a base class for other converters that will be persisting the
toString() representation.

* Add DB and formatting changes

* Add tests, make minor fixes
2019-10-31 15:26:40 -04:00
Shicong Huang
7099652159 Upgrade org.apache.beam related packages to latest version (#333) 2019-10-30 11:44:26 -04:00
Lai Jiang
0667994b77 Instrument proxy frontend latency metric (#320)
* Instrument proxy frontend latency metric

Backend latency only captures the time spent waiting for the GAE backend
to respond to a request, which is not representitive of what clients
experience. For instance it does not take into account the time the
proxy spends on processing the requests and the time it takes to send a
response to the client.

This PR adds a metric for frontend latency, which is the time spent on
answering a client request. This should serve as a better proxy for the
latency clients observe.

* Rename AbstractMetrics to BaseMetrics
2019-10-29 16:02:47 -04:00
Weimin Yu
3763cc285d Add a test task for all schema-dependent tests (#325)
Such tests are used in server-schema compatibility tests.

Also added a test that verifies that all eligible tests
are included.
2019-10-25 13:25:01 -04:00
Lai Jiang
a6aa1ca9fe Protect KMS-secured data against destruction in upcoming google provider update. (#284)
Export of cl/270900150.

To refer to a KMS key or key ring, we should use the stable `.self_link`. Using `.id` instead provides an unstable identifier which may change (and it will change in the upcoming update of the google provider to 2.9.1). A change in the identifier will cause Terraform to destroy and recreate the key. Destroying the key means all data associated with it is lost; the key cannot be recreated.

This CL replaces `.id` with `.self_link`, so all of those problems will not happen. In addition, `prevent_destroy` protects the key against delete-and-recreate in general.
2019-09-27 12:12:58 -04:00
Weimin Yu
7db99e3308 Upgrade to Truth 1.0 (#281)
* Upgrade to Truth 1.0

Refactored fail(...) to assertWithMessage().fail().

Upgraded com.google.monitoring-client family of dependencies to 1.0.6

Also fixed bad use of io.StringIO (on binary buffer) recently introduced to
google-java-format-diff.py.
2019-09-24 10:23:58 -04:00
Shicong Huang
a85fcb6237 Make project core depend on db at test runtime (#278)
This PR also upgraded the version of Guava to 28.1-jre to
resolve missing stream package issue.
2019-09-23 14:12:24 -04:00
Weimin Yu
89404dc43d Fix dependency-locking config (#276)
* Fix dependency-locking config

Reenable dependency locking after a bug errorneouly turned it off.

Removed the guava-related workaround that forcefully resolve to
the -jre distribution.

Enabled locking for buildSrc by updating its property file.

Updated all lock files.
2019-09-19 11:37:20 -04:00
Lai Jiang
1779c63882 Update proxy deployment script description (#245) 2019-08-28 17:35:43 -04:00
Lai Jiang
43dc3d7c47 Fix expired access token bug in the proxy (#217)
https://github.com/google/nomulus/pull/129 migrated `GoogleCredential`
to `GoogleCredentialsBundle` and introduced a subtle bug. I don't fully
understand why but there are times when the access token is null but
`credentials.refresh()` is not called, resulting in NullPointerException
when `credentials.getAccessToken().getTokenValue()` is called.

Since the new GoogleCredentials class supports `shouldRefresh()`, we now
just rely on it to make sure that we always get a value access token.
2019-08-08 14:44:33 -04:00
Lai Jiang
8240dee691 Add a Gradle task to deploy the proxy (#214)
Also refactored some common code regarding the mapping between
environments and projects.

The script to update proxy deployments and kill all pods are lifted from
https://cs.corp.google.com/piper///depot/google3/domain/registry/tools/bashrc?l=163
2019-08-06 11:45:34 -04:00
gbrodman
2a381b7071 Clean up Gradle stylings and fix issues IDed in Prober (#212) 2019-08-05 15:54:20 -04:00