Commit graph

19 commits

Author SHA1 Message Date
Lai Jiang
65ae5f4a30 Remove the use of GCS APIs provided from GAE SDK (#1228)
The API provided by the GAE SDK will not be available outside GAE
runtime. This presents a problem when we migrate off of GAE. More
pressingly, the RDE pipeline migration to Beam requires that we write to
GCS on GCE. Previously we were able to sidestep the issue by delegating
the writes to FileIO provided by Beam, which knows how to write to GCS.
However the RDE pipeline cannot use FileIO directly as it needs to write
to multiple files in one go and explicit use of GCS API is needed.

An unfortunate side effect of the API migration is that the new testing
library contains a bug which makes serializing GcsUtils impossible. It
is fixed upstream but not released yet. The fix has been backported for
the time being.

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1228)
<!-- Reviewable:end -->
2021-07-13 14:52:37 -04:00
Ben McIlwain
8162933fca Update more dependencies to newer versions (#999)
* Update more dependencies to newer versions

* Add lockfiles and back out 2 problematic dep updates

* Fix the build (backs out more changes)

* Back out qdox 2.0 too
2021-03-10 11:22:07 -05:00
Ben McIlwain
603024d11d Update a lot of Cloud API dependencies (#994)
Hopefully this will help us to resolve our 'gRPC is too old' warning (gRPC isn't
a direct dependency so it's likely coming in through these).
2021-03-09 13:24:41 -05:00
Weimin Yu
aafe11847d Properly set up JPA in BEAM workers (#965)
* Properly set up JPA in BEAM workers

Sets up a singleton JpaTransactionManger on each worker JVM for all
pipeline nodes to share.

Also added/updated relevant dependencies. The BEAM SDK version change
caused the InitSqlPipeline's graph to change.
2021-02-22 18:11:32 -05:00
Lai Jiang
0263428b41 Upgrade to Gradle 6.8.2 (#957) 2021-02-08 09:35:43 -05:00
Weimin Yu
31c16e8369 Add a Secret Manager client for Nomulus (#872)
* Add a Secret Manager client for Nomulus
2020-11-12 17:12:52 -05:00
Weimin Yu
9ddde4799c Upgrade error-prone to 3.3.4 (#848)
* Upgrade error-prone to 3.3.4

This would fix the failure with openjdk 11.0.9 in
3.3.3.

Fixed new antipatterns raised by the new version:
- Replaced unnecessary lambdas with methods.
- Switched wait/sleep calls to equivalent methods using java.time types
- Types inheriting Object.toString() should not be assigned to string
parameter in logging statements.
2020-10-23 11:17:57 -04:00
Lai Jiang
f6749ad663 Enable Java 11 features (#737)
* Enable Java 11 features

As of this commit Java 11 must be used to build. The generated bytecode
is still at Java 8 due to App Engine task queue limit.

Also fixed a bug where the included google-java-format jar file is not
used, requiring the user to install it separately.

See: https://cloud.google.com/appengine/docs/standard/java/taskqueue/push
2020-08-03 16:08:29 -04:00
Ben McIlwain
0d857f8291 Get rid of all remaining JUnit 4 usages except in prober & proxy (#731)
* Get rid of all remaining JUnit 4 usages except in prober & proxy subprojects

Caveat: Test suites aren't yet implemented in JUnit 5 so we still use the ones
from JUnit 5 in the core subproject.

* Fix some build errors
2020-07-30 20:29:00 -04:00
Lai Jiang
043c0aaeb8 Clean up generated sources configuration (#724)
1. The Gradle apt plugin is no longer needed to process annotations.
2. Without the apt plugin, Gralde puts the source files generated by
   annotation processors in build/generated/sources/annotationProcessor.
3. Change the location of custom generated files to be consistent.
4. Fix a javadoc formatting error.
2020-07-30 15:43:42 -04:00
Ben McIlwain
10b23e590e Convert RDE tests (and some test rules) from JUnit 4 to JUnit 5 (#677)
* Add JUnit Params and start using it

* Convert rest of RDE tests

* Don't check headers for generated tests

* Expand visibility to fix build breakage

* Bump JUnit versions to 5.6.2
2020-07-10 21:32:36 -04:00
Weimin Yu
3ac5f06991 Add testcontainers' Junit5 support dependency (#640)
* Add testcontainers' Junit5 support dependency

Also updated guava, dagger, hibernate, postgresql, and cloud socket factory
to latest version.

Migrated PersistenceModuleTest as an example.

Real changes:
- dependencies.gradle
- core/build.gradle
- PersistenceModuleTest.java
2020-06-22 13:12:58 -04:00
Weimin Yu
5face3b2fe Add Test suite support for JUnit 5 classes (#549)
* Add Test suite support for JUnit 5 classes

Added Gradle dependencies and updated lockfiles.

Updated SqlInegrationTestSuite to use new annotations.

Migrated one member class in SqlIntegrationTestSuite (CursorDaoTest)
to JUnit 5, and verified that the new Suite runner can handle a
mixture of JUnit 4 and 5 tests in one suite.

Note that Gradle tests that run TestSuites must choose JUnit 4.
Updated core/build.gradle and integration/build.gradle.
2020-04-07 21:06:49 -04:00
Lai Jiang
456eefd95f Upgrade to Gradle 6.2.1 (#501) 2020-03-05 18:47:25 -05:00
Ben McIlwain
3139a2ffee Start using JUnit 5 (#488)
* Start using JUnit 5

This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration

In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.

* Delete no longer needed lockfiles

* Merge branch 'master' into first-junit5
2020-02-19 18:29:59 -05:00
Weimin Yu
74c614d002 Upgradle JUnit to 4.13 (#442)
* Upgradle JUnit to 4.13

Removed third_party/junit folder and all usage of the
JunitBackPort class. As a result, third_party is no
longer a Gradle subproject.

Minor code changes were needed to work around an
error-prone pattern: multiple statement in assertThrows'
runnable lambda.

Also third_party/activation and third_party/jsch. These
dependencies are loaded from remote maven repo. The local
copies are not in use.
2020-01-14 10:54:09 -05:00
Weimin Yu
b6daafd341 Use TextDiffSubject to compare multi-line text (#406)
* Use TextDiffSubject to compare  multi-line text

It illustrates differences better.

Moved TextDiffSubject.java to the common project for sharing.
2019-12-10 16:44:59 -05:00
Weimin Yu
1a1ad54a17 Allow project dependency to use runtimeClasspath (#395)
* Allow project dependency to use runtimeClasspath

Project dependency should use runtimeClasspath. However, if
left unspecified, it uses 'default', which is the same as
the legacy 'runtime' configuration. (runtimeOnly dependencies
are left out).

Since runtimeClasspath cannot be referenced directly, we use
a custom config (deploy_jar) as a proxy.

By excluding testjars (leaked into 'compile' by third-party
dependencies) from runtimeClasspath, we prevent them from
getting into release artifacts.

Two meaningful changes in appengine_war.gradle and java_common.gradle

TESTED=Diffed contents of services/{module}/build/exploded-*
       Only three jars are removed: hamcrest-core, junit, and
       mockito-core.
2019-12-02 16:10:13 -05:00
Weimin Yu
9359f40665 Break circular dependency between core and util (#379)
* Break circular dependency between core and util

Created a new :common project and moved a minimum
number of classes to break the circular dependency
between the two projects. This gets rid of the
gradle lint dependency warnings.

Also separated api classes and testing helpers into
separate source sets in :common so that testing
classes may be restricted to test configurations.
2019-11-21 15:36:55 -05:00