Commit graph

17 commits

Author SHA1 Message Date
Lai Jiang
bc62e13e41 Implement a util class to manage push queues using Cloud Tasks API (#1290)
* Implement a util class to manage push queues using Cloud Tasks API

Push queues were part of App Engine when they debuted. As a result the
Task Queue API were part of the App Engine SDK and can only be used in
App Engine classic runtime. The new Cloud Tasks API can be used in any
runtime but it only supports push queues. In this PR we implement a util
class (CloudTasksUtils) like TaskQueueUtils to handle enqueuing tasks to
push queues using Cloud Tasks. One action (TldFanoutAction) was
converted to use the new API as a demo. Mass migration of other call sites of
the old API will follow in a separate PR.

TESTED=deployed to alpha and verified that tasks are corrected enqueued
and executed.
2021-08-24 21:13:54 -04:00
Lai Jiang
65ae5f4a30 Remove the use of GCS APIs provided from GAE SDK (#1228)
The API provided by the GAE SDK will not be available outside GAE
runtime. This presents a problem when we migrate off of GAE. More
pressingly, the RDE pipeline migration to Beam requires that we write to
GCS on GCE. Previously we were able to sidestep the issue by delegating
the writes to FileIO provided by Beam, which knows how to write to GCS.
However the RDE pipeline cannot use FileIO directly as it needs to write
to multiple files in one go and explicit use of GCS API is needed.

An unfortunate side effect of the API migration is that the new testing
library contains a bug which makes serializing GcsUtils impossible. It
is fixed upstream but not released yet. The fix has been backported for
the time being.

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1228)
<!-- Reviewable:end -->
2021-07-13 14:52:37 -04:00
Ben McIlwain
8162933fca Update more dependencies to newer versions (#999)
* Update more dependencies to newer versions

* Add lockfiles and back out 2 problematic dep updates

* Fix the build (backs out more changes)

* Back out qdox 2.0 too
2021-03-10 11:22:07 -05:00
Ben McIlwain
603024d11d Update a lot of Cloud API dependencies (#994)
Hopefully this will help us to resolve our 'gRPC is too old' warning (gRPC isn't
a direct dependency so it's likely coming in through these).
2021-03-09 13:24:41 -05:00
Weimin Yu
aafe11847d Properly set up JPA in BEAM workers (#965)
* Properly set up JPA in BEAM workers

Sets up a singleton JpaTransactionManger on each worker JVM for all
pipeline nodes to share.

Also added/updated relevant dependencies. The BEAM SDK version change
caused the InitSqlPipeline's graph to change.
2021-02-22 18:11:32 -05:00
Weimin Yu
31c16e8369 Add a Secret Manager client for Nomulus (#872)
* Add a Secret Manager client for Nomulus
2020-11-12 17:12:52 -05:00
Lai Jiang
6a5dbf8817 Update dependencies to get rid of some illegal reflection warnings (#768)
For some inexplicable reasons I have to move the javax.mail package one
spot up to avoid its classes being shadowed by those provided in the
appengine package...
2020-08-14 16:53:22 -04:00
Weimin Yu
3ac5f06991 Add testcontainers' Junit5 support dependency (#640)
* Add testcontainers' Junit5 support dependency

Also updated guava, dagger, hibernate, postgresql, and cloud socket factory
to latest version.

Migrated PersistenceModuleTest as an example.

Real changes:
- dependencies.gradle
- core/build.gradle
- PersistenceModuleTest.java
2020-06-22 13:12:58 -04:00
Lai Jiang
e76da2c66d Update Netty to the latest version (#620)
* Upgrade to the latest version of Netty

* Update lock files
2020-06-10 16:08:11 -04:00
Shicong Huang
79a3605e9e Supress exccesive logging message from Cloud SQL (#540)
* Supress exccesive logging message from Cloud SQL

* Upgrade package versions that were downgraded before
2020-03-31 17:57:18 -04:00
Ben McIlwain
3139a2ffee Start using JUnit 5 (#488)
* Start using JUnit 5

This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration

In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.

* Delete no longer needed lockfiles

* Merge branch 'master' into first-junit5
2020-02-19 18:29:59 -05:00
Lai Jiang
cfd5307cef Use monitoring client v1.0.7 (#445)
This version of the monitoring client no longer depends on backported
JUnit classes.
2020-01-14 23:11:54 -05:00
Lai Jiang
bb88ef621f Remove SSL initializer from the prober (#378)
The prober now uses the common SSL initializer in the networking
subproject.

Also changed both initializers to take an ImmutableList of certificates
other than an array of those, for better immutability.

I have no idea where these lockfile changes are coming from. They seem
to be pure noise as far as code review is concerned.
2019-11-22 17:46:06 -05:00
Lai Jiang
d2ebb591a2 Upgrade to Gradle 6.0 (#364) 2019-11-13 13:17:29 -05:00
Weimin Yu
7db99e3308 Upgrade to Truth 1.0 (#281)
* Upgrade to Truth 1.0

Refactored fail(...) to assertWithMessage().fail().

Upgraded com.google.monitoring-client family of dependencies to 1.0.6

Also fixed bad use of io.StringIO (on binary buffer) recently introduced to
google-java-format-diff.py.
2019-09-24 10:23:58 -04:00
Shicong Huang
a85fcb6237 Make project core depend on db at test runtime (#278)
This PR also upgraded the version of Guava to 28.1-jre to
resolve missing stream package issue.
2019-09-23 14:12:24 -04:00
Weimin Yu
89404dc43d Fix dependency-locking config (#276)
* Fix dependency-locking config

Reenable dependency locking after a bug errorneouly turned it off.

Removed the guava-related workaround that forcefully resolve to
the -jre distribution.

Enabled locking for buildSrc by updating its property file.

Updated all lock files.
2019-09-19 11:37:20 -04:00