google-nomulus/core
Michael Muller 6d26f3cc0e Support testing SQL -> DS replication in ReplayExt (#1216)
* Support testing SQL -> DS replication in ReplayExt

Support testing of Postgres -> Datastore replication in the ReplayExtension
when running in SQL mode in a DualDatabaseTest.

This is currently only enabled for one test (HostInfoFlowTest) since this form
of replication is likely to be problematic in many cases.

As part of this change:

- Add a thread-local flag so that we don't attempt to do certain data
  transformations when serializing entities for storage in a Transaction
  record. (These typically need to be called in a datastore transaction).
- Replace tm() in datastore translators with ofyTm() (these should only be
  called from within an ofy transaction) and also in the replay system itself.
- Add a transactWithoutBackup() method for use within the replay itself.
- Prevent replication of entities that are not intended to be replicated.
- Make some of the ReplicateToDatastoreAction methods public so we can invoke
  them from ReplayExtension.
- Change the way that the test type is stored in the extension context in a
  DualDatabaseTest so that we can check for it from the ReplayExtension.

* Limit number of tests and show output

Trying to debug why these are failing in kokoro.

* Move HostInfoFlowTest to fragile for now

The test now manipulates a globel variable that causes problems for other
tests.  There's likely a better fix for this, but for purposes of this PR we
can just move it to "fragile."

* Fix a few more problems

-   "replay" flag should have been initialized to false -- as it stands,
    replay wasn't happening.
-   disable "always save with backup" in the datastore helper, we were
    apparently getting some unwanted commit log entries that were causing
    timestamp inversions in other tests.  Also clear out the replay queue
    just for good hygiene.
-   Check for a null replicator in replayToOfy before proceeding.
-   Use a local inOfyContext flag to track whether we're in ofy context, as
    the tm() function is less reliable in dual-database tests.
2021-06-29 10:00:39 -04:00
..
gradle/dependency-locks Update GCL dependency to avoid security alert (#1139) 2021-05-17 13:21:26 -04:00
src Support testing SQL -> DS replication in ReplayExt (#1216) 2021-06-29 10:00:39 -04:00
WEB-INF/appengine-generated Enable Cloud SQL when Datastore is enabled for unit test (#502) 2020-03-10 12:26:25 -04:00
build.gradle Support testing SQL -> DS replication in ReplayExt (#1216) 2021-06-29 10:00:39 -04:00
Dockerfile Build docker image of nomulus tool (#142) 2019-07-16 20:18:44 -04:00