* Make sure post load work happens in GracePeriod
The GracePeriod method with ofy @OnLoad annotation is not called.
Apparently Ofy only checks for @OnLoad on first-class entities,
not embedded ones.
Added a call to this method from DomainContent's OnLoad method.
Reproduced issue with a test and verified that the fix works.
* Rename DatastoreHelper -> DatabaseHelper
It already contains some functionality for dealing with Cloud SQL and will
increasingly contain more, so it should be renamed so that it does not falsely
imply it is specific only to Datastore.
* Allow addition of extra entity classes for VKey conversion
This allows us to create VKeys from Keys for test objects that may not
be part of the original codebase.
This isn't used anywhere directly yet but it will be useful in the
future when testing the replay of SQL transactions.
* Add an extension to verify transaction replay
Add ReplayExtension, which can be applied to test suites to verify that
transactions committed to datastore can be replayed to SQL.
This introduces a ReplayQueue class, which serves as a stand-in for the
current lack of replay-from-commit-logs. It also includes replay logic in
TransactionInfo which introduces the concept of "entity class weights."
Entity weighting allows us store and delete objects in an order that is
consistent with the direction of foreign key and deferred foreign key
relationships. As a general rule, lower weight classes must have no direct or
indirect non-deferred foreign key relationships on higher weight classes.
It is expected that much of this code will change when the final replay
mechanism is implemented.
* Minor fixes:
- Initialize "requestedByRegistrar" to false (it's non-nullable). [reverted
during rebase: non-nullable was removed in another PR]
- Store test entities (registrar, hosts and contacts) in JPA.
* Make testbed save replay
This changes the replay system to make datastore saves initiated from the
testbed (as opposed to just the tested code) replay when the ReplayExtension
is enabled. This requires modifications to DatastoreHelper and the
AppEngineExtension that the ReplayExtension can plug into.
This changes also has some necessary fixes to objects that are persisted by
the testbed (such as PremiumList).
* Make some columns nullable in History tables
xmlBytes is made nullable in all history tables since changes performed
by backend actions would not have it. In addition, epp requests are not saved to
ContactHistory since data may contain PII.
requestedByRegistrar in all history tables are made nullable. This
property is set from metadata in epp requests. Null means not provided.
* Add schema for GracePeriodHistory
Rebase on HEAD
Rebase on HEAD
Rebase on HEAD and rename column
Use OfyService to generate id
Refactor GracePeriodsSubject
Rebase on HEAD
Remove GracePeriodSubject and GracePeriodsSubject
Rebase on HEAD
Rebase on HEAD
Rebase on HEAD
Add gracePeriodHistoryRevisionId and remove some foreign key
* Rebase on HEAD
* Add SQL replay checkpoint object to Datastore
This will be part of the asynchronous commit-log replay to SQL. Whenever
we successfully export commits up to a particular time, we should
persist that time so we don't replay the same commits again (it is not
idempotent)
* Move SqlReplayCheckpoint from DS to SQL
* Responses to CR
* Small SQL persistence fixes to model classes
- Add a createVKey() method to Registry (Registry vkeys are composite)
- Add/fix toSqlEntities() methods in premium and reserved list classes.
* Remove fixes addressed by #866
* Add DuallyWrittenEntity convenience interface
For classes that are dually written we don't need to worry about replay
conversion, so this just allows us to remove unnecessary unimportant
methods
* Rename DWE to NonReplicatedEntity and enable test
* Persist two singleton entities in SQL tables
A table might not be the best place to store singleton entities, but by
doing this we ensure we can easily inspect them later and use the same
sort of persistence logic for these that we do elsewhere.
ServerSecret is stored upon retrieval so that we make sure that the same
secret is used in both Datastore and SQL (we wouldn't want to change
it).
* Responses to CR
* Don't have a separate ID for the singleton entities
* Rename secret UUID
* Rename and regenerate
* Allow InjectExtension to apply in "before each"
InjectExtension is non-standard in that, while it cleans up the changes it has
applied in its "after each" method, those changes must be applied by the test
driver or by another extension. This breaks extension ordering, which is
something we're going to need to make heavy use of for the ReplayExtension.
Allow changes to be specified (but not activated) after construction and apply
all unactivated changes during "before each."
* Fix entity annotations for a few classes
- Abstract classes shouldn't implement DatastoreEntity/SqlEntity
- We aren't persisting Modification in SQL
- Because DelegationSignerData is embedded, we don't need to worry about
converting and persisting it
- DomainDsDataHistory isn't persisted in Datastore
* Add a SQL schema and DAO for KmsSecretRevision
The dual-object nature of KmsSecret and KmsSecretRevision will not be
necessary once we have moved to SQL. In that world, the only object will
be the one now called KmsSecretRevision. KmsSecretRevision already
stores its parent so all we need to do is convert that key to the String
secretName (or from the secretName to the key, if loading from SQL) and
select the max revision ID for a given secret name.
In a future PR, we will add a dual-writing DAO to these objects and
perform the dual writes, similar to how ReservedList functions.
* Regenerate diagram
* Rename revisionId and cryptoKeyVersionName
* Fix SQL files and diagram
* Add SQL schema and DAO for SignedMarkRevocationList
This gets saved every day so we're not concerned about history, meaning
we can dual-write and/or dual-read without concern. The structure here
is somewhat similar to the ClaimsListDao and related classes.
* Update the DB files
* Restore ofy keys in GracePeriod objects
Restore the ofy keys when loading GracePeriod object from SQL. There's no
clear way to do this using the normal approach (fix-up during a PostLoad
method) because fixups to these violate immutability after hibernate has
already obtained their hash values. Instead, we force reconstitution of the
ofy keys in all public methods that access them (including equals() and
hashCode()) so that they can be generated before an invalid hash is generated.
As part of this change, convert the GracePeriod id from an autogenerated
sequence to a UUID allocated from ObjectifyService and enhance ImmutableObject
to allow it to exclude certain fields from hash/equals and print.
The ImmutableObject enhancements are necessary because we compare grace
periods against locally created test objects in a number of unit tests and
there's no way this can work with GracePeriods loaded from SQL currently, as
they will have an identifier field generated from the database and the test
objects will have an identifier field of null (or a new unique value, after
this change).
Removing autogeneration from GracePeriod ids ended up being likely not
strictly necessary for this change (it was a consequence of an earlier
iteration). However, it does alleviate the problem of mutation of an
immutable object after creation and is more in line with how we've decided to
allocate other identifiers.
* Changed needed after rebase.
* Add certificate checks to RegistrarSettingsAction
* Add some comments
* Add more functionality to CertificateChecker and update call sites
* Small code cleanups
* Small format fix
* Create SQL schema for RdeRevision
* Split RdeRevision IDs into three separate DB fields as unified pkey
* Rename variable
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Rename variable in one other location
* Implement no-op toDatastore/Sql for RdeRevision
* Responses to CR
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Use a date for the date column
* Fix exception messages in tests
* Regen diagram to fix the test
* Use assignment in static factory methods
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Upgrade error-prone to 3.3.4
This would fix the failure with openjdk 11.0.9 in
3.3.3.
Fixed new antipatterns raised by the new version:
- Replaced unnecessary lambdas with methods.
- Switched wait/sleep calls to equivalent methods using java.time types
- Types inheriting Object.toString() should not be assigned to string
parameter in logging statements.
* Add certificatechecks for create and update registrar commands
* Add CertificateCheckerModule
* Remove commented out code
* Still tring to get dependency injection to work
* Get this actually working
* Add tests for multiple violations
* Small formatting fixes
* Rename configs and fix collectors
* Add checks for failover client certificate
* Fix formatting
* Restore ofy keys in DomainTransferData
Restore composite VKeys correctly in DomainTransferData (they were previously
missing their ofy keys).
* Use "AlsoLoad" to populate history ids
* Use the parent to store the history repo ID and fill in the base object
Storing the repo ID in the parent and in the base object has two primary
benefits.
First, it unifies the parent information in the HistoryEntry's `parent`
object. This simplifies the builders and the data flow.
Second, when possible (which should be always, post-migration) we fill
out the DomainContent's repo ID (similarly for the other EPP resources)
which means that when reconstituting the ofy keys we don't need to pass
the repo ID in from a separate object. This way, all the data are
encapsulated where they should be.
The primary downside here is that it further reduces the "immutability"
of the history objects (since we're using the Hibernate setter for the
parent repo ID) but we weren't immutable anyway.
* Respond to CR
- compare the entire vkeys in tests
- always return the parent for repo ID
* Simplify creation of parent VKeys
* Fix flipped isAssignableFrom check in VKey
* Merge remote-tracking branch 'origin/master' into historyRepoId
* Restore composite vkeys in DomainContent
PollMessage/BillingEvent vkeys in DomainContent must have their ofy keys
restored from other fields in DomainContent (namely the repo id and their
specific history event ids).
Add PostLoad methods to DomainContent and DomainHistory to do the restoration.
* Fixes for review.
* Deal with foreign-key cycles
* CertificateChecker with checks for expiration and key length
* Add validity length check
* Get rid of hard-coded constants and DSA checks
* add files that for some reason weren't included in last commit
* Rename violations and other fixes
* Add displayMessage to CertificateViolation enum
* Switch violations from an enum to a class
* small changes
* Get rid of ECDSA checks
* add checks for old validity length
* Change error message for validity length
When loading the VKeys for the BillingEvents hierarchy, it is necessary to
restore the original concrete class for the type, otherwise we end up with a
different (and incompatible) VKey.
As part of this, convert the cancellation matching billing event to
VKey<Recurring>, which seems like the only thing it actually can be.
