* Add SQL schema and DAO for SignedMarkRevocationList
This gets saved every day so we're not concerned about history, meaning
we can dual-write and/or dual-read without concern. The structure here
is somewhat similar to the ClaimsListDao and related classes.
* Update the DB files
* Sync the live folder after Nomulus rollback
To update the nomulus tool on corp desktop, the artifacts from the
rollback target release should be copied to the 'live' folder.
* Fix a test
* An automated rollback tool for Nomulus
A tool that directs traffic between deployed versions. It handles the
conversion between Nomulus tags and AppEngine versions, executes schema
compatibility tests, ensures that steps are executed in the correct order,
and updates deployment records appropriately.
* Restore ofy keys in GracePeriod objects
Restore the ofy keys when loading GracePeriod object from SQL. There's no
clear way to do this using the normal approach (fix-up during a PostLoad
method) because fixups to these violate immutability after hibernate has
already obtained their hash values. Instead, we force reconstitution of the
ofy keys in all public methods that access them (including equals() and
hashCode()) so that they can be generated before an invalid hash is generated.
As part of this change, convert the GracePeriod id from an autogenerated
sequence to a UUID allocated from ObjectifyService and enhance ImmutableObject
to allow it to exclude certain fields from hash/equals and print.
The ImmutableObject enhancements are necessary because we compare grace
periods against locally created test objects in a number of unit tests and
there's no way this can work with GracePeriods loaded from SQL currently, as
they will have an identifier field generated from the database and the test
objects will have an identifier field of null (or a new unique value, after
this change).
Removing autogeneration from GracePeriod ids ended up being likely not
strictly necessary for this change (it was a consequence of an earlier
iteration). However, it does alleviate the problem of mutation of an
immutable object after creation and is more in line with how we've decided to
allocate other identifiers.
* Changed needed after rebase.
* Add certificate checks to RegistrarSettingsAction
* Add some comments
* Add more functionality to CertificateChecker and update call sites
* Small code cleanups
* Small format fix
* Create SQL schema for RdeRevision
* Split RdeRevision IDs into three separate DB fields as unified pkey
* Rename variable
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Rename variable in one other location
* Implement no-op toDatastore/Sql for RdeRevision
* Responses to CR
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Use a date for the date column
* Fix exception messages in tests
* Regen diagram to fix the test
* Use assignment in static factory methods
* Merge remote-tracking branch 'origin/master' into rdeRevision
* Upgrade error-prone to 3.3.4
This would fix the failure with openjdk 11.0.9 in
3.3.3.
Fixed new antipatterns raised by the new version:
- Replaced unnecessary lambdas with methods.
- Switched wait/sleep calls to equivalent methods using java.time types
- Types inheriting Object.toString() should not be assigned to string
parameter in logging statements.
* Add certificatechecks for create and update registrar commands
* Add CertificateCheckerModule
* Remove commented out code
* Still tring to get dependency injection to work
* Get this actually working
* Add tests for multiple violations
* Small formatting fixes
* Rename configs and fix collectors
* Add checks for failover client certificate
* Fix formatting
* Restore ofy keys in DomainTransferData
Restore composite VKeys correctly in DomainTransferData (they were previously
missing their ofy keys).
* Use "AlsoLoad" to populate history ids
* Use the parent to store the history repo ID and fill in the base object
Storing the repo ID in the parent and in the base object has two primary
benefits.
First, it unifies the parent information in the HistoryEntry's `parent`
object. This simplifies the builders and the data flow.
Second, when possible (which should be always, post-migration) we fill
out the DomainContent's repo ID (similarly for the other EPP resources)
which means that when reconstituting the ofy keys we don't need to pass
the repo ID in from a separate object. This way, all the data are
encapsulated where they should be.
The primary downside here is that it further reduces the "immutability"
of the history objects (since we're using the Hibernate setter for the
parent repo ID) but we weren't immutable anyway.
* Respond to CR
- compare the entire vkeys in tests
- always return the parent for repo ID
* Simplify creation of parent VKeys
* Fix flipped isAssignableFrom check in VKey
* Merge remote-tracking branch 'origin/master' into historyRepoId
* Minor python changes
Use dataclasses instead of attrs. The former is part of the standard lib
while the latter may need to be installed separately.
Also added python3 to the list of prerequisites.
* Maintain a release-to-Version map in deployment
Keep track of the mapping between Nomulus release tags and AppEngine
version ids with a mapping file. This is necessary because AppEngine
does not support custom versioning. With this mapping, rollbacks could
be automated. Automation of rollbacks is important since there are
test-supporting metadata to be updated, but are easily forgotten.
During the last stage of deployment, current per-service version ids
are fetched using gcloud and are appended to a file on GCS. Each line
is of the format "{RELEASE_TAG},{APPENGINE_SERVICE},{APPENGINE_VERSION}.
This change has been tested in crash. The rollback script is still a
work in progress.
* Convert CertificateViolation into an enum
This ends up being nicer to deal with from callsites than class instances, while
still permitting full configurability of all parameters. There are various other
changes/fixes as well.
* Restore composite vkeys in DomainContent
PollMessage/BillingEvent vkeys in DomainContent must have their ofy keys
restored from other fields in DomainContent (namely the repo id and their
specific history event ids).
Add PostLoad methods to DomainContent and DomainHistory to do the restoration.
* Fixes for review.
* Deal with foreign-key cycles
* CertificateChecker with checks for expiration and key length
* Add validity length check
* Get rid of hard-coded constants and DSA checks
* add files that for some reason weren't included in last commit
* Rename violations and other fixes
* Add displayMessage to CertificateViolation enum
* Switch violations from an enum to a class
* small changes
* Get rid of ECDSA checks
* add checks for old validity length
* Change error message for validity length
When loading the VKeys for the BillingEvents hierarchy, it is necessary to
restore the original concrete class for the type, otherwise we end up with a
different (and incompatible) VKey.
As part of this, convert the cancellation matching billing event to
VKey<Recurring>, which seems like the only thing it actually can be.
Add a restoreOfy() instance method and a restoreOfyFrom() static method to
assist in restoring the objectify key for classes that have composite keys
that do not restore automatically.
* Create a flyway index file and verify correctness
Create an index file (flyway.txt) containing the names of all of the flyway
files and verify that it is ordered and in sync with the actual contents of
the flyway directory. Also provide a target (generateFlywayIndex) to
automatically generate it.
The purpose of flyway.txt is to cause a merge conflict in the event that two
different developers add a flyway file with the same sequence number, an event
which has occurred multiple times.
Without it we kept getting the following warning:
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
* Enhance the test for forbidden Schema changes
Current test is git-based. It is difficult to maintain and does not
catch out-of-order version numbers. It is also more aggressive than
necessary, failing on changes to submitted scripts that have not been
deployed yet.
The new test starts a database, deploys the current schema to it,
then deploys the set of Flyway scripts in this repository to the database.
* Allow explicitly for null EPP resources in History objects
* Repo IDs should always be nonnull
* Add a test to verify loading / comparison of legacy HistoryEntry objects
* Format javadoc + annotations
* More javadoc changes
* V52 -> V56
* V56 -> V57
* saveNew -> insert in new tests
* Improve naming of TransactionManager methods
Per internal discussion, convert names of methods as follows:
saveNew -> insert
saveNewOrUpdate -> put
checkExists -> exists
Likewise, convert derived names to their corresponding forms, e.g.
saveNewOrUpdateAll -> putAll.
* Add domain-specific history fields to DomainHistory objects
* Add javadoc for Hibernate-only methods
* V52 -> V54
* Use only a single DomainTransactionRecord table
* Add nullables and fix up a comment
* V54 -> V55
* Regenerate db schema
* Regen SQL file
* Create a separate per-tld registry lock/unlock cost
Currently we use the standard server status change cost for this, but
this might not be ideal at some point in the future if we wish to allow
manual forced updates outside of the standard registry lock system (we
would charge for these manual forced updates, even if we don't charge
for registry locks).
* Remove period
1. It appears that when we have a 200 response, the response content is
garbled, but we don't care since we know the request is successful.
When we have a 400 response, the response is indeed UTF-8 encoded.
Print the stack trace of the HTTP exception doesn't help anymore.
2. The result code is a complex type which includes the xml element with
all its attributes, which don't care. We only want to print the
value.
* Add TLD table
* Change reservedLists to array
* Change ReservedLists back to a set
* Rename reservedListKeyConverter to ReservedListKeySetConverter
* Add a postload method
