* Restore ofy keys in DomainTransferData
Restore composite VKeys correctly in DomainTransferData (they were previously
missing their ofy keys).
* Use "AlsoLoad" to populate history ids
* Use the parent to store the history repo ID and fill in the base object
Storing the repo ID in the parent and in the base object has two primary
benefits.
First, it unifies the parent information in the HistoryEntry's `parent`
object. This simplifies the builders and the data flow.
Second, when possible (which should be always, post-migration) we fill
out the DomainContent's repo ID (similarly for the other EPP resources)
which means that when reconstituting the ofy keys we don't need to pass
the repo ID in from a separate object. This way, all the data are
encapsulated where they should be.
The primary downside here is that it further reduces the "immutability"
of the history objects (since we're using the Hibernate setter for the
parent repo ID) but we weren't immutable anyway.
* Respond to CR
- compare the entire vkeys in tests
- always return the parent for repo ID
* Simplify creation of parent VKeys
* Fix flipped isAssignableFrom check in VKey
* Merge remote-tracking branch 'origin/master' into historyRepoId
* Restore composite vkeys in DomainContent
PollMessage/BillingEvent vkeys in DomainContent must have their ofy keys
restored from other fields in DomainContent (namely the repo id and their
specific history event ids).
Add PostLoad methods to DomainContent and DomainHistory to do the restoration.
* Fixes for review.
* Deal with foreign-key cycles
* CertificateChecker with checks for expiration and key length
* Add validity length check
* Get rid of hard-coded constants and DSA checks
* add files that for some reason weren't included in last commit
* Rename violations and other fixes
* Add displayMessage to CertificateViolation enum
* Switch violations from an enum to a class
* small changes
* Get rid of ECDSA checks
* add checks for old validity length
* Change error message for validity length
When loading the VKeys for the BillingEvents hierarchy, it is necessary to
restore the original concrete class for the type, otherwise we end up with a
different (and incompatible) VKey.
As part of this, convert the cancellation matching billing event to
VKey<Recurring>, which seems like the only thing it actually can be.
Add a restoreOfy() instance method and a restoreOfyFrom() static method to
assist in restoring the objectify key for classes that have composite keys
that do not restore automatically.
Without it we kept getting the following warning:
ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
* Allow explicitly for null EPP resources in History objects
* Repo IDs should always be nonnull
* Add a test to verify loading / comparison of legacy HistoryEntry objects
* Format javadoc + annotations
* More javadoc changes
* V52 -> V56
* V56 -> V57
* saveNew -> insert in new tests
* Improve naming of TransactionManager methods
Per internal discussion, convert names of methods as follows:
saveNew -> insert
saveNewOrUpdate -> put
checkExists -> exists
Likewise, convert derived names to their corresponding forms, e.g.
saveNewOrUpdateAll -> putAll.
* Add domain-specific history fields to DomainHistory objects
* Add javadoc for Hibernate-only methods
* V52 -> V54
* Use only a single DomainTransactionRecord table
* Add nullables and fix up a comment
* V54 -> V55
* Regenerate db schema
* Regen SQL file
* Create a separate per-tld registry lock/unlock cost
Currently we use the standard server status change cost for this, but
this might not be ideal at some point in the future if we wish to allow
manual forced updates outside of the standard registry lock system (we
would charge for these manual forced updates, even if we don't charge
for registry locks).
* Remove period
1. It appears that when we have a 200 response, the response content is
garbled, but we don't care since we know the request is successful.
When we have a 400 response, the response is indeed UTF-8 encoded.
Print the stack trace of the HTTP exception doesn't help anymore.
2. The result code is a complex type which includes the xml element with
all its attributes, which don't care. We only want to print the
value.
* Add TLD table
* Change reservedLists to array
* Change ReservedLists back to a set
* Rename reservedListKeyConverter to ReservedListKeySetConverter
* Add a postload method
See b/168508962 for context.
1. Changed the contact to iana-contact@google.com
2. Changed the header from "Script" to "Language" for zh-Hans and
zh-Hant.
3. Commented out the references in zh-Hans and Zh-Hant
* Fix accessing superclass fields in checkExists()
JpaTransactionManagerImpl doesn't respect @Id fields in mapped superclasses.
Replace calls to getDeclaredId() and getDeclaredField() with superclass
friendly counterparts.
* Change disable invoicing flag to enable invoicing flag
This flag will be the sole determinor on if invoicing is enabled,
regardless of TLD types.
Once this PR is deployed we will need to run the nomulus command to
update this flag on all launched open TLDs.
For context on why this change is made, see b/159626744.
* Rename enableInvoicing to InvoicingEnabled
* Implement DatastoreEntity/SqlEntity for many more classes
We still have many more classes to go, but this gets us closer to
guaranteeing that we can convert from Datastore to SQL objects and back
again.
* Shift SqlEntity impl to HistoryEntry
* Make ICANN reporting not fail on success upload
According to the spec
(https://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-13#page-16),
when an upload succeeds (HTTP response code 200), the result code
contained in the response message is always 1000 (success). So there is
no need to parse the response content and check the result code. Given
that we are having a problem parsing the response content due to encoding,
it is best that we don't check it so as to not get false negative
alerts when the upload is successful.
The current logic also has a bug: HttpRequest.execute() will by default
throw when the response code is non-20X. Therefore for a 400 response,
our parsing logic never runs on it. Coincidentally, this month when we
uploaded the July activity report (due to stale cursors), we get 400
responses (due to existing reports on the ICANN servers). The stack
trace printed for the thrown exceptions from the 400 responses contained
correctly parsed response contents. This lead us to believe that the issue with
encoding was transient last month. However when we tried again to upload this
month's report, our parser failed again (because the response code was 200 this
time, and our parser actually ran on the response contents).
This seems to suggest that ICANN is sending back readable response
contents, but our parser somehow failed to understand it, assuming that
ICANN is using the same encoding for 200 (which we tried and failed to
parse) and 400 response contents (which caused an exception and was printed
corrected in the stack trace).
This PR changed the transport behavior so that it doesn't throw
automatically for non-20X responses. We will print the content for both
200 and 400 responses, but only try to parse 400 response content. We
put the 400 response in an HttpResponseException and print stack trace
from it, which should display the content correctly so that we can
compare it with the result of our own parsing.
* Add tests
* Use composite primary key for DomainHistory
* Move History table's SequenceGenerator to orm.xml
* Rebase on HEAD and remove default value for key in History tables
* Use primitive type for id.
* Revert the cache change
* Fix flaky web driver tests
Identified two flaky tests in RegistrarConsoleScreenshotTest through
local testing and fixed them by waiting for specific web elements instead
of using fixed delays.
Refactored the wait methods to support different test scenarios,
and removed unnecessary delays.
Extensively tested locally. Also ran multiple presubmits on Kokoro.
* Add success/failure notifications for the RelockDomainAction
If a relock fails for some reason, we should noisily notify both our
alerting email and also the registry lock contacts for the registrar in
question. The consequences of a silent failure could be large so it's
something we want to avoid if at all possible.
In addition, we only retry tasks up to two times (one in 5min, one in
10min).
This model of retries / notifications, as well as the language contained
in the emails, have been LGTMed by Bruno and Kirsten
* Change the wording on the success email
* Change the times in which we send emails
For transient failures:
- Retry every ten minutes for six hours
- Send an email after a half hour (three failures) saying that we'll
retry
- Send a success email if we succeed any time after that
For non-transient failures:
Send an email with the error message and don't retry
* Add a test for the max-failure-email
* Responses to CR
- retry indefinitely
- send an email to just the alert address if we can't find the lock
- refactor the task enqueuer a bit
* non-transient -> non-retryable
* Use a lenient stubber for the AESU
* Add a DS transaction around the re-lock
* Persist *History objects as HistoryEntry objects
While Datastore is the primary database, we will store *History objects
as HistoryEntry objects and convert to/from the proper objects in the
Datastore transaction manager. This means that History objects will not
properly store the copy of the EppResource until we move to SQL as
primary, but this is the way the world exists anyway so it's not a
problem.
* Format code and simplify the bulk loading
* Add comments with context
* Fix JPA setup in Nomulus tool
Hibernate unnecessarily scans third-party classes in the Nomulus tool,
hitting a bug and fails to set up.
In this change we properly configured persistence.xml to include the orm mapping file (orm.xml) and disable
auto detection, and provided a custom (NOOP) scanner
to work around Hibernate scanner bugs.
Also improved on the :core:registryIntegrationTest task to test for
JPA setup as well as dependency-packaging.
* Fix missing-driver in HibernateSchemaExporterTest
HibernateSchemaExporterTest is failing with "Driver not found" error
after Java 11 upgrade. Reason is that ServiceLoader now only checks
modules for services.
Proper fix is to define modules.
This short term fix is to declare the driver class explicitly.
* Add check to prevent creating VKeys with incorrectly null parents
Datastore entities that are not the roots of entity groups are uniquely defined
by an inheritance chain in addition to the id/name, not just by id/name.
* Fix empty Domain nameserver loads
Domains with no nameservers were being loaded from SQL as an empty set instead
of null as they should be.
Discovered this will trying to test updates, so added a test for updates in
the course of it.
