* Allow explicitly for null EPP resources in History objects
* Repo IDs should always be nonnull
* Add a test to verify loading / comparison of legacy HistoryEntry objects
* Format javadoc + annotations
* More javadoc changes
* V52 -> V56
* V56 -> V57
* saveNew -> insert in new tests
* Improve naming of TransactionManager methods
Per internal discussion, convert names of methods as follows:
saveNew -> insert
saveNewOrUpdate -> put
checkExists -> exists
Likewise, convert derived names to their corresponding forms, e.g.
saveNewOrUpdateAll -> putAll.
* Add domain-specific history fields to DomainHistory objects
* Add javadoc for Hibernate-only methods
* V52 -> V54
* Use only a single DomainTransactionRecord table
* Add nullables and fix up a comment
* V54 -> V55
* Regenerate db schema
* Regen SQL file
* Create a separate per-tld registry lock/unlock cost
Currently we use the standard server status change cost for this, but
this might not be ideal at some point in the future if we wish to allow
manual forced updates outside of the standard registry lock system (we
would charge for these manual forced updates, even if we don't charge
for registry locks).
* Remove period
1. It appears that when we have a 200 response, the response content is
garbled, but we don't care since we know the request is successful.
When we have a 400 response, the response is indeed UTF-8 encoded.
Print the stack trace of the HTTP exception doesn't help anymore.
2. The result code is a complex type which includes the xml element with
all its attributes, which don't care. We only want to print the
value.
* Add TLD table
* Change reservedLists to array
* Change ReservedLists back to a set
* Rename reservedListKeyConverter to ReservedListKeySetConverter
* Add a postload method
See b/168508962 for context.
1. Changed the contact to iana-contact@google.com
2. Changed the header from "Script" to "Language" for zh-Hans and
zh-Hant.
3. Commented out the references in zh-Hans and Zh-Hant
* Fix accessing superclass fields in checkExists()
JpaTransactionManagerImpl doesn't respect @Id fields in mapped superclasses.
Replace calls to getDeclaredId() and getDeclaredField() with superclass
friendly counterparts.
* Change disable invoicing flag to enable invoicing flag
This flag will be the sole determinor on if invoicing is enabled,
regardless of TLD types.
Once this PR is deployed we will need to run the nomulus command to
update this flag on all launched open TLDs.
For context on why this change is made, see b/159626744.
* Rename enableInvoicing to InvoicingEnabled
* Implement DatastoreEntity/SqlEntity for many more classes
We still have many more classes to go, but this gets us closer to
guaranteeing that we can convert from Datastore to SQL objects and back
again.
* Shift SqlEntity impl to HistoryEntry
* Make ICANN reporting not fail on success upload
According to the spec
(https://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-13#page-16),
when an upload succeeds (HTTP response code 200), the result code
contained in the response message is always 1000 (success). So there is
no need to parse the response content and check the result code. Given
that we are having a problem parsing the response content due to encoding,
it is best that we don't check it so as to not get false negative
alerts when the upload is successful.
The current logic also has a bug: HttpRequest.execute() will by default
throw when the response code is non-20X. Therefore for a 400 response,
our parsing logic never runs on it. Coincidentally, this month when we
uploaded the July activity report (due to stale cursors), we get 400
responses (due to existing reports on the ICANN servers). The stack
trace printed for the thrown exceptions from the 400 responses contained
correctly parsed response contents. This lead us to believe that the issue with
encoding was transient last month. However when we tried again to upload this
month's report, our parser failed again (because the response code was 200 this
time, and our parser actually ran on the response contents).
This seems to suggest that ICANN is sending back readable response
contents, but our parser somehow failed to understand it, assuming that
ICANN is using the same encoding for 200 (which we tried and failed to
parse) and 400 response contents (which caused an exception and was printed
corrected in the stack trace).
This PR changed the transport behavior so that it doesn't throw
automatically for non-20X responses. We will print the content for both
200 and 400 responses, but only try to parse 400 response content. We
put the 400 response in an HttpResponseException and print stack trace
from it, which should display the content correctly so that we can
compare it with the result of our own parsing.
* Add tests
* Use composite primary key for DomainHistory
* Move History table's SequenceGenerator to orm.xml
* Rebase on HEAD and remove default value for key in History tables
* Use primitive type for id.
* Revert the cache change
* Fix flaky web driver tests
Identified two flaky tests in RegistrarConsoleScreenshotTest through
local testing and fixed them by waiting for specific web elements instead
of using fixed delays.
Refactored the wait methods to support different test scenarios,
and removed unnecessary delays.
Extensively tested locally. Also ran multiple presubmits on Kokoro.
* Add success/failure notifications for the RelockDomainAction
If a relock fails for some reason, we should noisily notify both our
alerting email and also the registry lock contacts for the registrar in
question. The consequences of a silent failure could be large so it's
something we want to avoid if at all possible.
In addition, we only retry tasks up to two times (one in 5min, one in
10min).
This model of retries / notifications, as well as the language contained
in the emails, have been LGTMed by Bruno and Kirsten
* Change the wording on the success email
* Change the times in which we send emails
For transient failures:
- Retry every ten minutes for six hours
- Send an email after a half hour (three failures) saying that we'll
retry
- Send a success email if we succeed any time after that
For non-transient failures:
Send an email with the error message and don't retry
* Add a test for the max-failure-email
* Responses to CR
- retry indefinitely
- send an email to just the alert address if we can't find the lock
- refactor the task enqueuer a bit
* non-transient -> non-retryable
* Use a lenient stubber for the AESU
* Add a DS transaction around the re-lock
* Persist *History objects as HistoryEntry objects
While Datastore is the primary database, we will store *History objects
as HistoryEntry objects and convert to/from the proper objects in the
Datastore transaction manager. This means that History objects will not
properly store the copy of the EppResource until we move to SQL as
primary, but this is the way the world exists anyway so it's not a
problem.
* Format code and simplify the bulk loading
* Add comments with context
* Fix JPA setup in Nomulus tool
Hibernate unnecessarily scans third-party classes in the Nomulus tool,
hitting a bug and fails to set up.
In this change we properly configured persistence.xml to include the orm mapping file (orm.xml) and disable
auto detection, and provided a custom (NOOP) scanner
to work around Hibernate scanner bugs.
Also improved on the :core:registryIntegrationTest task to test for
JPA setup as well as dependency-packaging.
* Fix missing-driver in HibernateSchemaExporterTest
HibernateSchemaExporterTest is failing with "Driver not found" error
after Java 11 upgrade. Reason is that ServiceLoader now only checks
modules for services.
Proper fix is to define modules.
This short term fix is to declare the driver class explicitly.
* Add check to prevent creating VKeys with incorrectly null parents
Datastore entities that are not the roots of entity groups are uniquely defined
by an inheritance chain in addition to the id/name, not just by id/name.
* Fix empty Domain nameserver loads
Domains with no nameservers were being loaded from SQL as an empty set instead
of null as they should be.
Discovered this will trying to test updates, so added a test for updates in
the course of it.
* Add a SQL schema to AllocationToken
* Respond to CR
- rename field in tests
- rename allowed_registrar_ids field
- remove unnecessary db load in GATC
* Add TODO for HistoryEntry vkeys
* Run autoformat
* V48 -> V49
For some inexplicable reasons I have to move the javax.mail package one
spot up to avoid its classes being shadowed by those provided in the
appengine package...
* Create DAO for Spec11ThreatMatch
* Add tests
* Execute SQL for deleteEntriesByDategit status
* Remove testing line
* Rename createSpec11ThreatMatch()
* Add comments about jpaTm and use jpaTm() in test
* Fix technicality in comment
* Remove a new line
* Truth chaining for comparing ImmutableLists of matches
* Javadoc formatting
* Add some missing @Nullables and types
Also deletes two unused VKey.createOfy() methods that simply don't work, because
a kind and an id is not enough to create a Datastore key; you also need the full
entity group inheritance chain for entities that are not roots themselves
(which is most of the entities in our schema).
* Merge branch 'master' into add-missing-nullables
* Throw UnsupportedOperationException for contacts/hosts too
* Merge branch 'master' into add-missing-nullables
* Update BEAM SDK to work with Java 11
Upgraded BEAM dependencies to 2.23.0.
Updated Spec11 and invoice pipelines:
- Added the required region parameter.
- Removed the workaround code for staging.
Verified that staging is successful in alpha:
./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_spec11_pipeline --project domain-registry-alpha'
and
./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_invoicing_pipeline'
* Double the timeout length for WebDriver screenshot tests
My theory is that this timeout is being applied to all retries of a failing test
rather than each one, and thus flaky screenshot tests aren't being given
sufficient time to complete any attempt past the first.
This test is flaky and the going theory is that we take a screenshot
immediately after we click the "Add" button and the CSS hover effect is
somehow still playing on the button.
* Restore original jpa manager after tests
AppEngineExtensionTest fails to restore original jpa manager in some
tests. This results in flakiness in DummyJpaTransactionManagerTest.
