* Add a cursor for tracking monthly uploads of the transaction report to ICANN
* Add cursors to track activity, transaction, and manifest report uploads.
* Address comments
* Add @Nullable annotation to manifestCursor
* Add lock and batch load cursors.
* Add string formatting, autovalue CursorInfo object, and handling for null cursors
* Add some helper functions for loadCursors and restructure to require less round trips to the database
* Switch new cursors to be created with cursorTime at first of next month
* Make dev project configurable
We should not hardcode our dev project in the public config file.
* Remove the use of .ext when using external properties
They are only needed when defining properties.
* Update the Registries cache to leverage/populate the Registry cache
This is accomplished by also providing a loadAll() method on the Registry cache
that can be used to load an entire batch of Registry objects at once.
This improves efficiency, because now, any operation on Registries that loads
all the Registry entities (getTlds(), getTldsOfType(), and getTldEntities()), is
plumbed through the Registry cache, therefore loading it from that cache if it
exists and only hitting the DB if not. If not, this populates the Registry cache
upon loading, so that subsequent calls to Registry.get() will now hit the cache
instead of the DB.
To give a concrete example, the following code:
for (String tld : Registries.getTlds()) {
// ...
doSomethingWith(Registry.get(tld));
// ...
}
is now much more efficient, because the initial call to Registries.getTlds()
populates all the entities in cache, and the subsequent individual calls to
Registry.get(tld) now retrieve them from the cache. Prior to this change,
Registries.getTlds() did not populate the Registry cache, and each subsequent
Registry.get(tld) had the potential to trigger an individual round-trip to the
DB, which is obviously bad for performance.
* Break circular dependency between core and util
Created a new :common project and moved a minimum
number of classes to break the circular dependency
between the two projects. This gets rid of the
gradle lint dependency warnings.
Also separated api classes and testing helpers into
separate source sets in :common so that testing
classes may be restricted to test configurations.
* Allow schema-loading from arbitrary url in tests
Server/Schema compatibility tests must be able to load different versions
of the SQL schema. This change allows test runners to override the
schema location using a system property.
Note: due to dependency-locking, we cannot manipulate the dependencies
closure in the build script to load different schema jars. The jars
must not be on the classpath.
* Add a new method to load all Registry entities of a given type
This is useful for things that need to know more than just the TLD strings
themselves, which is all Registries currently provides.
* Require explict tag when starting psql docker
Defined a util class to return docker tag of desired PSQL version.
Class is defined in ':db' and shared by ':db' and ':core'. Used
an artifact declaration to exclude unnecesary compile dependencies.
Added a presubmit check for instantiations without explicit tag.
* Update schema deployment doc and flyway tool
Disabled Flyway Gradle tasks with side effects on Cloud SQL
instances. They can still be used on local databases.
Also switched Flyway Gradle tasks to get credentials from
new locations (in domain-registry-dev).
Updated README file on schema push process. Also reformatted
the entire file.
* Add by-repo-id method to RegistryLockDao
When we create new registry lock objects on user request, we will want
to make sure that there are no pending lock/unlock actions on this
domain. In order to do that, we will need to know what lock actions
there have been for this domain.
* Inline the query into a single statement
* whoops
* comments
Replace deprecated bouncycastle class in SslInitializerTestUils.
Generic array as vargs: worked around it in ProbingAction and
removed unused method in CircularList.
Defined Docker image for schema deployment.
Included schema deploymer docker in the Cloud Build release process.
Defined Cloud Build config for schema deployment.
TESTED=Used cloud-build-local to test deployment flow
TESTED=Used docker to test schema deployer image in more ways
Implementation of CircularList is an overkill for its stated
purpose. Remove an unnecessary usage so that the remaining
use case may be refactored in isolation.
* Refactor UI actions to be more reliable
- Created HtmlAction to handle the nitty-gritty of login and setting up
HTML pages
- Created a test to verify that all UI actions implement JSON GET, JSON
POST, or HTML classes
- Move CSS renaming into a utility class
* Move logging of request into HtmlAction
* Comment and wording in exception
* mention JsonGetAction in the comment
* JsonGetAction extends Runnable
* Run both the AppEngineRule and JpaTransactionManagerRule in tests
As previously written, the AppEngineRule wraps the infinite loop of the
server handling requests. This changes the code so that we'll have a
chain that does both the AppEngine management and the JPA transaction
manager management so we can actually use SQL in tests and in the test
server.
* remove log line
We should communicate to the users why this command failed, that they
are not allowed to use discounted allocation tokens on premium domains.
Currently it still fails, but we don't yet tell them why.
* Add a GET action and tests for registry lock retrieval
* Create isVerified method
* Allow lock access for admins even if they're not enabled on the registrar
* Simple CR responses
* Move locks retrieval to the GET action
* add newline at eof
* Switch to using ID
* Improve FilteringTest so it works in all cases
- Disable failOnMatchingTests so we don't fail if we filter out all of the
tests for any given task.
- Add excludeTestCases flag so that we can turn this behavior off for test
tasks that need to run TestCas/TestSuite classes.
- Add includeAllTests() function for test tasks where we don't explicitly
include our required tests.
This makes all test tasks in core FilteringTest, with the exception of the
default test task which now does nothing (it just depends on the other test
tasks).
* Improve wording of excludeTestCases comment
When we add the extra test entity to the current JpaTransactionManagerRule by calling jpaRule.withEntityClass(TestEntity.class) and jpaRule.withProperty(Environment.HBM2DDL_AUTO, "update"), the rule would override the golden database scheme with the schema derived from the current entity class(This is an expected behavior by enabling HBM2DDL_AUTO). This behavior prevents us from detecting breaking changes in ORM classes.
This PR fixed the issue by adding HibernateSchemaExporter to export the DDL script for given extra entity class, and make JpaTransactionManagerRule execute the DDL script to create the corresponding table for the extra entity when initializing the database. This PR also simplified the code when adding extra entity class for testing. For now, you don't need to(and shouldn't) call jpaRule.withProperty(Environment.HBM2DDL_AUTO, "update").
* Move soyutils_usegoog.js out of node_modules
Everytime the npmInstall runs, it removes this file from node_modules.
Move it outside the folder to prevent this from happening.
* Move karma.conf.js and soyutils_usegooge.js
* Move karma.conf.js to be under core
* Release SQL schema in Cloud Build
Tentatively release SQL schema at the same time as the server release.
Publish schema jar to gs://domain-registry-maven-repository/nomulus
and also upload it with server artifacts.
Also removed the Gradle 'version' variable which is not used.
Tested=On cloud-build with a simplified version of
cloudbuild-nomulus.yaml.
* Don't wrap exceptions thrown inside Cloud SQL transactions
There's no reason to wrap them in PersistenceException, and it makes dealing
with thrown exceptions harder as seen in the diffs on test classes in this
commit.
* Add a converter for CurrencyUnits stored in the database
This uses the well-known String representation for currency units. It also
provides a base class for other converters that will be persisting the
toString() representation.
* Add DB and formatting changes
* Add tests, make minor fixes
* Save release tag during deployment
* Save current tag for every environment
Store tag of the current deployment in each environment.
This is used by the server-sql compatibility test.
* Save current tag for every environment
Store tag of the current deployment in each environment.
This is used by the server-sql compatibility test.
* Instrument proxy frontend latency metric
Backend latency only captures the time spent waiting for the GAE backend
to respond to a request, which is not representitive of what clients
experience. For instance it does not take into account the time the
proxy spends on processing the requests and the time it takes to send a
response to the client.
This PR adds a metric for frontend latency, which is the time spent on
answering a client request. This should serve as a better proxy for the
latency clients observe.
* Rename AbstractMetrics to BaseMetrics
"Public Domain" license must be reviewed case by case.
Removed blanket acceptance and named accepted dependencies
individually.
Also added a README file to warn about this license and WTFPL.
* Don't retry permanent failures when uploading ICANN monthly reports
There are two kinds of permanent failures that this checks for that we know will
never succeed, so it makes no sense to continue retrying 11 more times before
moving onto the next file to upload. These errors are:
1.
com.google.api.client.http.HttpResponseException: 403
Your IP address xx.xx.xx.xx is not allowed to connect
2.
com.google.api.client.http.HttpResponseException: 400
<?xml version="1.0" encoding="UTF-8" standalone="yes"?><response xmlns="urn:ietf:params:xml:ns:iirdea-1.0"><result code="2002"><msg>A report for that month already exists, the cut-off date already passed.</msg><description>Date: 2019-09</description></result></response>
In order to implement this new functionality, this commit also adds a new way to
call Retriable that allows specifying the isRetryable Predicate (which is quite
useful).
