* Make Jacoco work with multiple test tasks
By default Jacoco only looks at execution data from
the 'test' task. This is a problem to 'core' which
has multiple test sets.
A helper script to run server/schema compatibility tests from
a CI platform. It allows the caller to choose a system under
test (sql or nomulus) and to specify the deployed version
by environment (sandbox or production).
* Implement changes suggested by testcontainer dev
See https://github.com/google/nomulus/issues/401
Specifically:
- Use getContainerIpAddress() instead of localhost to insulate us from
off-machine docker usage.
- Remove shutdown hook to close the container, as testcontainers does this for
us.
* Relax premium list existence check to allow Cloud SQL migration
We need to be able to simultaneously update premium lists that already exist in
Datastore and create them in Cloud SQL (because they haven't been migrated over
yet). This temporarily relaxes the existence check for Cloud SQL so that
"updates" will work even when the list doesn't yet exist there.
* Fix IcannReportingUploadAction to upload reports from the previous month
getFileName now sets the file name of the report to upload to use the month before cursor time.
IcannReportingUploadAction no longer uploads the MANIFEST.txt file since it is not required based on (https://tools.ietf.org/html/draft-lozano-icann-registry-interfaces-07#page-9) and the previous implementation of this action did not upload it.
Deletes the ICANN_UPLOAD_MANIFEST cursor since it is no loner needed.
* Add the ICANN_UPLOAD_MANIFEST cursor back
* Fix flakiness in JodaMoneyConverterTest
JpdaMoneyConverterTest relies on Hibernate to deploy its schema.
This introduces an extra jdbc connection in the middle of a test
suite, and may break the connection count checks between tests made
by JpaTransactionManagerRule.
This change updated HibernateSchemaExporter to include Hibernate
proprietary mappings in META-INF/orm.xml.
This change also disabled Hibernate schema push for all tests,
and enabled sql statement logging.
* Support premium list updating in Cloud SQL
This also removes the requirement to specify --also_cloud_sql in nomulus premium
list tooling, instead always persisting to Cloud SQL. It removes a non-USD
premium label in the global test premium list (the Cloud SQL schema doesn't
support a mix of currency units in a single premium list). And it adds a method
to PremiumListDao to grab the most recent version of a given list.
* Merge branch 'master' into premium-lists-always-cloud-sql
* Revert test change
* Create new PremiumListUtils class and refactor out existing method
* Fix tests and update an existing premium price
* Add unlock fields to RegistryLocks
This will make it easier to reason around inter-connected registry lock
objects (like when we add dependent roids). It will make it easier to
answer the question of "Have all locks associated with this host/contact
roid been unlocked?", as well as the question of "Was the last lock
object associated with this domain unlocked?"
* Responses to CR
* Make the DAO API more specific
* whoops, undo rename
* Consolidate certificate supplier module
Both the proxy and the proxy needs certificate suppliers. The PR
consolidates the module that providings those bindings to a shared
module and switched the proxy to use that module. The prober currently
uses P12 file to store its certificates. I am debating keeping that
supplier ro converting them to PEM files for simplicity.
* Rename mode enum values to be more descriptive
* Update annotation names to be more descriptive
* Separate JPA rules for unit and integration tests
Define two subclasses of JpaTransactionManagerRule, one for unit
tests and the other for integration tests. The difference is that
the former does not need nomulus schema and need not be included
in server/schema compatibility tests.
* Separate JPA rules for unit and integration tests
Define two subclasses of JpaTransactionManagerRule, one for unit
tests and the other for integration tests. The difference is that
the former does not need nomulus schema and need not be included
in server/schema compatibility tests.
* Add initial support to write reserved list to Cloud SQL
* Add support to update reserved list in Cloud SQL
* Fix wrong check when override is enabled in create command
* Add sql dependent tests to the suite
* Address comment
* Make invocation of super.execute more readable
* Add test to check upper case and non-puny code label
* Move ReservedListDao related classes to schema package
* Simplify a function name
The promise should be set outside the try block because if we want
warning only, we still want the promise to be set even if the
clientCertificate.checkValidity() throws an error.
* Remove the schema jar url override for tests
Revert pull request 374: "Allow schema-loading from arbitrary url in tests".
Pull request 403 (f48e3933f5) is a more
general solution.
This reverts commit 68887d427f.
* Create a new app to hold GenerateSqlSchemaCommand
GenerateSqlSchemaCommand starts postgresql using testcontainer.
This makes junit etc a runtime dependency, allowing them to get
into release artifacts.
By moving this command to a separate tool, we can remove junit
etc as compile/runtime dependency.
* Fix sql script name conflict
There are two V11__ files due to concurrent merge. Renamed one
to V12__
Also removed a @NotNull annotation, which is the fist in the code base.
Most of the code base use @Nullable instead. If we do want to use
@NotNull, we may want to use the javax one instead.
* Run cross-release SQL integration tests
Run SQL integration tests across arbitrary schema and server
releases.
Refer to integration/README.md in this change for more information.
TESTED=Cloud build changes tested with cloud-build-local
Used the published jars to test sqlIntegration task locally.
* Add Cloud SQL premium list caches and compare prices with Datastore
Nothing will fail if the prices can't be loaded from Cloud SQL, or if the prices
are different. All that happens is that the error is logged. Then, once this is
running in production for awhile, we'll look at the logs and see if there will
be any pricing implications from switching over to the Cloud SQL version of the
premium lists.
* Add setMaxResults(1) per code review
* Add tests and reorder public functions
* Don't statically import caches
* Improve test pass rate
* Merge branch 'master' into dual-read-premium
* Add PremiumEntry mapping
* Allow update
* Revert column order
* Alphabetize PremiumEntry columns
* Don't bother trying to enforce order
* Private constructor
* Make devProject a project property
Properties set in rootProject's ext block are not overridden by
commandline flags.
* Make flyway commands runnable from FOSS repo
rootProject.devProject, defined in projects.gradle, cannot be
overridden by commandline flags. Added a flag check in :db's
project script.
* Add schema for Cursor
* Add CursorDao and CursorDaoTest
* Fix comment on getTld
* Change tld column to scope
* Fix cursorTime to be converted to DateTime internally and other small fixes
* Add a CursorType enum and a createGlobal constructor for Cursor
* Rename flyway file
* Use cursorType from common/Cursor.java and add null checks
* Verify RegistryTool can instantiate
Add a task that instantiates all command classes in RegistryTool
with runtimeClasspath.
Also make sure that runtimeClasspath is a superset of
compileClasspath.
* Adding junit back into the runtime classpath
Unfortunately, GenerateSqlSchemaCommand depends on junit via testcontainers.
We should really move GenerateSqlSchemaCommand out of nomulus tool (we only
use it during development) but this gets nomulus tool working for the time
being.
* Removed unnnecessary trace line.
* lockfiles generated after update_dependency.sh
In some cases, we define JpaTransactionManagerRule in a TestCase
class which is extended by the concrete test class. So, we need
to check if JpaTransactionManagerRule is also defined in the super
class.
Endpoints annotated with AUTH_INTERNAL_ONLY used to be accessible
manually with an internal RPC tool that adds App Engine specific HTTP
headers to a request to make it look like it comes from App Engine
(hence internal). This tool is used by admins to hit such endpoints
during debugging, making them effectively AUTH_INTERNAL_OR_ADMIN.
This RPC tool has never been made available outside Google so the open
source admins do not have such ability. A recent change in the RPC tool
made this hack stop working internally as well. This PR replaces all
all occurances of AUTH_INTERNAL_ONLY with AUTH_INTERNAL_OR_ADMIN and
brings the open source build into feature parity with the internal
version.
Also fixed a few issues the router tests.
* Allow project dependency to use runtimeClasspath
Project dependency should use runtimeClasspath. However, if
left unspecified, it uses 'default', which is the same as
the legacy 'runtime' configuration. (runtimeOnly dependencies
are left out).
Since runtimeClasspath cannot be referenced directly, we use
a custom config (deploy_jar) as a proxy.
By excluding testjars (leaked into 'compile' by third-party
dependencies) from runtimeClasspath, we prevent them from
getting into release artifacts.
Two meaningful changes in appengine_war.gradle and java_common.gradle
TESTED=Diffed contents of services/{module}/build/exploded-*
Only three jars are removed: hamcrest-core, junit, and
mockito-core.
Client SSL handler already performs the necessary validation. Only tests are
added.
Server SSL handler does not currently check for the validity period of
the client certificate as the insecure trust manager is used. This PR
added the check but does not actually terminate the connection yet. It
will log the expired certificates so that we can contact the registrars
to update them.
Once we are certain that all certificates are updated, we can turn off
dryrun mode.
We should also consider checking if the certificate has too long a
validity period as it defeats the purpose of using regularly updated
certificates to deprecate insecure cipher suites.
* Stop publish Cloud SQL schema jar to maven repo
The original purpose of the maven publication is for
use in server/schema compatibility tests. A commandline
flag can direct a test run to use different versions of
the schema jar. However, this won't work due to dependency
locking.
The prober now uses the common SSL initializer in the networking
subproject.
Also changed both initializers to take an ImmutableList of certificates
other than an array of those, for better immutability.
I have no idea where these lockfile changes are coming from. They seem
to be pure noise as far as code review is concerned.
* Add a cursor for tracking monthly uploads of the transaction report to ICANN
* Add cursors to track activity, transaction, and manifest report uploads.
* Address comments
* Add @Nullable annotation to manifestCursor
* Add lock and batch load cursors.
* Add string formatting, autovalue CursorInfo object, and handling for null cursors
* Add some helper functions for loadCursors and restructure to require less round trips to the database
* Switch new cursors to be created with cursorTime at first of next month
* Make dev project configurable
We should not hardcode our dev project in the public config file.
* Remove the use of .ext when using external properties
They are only needed when defining properties.
* Update the Registries cache to leverage/populate the Registry cache
This is accomplished by also providing a loadAll() method on the Registry cache
that can be used to load an entire batch of Registry objects at once.
This improves efficiency, because now, any operation on Registries that loads
all the Registry entities (getTlds(), getTldsOfType(), and getTldEntities()), is
plumbed through the Registry cache, therefore loading it from that cache if it
exists and only hitting the DB if not. If not, this populates the Registry cache
upon loading, so that subsequent calls to Registry.get() will now hit the cache
instead of the DB.
To give a concrete example, the following code:
for (String tld : Registries.getTlds()) {
// ...
doSomethingWith(Registry.get(tld));
// ...
}
is now much more efficient, because the initial call to Registries.getTlds()
populates all the entities in cache, and the subsequent individual calls to
Registry.get(tld) now retrieve them from the cache. Prior to this change,
Registries.getTlds() did not populate the Registry cache, and each subsequent
Registry.get(tld) had the potential to trigger an individual round-trip to the
DB, which is obviously bad for performance.
