* Update to generic Spec11ThreatMatch table
* Fix SQL syntax
* Make changes to the schema and add a test for null and empty threatTypes
* Fix a small typo
* Change the exception thrown with illegal arguments
Change the import for isNullOrEmpty
* Fix import for checkArgument
* Added a threat to test multiple threat types
* Squash everything together
Create SafeBrowsing_Threats table
Create LocalDateConverter and add indexes to SafeBrowsingThreats
Add indexes to SafeBrowsingThreats and make small style changes
Pass in DateTimeFormatter
Delete LocalDateConverterTest.java
Rebase
Make changes to ThreatType comments
Create LocalDateConverterTest
Add review changes
Add SafeBrowsingThreatTest
Rename repoId, refactor LocalDateConverterTest/SafeBrowsingThreatTest, add foreign keys
Change imports
Add foreign keys and rename version number
Add new generated db-schema file
Clean up null test cases
Add changes
Add foreign keys into SafeBrowsingThreatTeat and apply style checks
Add SafeBrowsingThreatTest into SqlIntegrationTestSuite and change golden file
Make small changes to SafeBrowsingThreatTest
Add tests for ForeignKeyViolations and remove setId in SafeBrowsingThreat
* Change V35 -> V36
* Add a foreign key test for a reference to Registrar
* Move some variables around
* Add testcontainers' Junit5 support dependency
Also updated guava, dagger, hibernate, postgresql, and cloud socket factory
to latest version.
Migrated PersistenceModuleTest as an example.
Real changes:
- dependencies.gradle
- core/build.gradle
- PersistenceModuleTest.java
* Remove 'fullyQualified' from host and domain names
We don't actually enforce that these are properly fully-qualified
(there's no dot at the end) and we specifically use the term "label
name" when talking about labels.
Note: this doesn't convert FQDN -> DN (et al) in at least two types of
cases:
1. When the term is part of the XML schema
2. When the term is used by some external system, e.g. SafeBrowsing API
* Add TODO to rename fields
* Add proof of concept for HostBase and HostHistory classes
* Use a PROPERTY accessor for @Ids
* Add an unused setter method for Hibernate's sake
* Refactor HostHistory
* Some responses to CR
* Fix relationship and test
* Manually manage the foreign keys for HostHistory
* Protect HostBase's builder and use text for the enum type
* Add responses to CR
- Add javadocs
- Create an ID sequence for host history objects
* Don't try to set the ID
* Use a Long and remove the setter
* Add some comments and rename a couple fields
* Don't change Datastore schema
* Use Long in the Datastore schema
* Add new createVKey method
* Add comments and rename fields
* Rename v27->v31 and regenerate the golden
* Fix superordinateDomain and inetAddresses in HostHistory
* V31 -> V32
* Fix SQL files that got messed up in the merge
* Configure and use a manually-created history ID sequence
* Add three more indices to HostHistory
* Add DomainTransferData and ContactTransferData
* Refactor TransferData to remove unused fields in Contact table
* Add scope for TransferData's type parameter
We'll eventually want to shift everything over to using registrar_id and
registrarId rather than client_id and clientId but for the sake of the
Datastore schema and existing code, we won't change the Java identifier
for now. Once we're completely and only on SQL, we can rename the Java
field easily.
* Create a converter for sets of inetAddresses and use it in HostResource
This can just be a set of strings where each string represents an
address; there's no need for it to be a separate table. This allows
for simplification of the SQL schema.
* Regenerate golden SQL file after renaming v28 -> v29
* Add more tests and rename a typo in the file
* Refactor common test code and use tm methods
* Use JUnit5 API
* Rename test entity
* Add columns for TransferData in Domain and Contact
* Rename flyway file and foreign key
* Rebase on master and address comment
* Compileable commit
* Fix unit test
* Refactor TransferServerApproveEntity
* Use tm().delete(vkeys)
* Rename transfer_period fields
* Rename client_id to registrar_id
* Rebase on master
* Resolve comment
* Rebase on master
* Store the superordinateDomain reference as a VKey rather than Key
This is a reference to a Domain object, so we should store it as a VKey
in reference to the Domain table. This should not affect any business
logic, but rather will allow us to set up the SQL tables for
HostResource et al. properly.
* Generate sql schema for BillingEvent
* Change to use sequence
* Address comments
* Resolve warnings and remove duplicate cost related fields
* Increase the flayway file version to V25
* Remove extra space
* Split to 3 tables, merge VKey
* Rename talbes
* Rename repoId to domainRepoId
* Exclude VKey in schema.txt
* Rename target_id to domain_name
* Fix javadoc
* Resolve comments
* Make VKey persist to datastore as a key
Convert nsHosts entirely to VKey as a proof-of-concept.
Tested as follows:
1) Deployed to crash, verified that nameservers were visible for several
domains (indicating that we are able to load a set of Keys as VKeys)
2) Updated the set of nameservers for a domain (removing some initial
hosts) and verified that the changes went through.
3) Deployed the old version to crash, verified that I was able to retrieve
the newly saved VKeys as Keys.
4) Modified the hosts for the same domain (adding back one of the hosts)
and verified that the change took effect.
5) Redeployed this change to crash, again updated the nameservers to add
another host.
6) Again restored the old version, verified that the new hosts were
visible.
* Changes in response to review
* Convert to a single VKeyTranslatorFactory instance
* Moved vkey field rename to V25
* Convert DomainBase's contacts to VKeys
Convert usage of DomainBase contacts from Key to VKey. This is the same
change as done for nameserver hosts, as it affects all external interfaces.
As with nameserver hosts, we preserve the existing representation so as not to
afffect the datastore representation.
* Add JPA annotations to ContactResource and generate schema
* Resolve comments
* Resolve comments
* Manually add foreign key constraints
* Run with junit5
* Rebase on HEAD
* Fix DomainBaseSqlTest
* Add Test suite support for JUnit 5 classes
Added Gradle dependencies and updated lockfiles.
Updated SqlInegrationTestSuite to use new annotations.
Migrated one member class in SqlIntegrationTestSuite (CursorDaoTest)
to JUnit 5, and verified that the new Suite runner can handle a
mixture of JUnit 4 and 5 tests in one suite.
Note that Gradle tests that run TestSuites must choose JUnit 4.
Updated core/build.gradle and integration/build.gradle.
* Add a registryLockEmailAddress field to RegistrarConctact objects
Because we need to manage the login email, it should be on an account
that we manage. However, for registry lock, we would want to send the
verification emails to a separate email address that the user can use.
As a result, we will use a second field for a user-accessible registry
lock email address. This must be set on the contact when enabling
registry lock for this contact.
* Responses to CR
* derp
* Add a relockDuration to the RegistryLock SQL object
This is the length of time after an unlock that we will re-lock the
domain in question.
* Sort by domain name for stability
Note: this is likely not the best solution for the UI but we can iterate
on this.
* Add nullable
* Add a converter for Duration
* Add a RelockAction and reference to relocks in RegistryLocks
* Respond to CR
- refactor the request param exception logging a bit
- don't log an error if the domain was already locked, just skip
* Save a relock for all locks (if possible)
* derp
* Long -> long + remove unnecessary transact
* semantic merge conflict woo
* fix another semantic merge conflict
* Fix broken builds when Maven Central is used
Gradle 6.2.1 apparently introduces a behavior change wrt boolean
expression: empty string used to eval to false, but now evals to
true.
Pre Gradle 6.2.1, root project's Gradle properties apparently were
not set to buildSrc. Now they are passed on to buildSrc -- mavenUrl
in buildSrc changes from null to "".
Both changes break the project when mavenUrl and/or pluginsUrl are
not set on command line.
Also added junit.jupiter-api as testCompile dependencies to projects.
This is a directly used dependency, whose absence causes a Lint
warning.
* Start using JUnit 5
This converts a single test class over to JUnit 5 (YamlUtilsTest). The main
differences you'll notice are that @RunWith isn't needed anymore, test classes
and test methods can now be package-private, and the @Test annotation comes from
the org.junit.jupiter.api package instead of org.junit. There's a lot more
differences between 4 and 5 than this that we'll need to keep in mind when
converting more test classes; for some more details, see:
https://www.baeldung.com/junit-5-migration
In order to allow JUnit 4 and 5 test classes to coexist, I've had to add two new
dependencies, org.junit.jupiter:junit-jupiter-engine and
org.junit.vintage:junit-vintage-engine, which exist in addition to junit:junit
for now. Eventually, once we've completed migrating over all JUnit 4 test
classes, then we can remove junit and junit-vintage-engine and just be left with
junit-jupiter-engine.
* Delete no longer needed lockfiles
* Merge branch 'master' into first-junit5
* Add the schema and DAO for Locks in CloudSQL
* Addresses some comments
* Change number on flyway file
* Small changes
* More small changes
* Use checkArgumentNotNull instead of checkNotNull
* Address comments
* fix javadocs
* update persistence
* Add JPA annotations to class Registrar
* Use array for Java list
* Exclude parent field
* Use 3 columns for address and use text for enum
* Use EnumParameter and 3 properties in Address
* Rename columns and rebase on HEAD
* Get persistence of DomainBase actually working
Fix all of the existing problems with DomainBase persistence:
- Remove "final" keywords on getters that cause errors during startup.
- Remove Transient from creationTime (since there's a converter for
CreateAutoTimestamp)
- Fix DesignatedContext persistence so that it only creates a single table.
This is a lot more efficient given that these are many-to-one with their
domains.
- Add a flyway script, update the golden schema.
- Create a unit test, add it to the integration test suite.
* Changes request in review
* Regenerated generated schema file.
* Changes for review
* Persist status value enum set
* Changes in response to review
* Changes requested in review
* Fixes for #456
* Rename Domain "status" column to "statuses"
* Upgradle JUnit to 4.13
Removed third_party/junit folder and all usage of the
JunitBackPort class. As a result, third_party is no
longer a Gradle subproject.
Minor code changes were needed to work around an
error-prone pattern: multiple statement in assertThrows'
runnable lambda.
Also third_party/activation and third_party/jsch. These
dependencies are loaded from remote maven repo. The local
copies are not in use.
* Add unlock fields to RegistryLocks
This will make it easier to reason around inter-connected registry lock
objects (like when we add dependent roids). It will make it easier to
answer the question of "Have all locks associated with this host/contact
roid been unlocked?", as well as the question of "Was the last lock
object associated with this domain unlocked?"
* Responses to CR
* Make the DAO API more specific
* whoops, undo rename
* Fix sql script name conflict
There are two V11__ files due to concurrent merge. Renamed one
to V12__
Also removed a @NotNull annotation, which is the fist in the code base.
Most of the code base use @Nullable instead. If we do want to use
@NotNull, we may want to use the javax one instead.
* Run cross-release SQL integration tests
Run SQL integration tests across arbitrary schema and server
releases.
Refer to integration/README.md in this change for more information.
TESTED=Cloud build changes tested with cloud-build-local
Used the published jars to test sqlIntegration task locally.
* Add Cloud SQL premium list caches and compare prices with Datastore
Nothing will fail if the prices can't be loaded from Cloud SQL, or if the prices
are different. All that happens is that the error is logged. Then, once this is
running in production for awhile, we'll look at the logs and see if there will
be any pricing implications from switching over to the Cloud SQL version of the
premium lists.
* Add setMaxResults(1) per code review
* Add tests and reorder public functions
* Don't statically import caches
* Improve test pass rate
* Merge branch 'master' into dual-read-premium
* Add PremiumEntry mapping
* Allow update
* Revert column order
* Alphabetize PremiumEntry columns
* Don't bother trying to enforce order
* Private constructor
* Make devProject a project property
Properties set in rootProject's ext block are not overridden by
commandline flags.
* Make flyway commands runnable from FOSS repo
rootProject.devProject, defined in projects.gradle, cannot be
overridden by commandline flags. Added a flag check in :db's
project script.
* Add schema for Cursor
* Add CursorDao and CursorDaoTest
* Fix comment on getTld
* Change tld column to scope
* Fix cursorTime to be converted to DateTime internally and other small fixes
* Add a CursorType enum and a createGlobal constructor for Cursor
* Rename flyway file
* Use cursorType from common/Cursor.java and add null checks
* Allow project dependency to use runtimeClasspath
Project dependency should use runtimeClasspath. However, if
left unspecified, it uses 'default', which is the same as
the legacy 'runtime' configuration. (runtimeOnly dependencies
are left out).
Since runtimeClasspath cannot be referenced directly, we use
a custom config (deploy_jar) as a proxy.
By excluding testjars (leaked into 'compile' by third-party
dependencies) from runtimeClasspath, we prevent them from
getting into release artifacts.
Two meaningful changes in appengine_war.gradle and java_common.gradle
TESTED=Diffed contents of services/{module}/build/exploded-*
Only three jars are removed: hamcrest-core, junit, and
mockito-core.
* Make dev project configurable
We should not hardcode our dev project in the public config file.
* Remove the use of .ext when using external properties
They are only needed when defining properties.
* Require explict tag when starting psql docker
Defined a util class to return docker tag of desired PSQL version.
Class is defined in ':db' and shared by ':db' and ':core'. Used
an artifact declaration to exclude unnecesary compile dependencies.
Added a presubmit check for instantiations without explicit tag.
* Update schema deployment doc and flyway tool
Disabled Flyway Gradle tasks with side effects on Cloud SQL
instances. They can still be used on local databases.
Also switched Flyway Gradle tasks to get credentials from
new locations (in domain-registry-dev).
Updated README file on schema push process. Also reformatted
the entire file.
