Commit graph

4278 commits

Author SHA1 Message Date
Pavlo Tkach
f65dba8ec5 Add registry email to bcc for outgoing DNS failure emails (#1755) 2022-08-25 14:15:20 -04:00
gbrodman
dbc6cd2377 Allow UserAuthInfo to contain either old GAE Users or new console Users (#1744)
This means that LegacyAuthenticationMechanism or a to-be-created
OAuth2AuthenticationMechanism) can return a UserAuthInfo object that
contains either the GAE User or the console User as appropriate. The
goal is that the non-auth flows shouldn't have to know about which user
type it is. Note: the registry lock flow (for now) needs to know about
the separate types of auth because it is a separate level of auth from
the standard AuthenticatedRegistrarAccessor.

The AuthenticatedRegistrarAccessor code is a bit odd because the new
role system doesn't quite fit neatly into the old registrar ->
OWNER,ADMIN system but this is a fine approximation. Basically, any
new registrar role will map to the old OWNER role.
2022-08-24 14:18:32 -04:00
sarahcaseybot
a6087bf328 Add the PackagePromotion table (#1745)
* Add the PackagePromotion table

* Add long id

* Add NOT NULL

* fix formatting

* make package price non null

* Add not nulls to java file

* Fix broken tests from merge conflicts
2022-08-24 14:16:34 -04:00
gbrodman
3c0805def5 Allow usage of allocation tokens in nomulus create_domain (#1756)
Useful when doing internal registrations like get.boo
2022-08-24 13:18:53 -04:00
sarahcaseybot
c118af0d6a Add currentPackageToken on create flow (#1751)
* Add currentPackageToken on create flow

* Change to Truth8 assertion

* Add check for specified renewal behavior
2022-08-23 14:47:41 -04:00
gbrodman
ad50d6b0c7 Reset the claims list cache in any test that saves to it (#1754) 2022-08-22 15:58:45 -04:00
gbrodman
280b56bd51 Update currently-active ICANN-provided SMD test file (#1753)
The test files they provided before have expired, and they only provide
one valid currently-active test file now, so only test that one.

The test files are located at https://newgtlds.icann.org/en/about/trademark-clearinghouse/registries-registrars
2022-08-22 13:59:38 -04:00
gbrodman
7b7c1e1ddc Add main method to ResaveAllEppResourcesPipeline (#1748)
Not sure how this got missed before, I am pretty sure we tested this on
alpha.
2022-08-22 12:39:34 -04:00
Lai Jiang
9e646af9a7 Remove redundant test extension (#1752)
This extension field is already defined in the super class.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1752)
<!-- Reviewable:end -->
2022-08-21 12:15:15 -04:00
Lai Jiang
cfa5699cc3 Remove ofy support from PollMessage (#1732)
Also deletes the autorenew poll message history revision id field in
Domain, which is only needed to recreate the ofy key for the poll
message. The column already contains null values in it, making it
impossible to depend on it. The column itself will be deleted from the
schema after this PR is deployed.

The logic to update autorenew recurrence end time is changed
accordingly: When a poll message already exists, we simply update the
endtime, but when it no longer exists, i. e. when it's deleted
speculatively after a transfer request, we recreate one using the
history entry id that resulted in its creation (e. g. cancelled or rejected
transfer).

This should fix b/240984498. Though the exact reason for that bug is
still unclear to me. Namely, it throws an NPE at this line during an
explicit domain transfer approval:

https://cs.opensource.google/nomulus/nomulus/+/master:core/src/main/java/google/registry/flows/domain/DomainFlowUtils.java;l=603;bpv=1;bpt=0;drc=ede919d7dcdb7f209b074563b3d449ebee19118a

The domain in question has a null autorenewPollMessageHistoryId, but
that in itself should not have caused an NPE because we are not
operating on the null pointer. On that line the only possible way to
throw an NPE is for the domain itself to be null, but if that were the
case, the NPE would have been thrown at line 599 where we called a
method on the domain object.

Regardless of the cause, with this PR we are using an explicitly
provided history id and checking for its nullness before using it. If a
similar issue arises again, we should have a better idea why.

Lastly, the way poll message id is constructed is largely simplified in
PollMessageExternalKeyConverter as a result of the removal ofy parent
keys in PollMessage. This does present a possibility of failure when
immediately before deployment, a registrar requests a poll message and
received the old id, but by the time the registrar acks the id, the new
version is deployed and therefore does not recognize the old key. The
likelihood of this happening should be slim, and we could have prevented
it by letting the converter recognize both the old and the new key.
However, we would like to eventually phase out the old key, and in
theory a registrar could ack a poll message at any time after it was
requested. So, there is not a safe time by which all the old ids are
acked, lest we develop some elaborate scheme to keep track of which
messages were sent with an old id when requested and which of these old
ids are acked. Only then can we be truly safe to phase out the old id.
The benefit does not seem to warrant the effort. If a registrar does
encounter a situation like this, they could open a support bug to have
us manually ack the poll message for them.
2022-08-19 14:24:03 -04:00
sarahcaseybot
aea98f4bc3 Flyway files for PackagePromotion table (#1746)
* Include missing file

* Fix merge conflicts

* make package price non null
2022-08-19 12:53:58 -04:00
gbrodman
57c0c4084a Add another TestCacheExtension usage (#1750) 2022-08-19 10:17:15 -04:00
gbrodman
347d3d52c5 Add TestCacheExtension in ResourceFlowTC to fix flakes (#1749)
Basically, what's happening here is that some flow tests are adding
things to the claims list cache which is stored statically, meaning that
some other tests can pick those up when they shouldn't. By adding the
extension in RFTC, it'll clear out the caches after each test.
2022-08-18 15:04:29 -04:00
gbrodman
8c1dc16848 Add Flyway and golden files for console User object (#1747) 2022-08-17 16:48:14 -04:00
Pavlo Tkach
2ffba93a73 Add email notification when DNS update fails (#1734) 2022-08-16 12:59:08 -04:00
gbrodman
0928ad26c7 Allow anchor tenant creation via allocation token behavior (#1735)
* Allow anchor tenant creation via allocation token behavior

This also enforces that non-superusers cannot create registrations on
trademarked names prior to the sunrise period, even if they have an
allocation token with ANCHOR_TENANT behavior.
2022-08-15 12:42:16 -04:00
Lai Jiang
6d7f4ee6c6 Remove InjectExtension (#1739)
It is only used to set the clock used by Ofy, and it admits itself being
an ugly hack...

Also applied IntelliJ suggestions on touched files.
2022-08-12 14:56:53 -04:00
gbrodman
6d66f52359 Create a registry lock permission and corresponding account manager role (#1740)
* Create a registry lock permission and corresponding account manager role

This allows us to distinguish between standard account managers and
users that might have the registry lock permission. This will make the
registry lock password-setting flow easier (user can reset their
password iff they have the REGISTRY_LOCK permission, instead of having a
separate boolean) and allows us to easily determine whether or not a
user should have access to registry lock views in the UI.
2022-08-12 12:18:09 -04:00
Lai Jiang
2f9a45c91f Delete SetClockExtension (#1738)
We no longer write to commit logs, and the ReplayExtension that this
extension is supposed to be used with is already deleted.
2022-08-11 13:16:30 -04:00
Pavlo Tkach
999f7d3cae Extend IP validation test with message verification (#1736) 2022-08-10 13:27:55 -04:00
sarahcaseybot
3bae99ace6 Add allocation token to transfer command (#1737) 2022-08-09 15:18:23 -04:00
sarahcaseybot
644090450e Add CurrentPackageToken to Domain table (#1720)
* Add allocation token to Domain table

* Add tests

* Change column name

* change test names
2022-08-09 11:23:21 -04:00
Lai Jiang
50acadcbe6 Remove unused columns in BillingCancellation (#1721)
We stopped using these columns after BillingVKey is removed in
https://github.com/google/nomulus/pull/1710.
2022-08-08 10:28:30 -04:00
gbrodman
1a4b1c45bd Add caches to ClaimsListDao and ClaimsList (#1731)
We cache the ClaimsList Java object for six hours (we don't expect it to
change frequently, and the cron job to update it only runs every twelve
hours). Subsequent calls to ClaimsListDao::get will return the cached
value.

Within the ClaimsList Java object itself, we cache any labels that we
have retrieved. While we already have a form of a cache here in the
"labelsToKeys" map, that only handles situations where we've loaded the
entire map from the database. We want to have a non-guaranteed cache in
order to make repeated calls to getClaimKey fast.
2022-08-05 17:29:59 -04:00
Ben McIlwain
53426a34d1 Make domain transfers use (and retain) the renewal price/behavior (#1701)
* Use the new renewal price logic in transfer flow

* Fix build

* Add renewal handling on all transfer flows

* Merge branch 'master' into transfer-retain-renewal-price

* Merge branch 'master' into transfer-retain-renewal-price

* Add more tests
2022-08-05 15:53:27 -04:00
gbrodman
6370baf9de Add base object classes for new user/role permissioning model (#1707)
* Add base object classes for new user/role permissioning model

- Adds the permissions themselves
- Adds the six roles that a user may have -- three global, three
  per-registrar
- Adds the mapping from role -> set of permissions
- Adds a UserRoles object to encapsulate the answer to the question of
  "does this user have this permission?"
- Adds a User class as a base to show how we will use the new UserRoles
  object
2022-08-05 14:18:16 -04:00
Ben McIlwain
a12a716806 Rename HostResource -> Host (#1733)
* Rename HostResource -> Host
2022-08-05 10:28:45 -04:00
Pavlo Tkach
4399067ddd Extend registrar allowed IPs auth exception text with IP address (#1726) 2022-08-03 15:24:00 -04:00
Lai Jiang
a04420c873 For some reason after the upgrade to Gradle, the core.jar file is no (#1730)
longer included in the generated WAR, even though the deploy_jar
configuration is specified as a dependency.

I could not figure out a way to tweak the configuration dependency to
have core.jar pulled into the .war, so I decided to just explicitly pick
it from its known location.

TESTED=deployed to alpha and verified that the instances can start.
2022-08-03 10:26:44 -04:00
Ben McIlwain
3e420071c1 Rename DomainContent -> DomainBase (#1729)
* Rename DomainContent -> DomainBase

This is a follow-up to PR #1725 which renamed DomainBase to Domain. Now, the
class naming hierarchy has the same structure as ContactBase/HostBase.
2022-08-02 17:21:17 -04:00
Pavlo Tkach
5c269023f8 Add docker prerequisite to install guide (#1728) 2022-08-02 16:20:59 -04:00
Ben McIlwain
68c80d3205 Rename DomainBase -> Domain (#1725)
* Rename DomainBase -> Domain

This was a long time coming, but we couldn't do it until we left Datastore, as
the Java class name has to match the Datastore entity name.

Subsequent PRs will rename ContactResource to Contact and HostResource to Host,
so that everything matches the SQL table names (and is shorter!).

* Merge branch 'master' into rename-domainbase
2022-08-02 16:03:30 -04:00
Lai Jiang
344b7c29ae Make Kythe run work with Gradle 7 (#1727)
The fix is based on b/240627423. I tested locally and was able to build
with the -PenableCrossReferencing=true flag successfully.

TESTED=run the kythe GCB pipeline locally.
2022-08-02 13:19:47 -04:00
Lai Jiang
d5be4dcc3b Remove ofy support from BillingEvent (#1710)
This PR turns out to be more massive than I would have liked but it
  deals with all billing event related stuff, which are more or link all
  intertwined:

  * Remove all billing events as Ofy entities.
  * Add a temporary annotation to allow BillingEvent's ID to be
    auto-allocated by ofy while not lacking the Ofy @Id annotation.
  * Remove Modification, which is only used in ofy.
  * Remove BillingVKey, as we do not need to store the ofy key parent
     information anymore. The VKey for a billing event now just contain
     its primary key, and can be converted by VKeyConverter.
  * Remove BigQuery related code in the billing pipeline.

  Note that after BillingVKey is removed, several columns in
  BillingCancellation are no longer needed. The change to database schema
  will be handled in https://github.com/google/nomulus/pull/1721 after
  this PR is deployed to production.

<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1710)
<!-- Reviewable:end -->
2022-08-02 11:36:28 -04:00
Ben McIlwain
7824185070 Revert "Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1714)" (#1724)
* Revert "Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1714)"

This partially reverts commit d8e77e2ab2 (it keeps
intact unrelated version upgrades).

We need to temporarily revert this because Spinnaker isn't quite yet playing
nice with the new <app-engine-apis> configuration option in appengine-web.xml
(it seems like this was added recently and Spinnaker is still stuck on App
Engine SDK version 1.9.82 which predates it). Hopefully we can get that
dependency updated in Spinnaker soon and then we can re-upgrade to Java 17.
2022-07-29 16:08:36 -04:00
sarahcaseybot
3cde188864 Flyway files for adding current_package_token column to Domain table (#1719)
* Flyway files for adding allocationToken column to Domain table

* Rename column to current_package_token

* Update er diagram

* Add foreign key to DomainHistory
2022-07-29 12:35:43 -04:00
Ben McIlwain
8138d73399 Delete the BackfillRegistrarBillingAccounts scrap command (#1722) 2022-07-29 11:40:11 -04:00
Ben McIlwain
8862c8510d Remove the withoutBackup methods from TransactionManager (#1723)
* Remove the withoutBackup methods from TransactionManager

This doesn't do anything in Cloud SQL (it was for Ofy only).
2022-07-29 11:39:56 -04:00
Ben McIlwain
c1cfb8bd71 Fix environment specification error message in build.gradle :deploy (#1717)
* Fix environment specification error message in build.gradle :deploy
2022-07-28 15:24:31 -04:00
gbrodman
711ddaf3d3 Don't include deleted domains in the ICANN reporting total_domains field (#1713)
This shouldn't matter for billing or anything like that because the
actual actions performed that month are still correct, but before this
PR we're including all domains ever created in the total_domains number,
including deleted domains

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1713)
<!-- Reviewable:end -->
2022-07-27 16:24:43 -04:00
Ben McIlwain
a5767e4de3 Upgrade App Engine Standard to Java 17 w/ bundled APIs (#1714)
* Upgrade App Engine Standard to Java 17 w/ bundled APIs

Note that this doesn't yet upgrade our actual Gradle scripts to use a more
recent of Java (that will happen separately); this solely affects the GAE
instances.

I followed the instructions here:
https://cloud.google.com/appengine/docs/standard/java-gen2/services/access

And note that I removed threadsafe true from appengine's XML config because
that doesn't do anything anymore and was just throwing errors (the new
instances handle multiple requests in parallel by default, no configuration
necessary).
2022-07-27 15:00:10 -04:00
Ben McIlwain
4b89cdf6c5 Add gnupg2 dependency to install guide (#1716) 2022-07-26 18:31:45 -04:00
Ben McIlwain
398fc26830 Fix else if on separate line causing compilation warning (#1715) 2022-07-26 17:03:47 -04:00
Michael Muller
169340298c Fixed distutils deprecation warning (#1711)
* Fixed distutils deprecation warning

Replace distutils.find_executable() with shutil.which().
2022-07-26 15:51:52 -04:00
gbrodman
5657089ffc Ignore TLD state on domain create when using corresponding token (#1709) 2022-07-26 13:54:12 -04:00
Michael Muller
06ca9266b4 Upgrade to Gradle 7.0 (#1712)
* Convert to gradle 7.

* More fixes, regenerated lockfiles.

* Update lockfiles for dependency update.

* Fix show_upgrade_diff for new lockfile format

* Add property for allowInsecureProtocol

Allow us to override the restriction against use of plain HTTP for
communication to dependency repositories.  We need this to be able to use a
local proxy for dependency gathering.

* Checking in missing gradle.lockfile
2022-07-26 11:41:27 -04:00
sarahcaseybot
68f975451e Implement EPP Allocation Token Extension for domain:renew (#1693)
* Implement EPP Allocation Token Extension for domain:renew

* Add more tests

* Fix AllocationTokenFlowUtilsTest

* Combine loadTokenAndValidateDomain with verifyAllocationTokenIfPresent

* Change to Optional.empty

* Remove unused variable
2022-07-26 10:28:32 -04:00
Lai Jiang
badfb29ce1 Remove ofy support from auto timestamp classes (#1708)
Also remove the use of ZonedDateTime, as joda DateTime can already be persisted to SQL with an existing converted.
2022-07-26 09:51:35 -04:00
sarahcaseybot
e0a0030837 Split failing dns update batches and kill after 10 retries (#1664)
* Split failing dns update batches and kill after 10 retries

* format fixes

* Add another test

* Switch to CloudTasks

* Change back to app engine header

* Change to immutableList and other changes

* Change to optional header

* Add bug ID to todo

* Switch to constructor injection

* Remove old queue

* Set response status

* Change to Optional<Integer>

* Rename action status

* Switched to use CLoudTaskHelper

* Remove spy in test
2022-07-25 10:45:10 -04:00
gbrodman
6af385bcf1 Add a registration_behavior column to AllocationToken (#1695)
This is, as of now, unused but we can use it for b/237683906 and
b/237800445 in the future to allow for special behavior dictated by
allocation tokens rather than having to reserve specific domains.

Note that we enforce a tied domain for ANCHOR_TENANT tokens (because
they should be matched to a domain) but not for BYPASS_TLD_STATE tokens.
2022-07-20 12:50:25 -04:00