Commit graph

1404 commits

Author SHA1 Message Date
Ben McIlwain
ad7d6f4873 Update IDN tables with latest approved by ICANN (#1995)
This also adds README files to explain the two different IDN table locations
(which have different purposes). See http://b/278565478 for more information.
2023-04-18 17:23:12 -04:00
sarahcaseybot
4434d807ad Change Registry class name to Tld (#1991)
* Change Registry class name to Tld

* Fix merge conflict

* Some capitalization fixes
2023-04-18 12:26:14 -04:00
gbrodman
f985cfd749 Add a command to update Recurrence objects' behavior (#1987)
We want to basically be able to change the renewal behavior, either
setting the behavior type (e.g. NONPREMIUM) or the specified renewal
price.
2023-04-17 11:36:12 -04:00
Lai Jiang
dcc7e65813 Add routing for ReadDnsRefreshRequestsAction (#1990)
It looks like we forgot this crucial part to actually add the necessary
routing the new action...

Also fixes a linter warning.
2023-04-12 15:17:21 -04:00
sarahcaseybot
e18dba472b Add tool commands to modify TTLs on a TLD (#1985)
* Add tool commands to modify TTLs on a TLD

* Small changes

* Add an example to the parameter description
2023-04-10 14:43:56 -04:00
Lai Jiang
d02b617e0f Add SQL-based DNS refresh processing mechanism (#1971) 2023-04-07 17:31:28 -04:00
gbrodman
a72f408366 Add Java code for storing and using IDN tables per-TLD (#1977)
This includes changes to make sure that we use the proper per-TLD IDN
tables as well as setting/updating/removing them via the Create/Update
TLD commands.
2023-04-06 17:33:23 -04:00
gbrodman
547fdaf87d Allow a credit of 0 when deleting a domain during a grace period (#1984)
There can be situations (anchor tenants, test tokens, other ways of
getting a domain to cost $0) where we may want to delete a domain during
the add grace period but the credit applied is 0. We should not fail on
those cases.

See b/277115241 for an example.
2023-04-06 15:58:53 -04:00
Ben McIlwain
59681640fa Add a new Unconfusable Latin table (#1981)
This new table has just been approved by ICANN. It is the same as our existing
Extended Latin table, except with the removal of some lesser-used characters
with diacritic marks that are confusable variants.

The filenames for the IDN tables are made explicit to improve code readability.

And this reverses the removal of G with stroke from the existing Extended Latin
table (see PR #1938), so that that table continues to accurately reflect the
state of our previously launched TLDs.

This is the full list of removed characters:

U+00E1                         # LATIN SMALL LETTER A WITH ACUTE
U+0101                         # LATIN SMALL LETTER A WITH MACRON
U+01CE                         # LATIN SMALL LETTER A WITH CARON
U+010B                         # LATIN SMALL LETTER C WITH DOT ABOVE
U+01E7                         # LATIN SMALL LETTER G WITH CARON
U+0123                         # LATIN SMALL LETTER G WITH CEDILLA
U+01E5                         # LATIN SMALL LETTER G WITH STROKE
U+0131                         # LATIN SMALL LETTER DOTLESS I
U+00ED                         # LATIN SMALL LETTER I WITH ACUTE
U+00EF                         # LATIN SMALL LETTER I WITH DIAERESIS
U+01D0                         # LATIN SMALL LETTER I WITH CARON
U+0144                         # LATIN SMALL LETTER N WITH ACUTE
U+014B                         # LATIN SMALL LETTER ENG
U+00F3                         # LATIN SMALL LETTER O WITH ACUTE
U+014D                         # LATIN SMALL LETTER O WITH MACRON
U+01D2                         # LATIN SMALL LETTER O WITH CARON
U+0157                         # LATIN SMALL LETTER R WITH CEDILLA
U+0163                         # LATIN SMALL LETTER T WITH CEDILLA
U+00FA                         # LATIN SMALL LETTER U WITH ACUTE
U+00FC                         # LATIN SMALL LETTER U WITH DIAERESIS
U+01D4                         # LATIN SMALL LETTER U WITH CARON
U+1E83                         # LATIN SMALL LETTER W WITH ACUTE
U+1E81                         # LATIN SMALL LETTER W WITH GRAVE
U+1E85                         # LATIN SMALL LETTER W WITH DIAERESIS
U+1EF3                         # LATIN SMALL LETTER Y WITH GRAVE
U+017C                         # LATIN SMALL LETTER Z WITH DOT ABOVE
2023-04-06 15:49:36 -04:00
Lai Jiang
9f65624f3e Temporarily disable contact history wipeout (#1982)
Makes the next run at the first Monday of December, which should give us
plenty of time to fix the issue with it wiping out PII in the most recent
contact history.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1982)
<!-- Reviewable:end -->
2023-04-06 13:41:51 -04:00
Lai Jiang
fea42831ae Remove cron.xml from sandbox (#1979)
It is somehow missed in #1965.
2023-04-06 11:30:07 -04:00
Pavlo Tkach
8d7088552f Transaction manager to not retry inner transactions (#1974) 2023-04-05 16:46:36 -04:00
sarahcaseybot
33b43c05e7 Check for default tokens in the renew flow (#1978)
* Check for default tokens in the renew flow

* Remove extra check

* Add allowed action
2023-04-05 12:25:09 -04:00
Lai Jiang
f8a820ba38 Remove unused GAE dependencies from NordnUploadAction (#1980) 2023-04-04 16:53:35 -04:00
sarahcaseybot
33b4ef36be Check allowedEppActions when validating tokens (#1972)
* Check allowedEppActions when validating tokens

* Reflect failed tokens in the fee check

* Add tests for domainCheckFlow

* Add hyphens to fee class name

* Add clarifying comment to catch block

* Add specific exception types
2023-04-04 14:29:50 -04:00
sarahcaseybot
fba137f1bb Add tool changes to modify allowedEppActions on allocation tokens (#1970)
* Add tool changes to modify allowedEppActions on allocation tokens

* Change enum value error message

* Remove unnecessary variable

* Prevent UNKNOWN command

* Check command name instead of string
2023-03-31 14:37:19 -04:00
Lai Jiang
28c671922c Delete cron.xml (#1965)
We've successfully migrated to using Cloud Scheduler.
2023-03-23 14:29:06 -04:00
sarahcaseybot
10b9951638 Add pricing logic for allocation tokens in domain renew (#1961)
* Add pricing logic for allocation tokens in domain renew

* Add clarifying comment

* Several fixes

* Add test for renewalPriceBehavior not changing
2023-03-23 14:00:36 -04:00
sarahcaseybot
273a633847 Add TTL configs to Registry object (#1968)
* Add TTL configs to Registry object

* Change A and AAAA records TTL field name
2023-03-22 13:56:11 -04:00
Pavlo Tkach
86ff8aae1c Make cloud scheduler deployment part of gradle deploy (alpha, qa and crash only) (#1969) 2023-03-20 11:10:00 -04:00
sarahcaseybot
cf5e887f55 Implement default tokens for the fee extension in domain check flow (#1950)
* Implement default tokens for the fee extension in domain check

* Add test for expired token

* Add test for alloc token and default token

* Fix formatting

* Always check for default tokens

* Change transaction time to passed in DateTime
2023-03-17 15:41:17 -04:00
Lai Jiang
0c7dbe8a34 Remove Nordn pull queue code (#1966)
The SQL-based flow is verified to work on production.
2023-03-16 17:37:48 -04:00
sarahcaseybot
815be8000a Add allowedEppActions to AllocationToken Java classes (#1958)
* Add allowedEppActions field to AllocationToken Java class and converter

* Add getter and setter
2023-03-16 15:45:34 -04:00
Lai Jiang
3d809e762b Create a DnsRefreshRequest entity backed by the corresponding table (#1941)
Also adds a DnsUtils class to deal with adding, polling, and removing
DNS refresh requests (only adding is implemented for now). The class
also takes care of choosing which mechanism to use (pull queue vs. SQL)
based on the current time and the database migration schedule map.
2023-03-16 13:02:20 -04:00
gbrodman
cd0703c815 Allow for multiple service accounts in authentication (#1963)
When submitting tasks to Cloud Tasks, we will use the built-in OIDC
authentication which runs under the default service account (not the
cloud scheduler service account). We want either to work for app-level
auth.
2023-03-15 10:20:58 -04:00
gbrodman
8ab2252272 Move CloudTasksUtils to core/ project (#1956)
This does nothing for now, but in the future this will allow us to refer
to the RegistryConfig and/or Service objects from the core project. This
will be necessary when changing CloudTasksUtils to not use the AppEngine
built-in connection (it will need to use a standard HTTP request
instead).
2023-03-14 15:15:05 -04:00
Pavlo Tkach
d996ef974c Expand ID Token Auth verifier to catch all exceptions (#1960) 2023-03-13 12:12:47 -04:00
Pavlo Tkach
de6d73930c Restore resaveAllEppResourcesPipeline as a cloud task (#1953) 2023-03-13 10:44:25 -04:00
Pavlo Tkach
7146260a08 Remove app engine deps from Lock (#1910) 2023-03-09 10:47:48 -05:00
gbrodman
253aa7d9ad Add create/delete/update commands for User objects (#1936)
This also includes the change of allowing the Java User object to have a
null GAIA ID (when creating user objects, we don't know what the GAIA ID
is).
2023-03-07 17:18:48 -05:00
Ben McIlwain
93de7d689f Double the number of frontend instances from 12 to 24 (#1954)
It seems like we're hitting App Engine capacity issues resulting in actual pages
now (for whatever reason, but likely one customer), and we obviously don't want
that.
2023-03-06 16:04:28 -05:00
Pavlo Tkach
f615e88ff6 Move App Engine cron jobs to cloud scheduler (#1939) 2023-03-01 13:40:56 -05:00
Lai Jiang
7f3a43f70d Remove dnsRefreshRequestTime from EppResources (#1943)
We have decided to use a separate table (#1940) to track DNS refresh requests
due to performance reasons.

See: go/registry-pull-queue-redesign
2023-03-01 13:40:30 -05:00
sarahcaseybot
9a212314ca Allow incorrect fee extensions on domain creates with default tokens (#1927)
* Modify fee extension to accept larger costs on creates with default tokens

* Add tests

* Add some comments to tests
2023-02-28 14:24:03 -05:00
Lai Jiang
c07728dab8 Add support for Nordn upload without using pull queues. (#1925)
This PR adds an alternative method to upload Lordn to Nordn server without
using App Engine pull queue. A new database migration stage is added to control
whether a new task is scheduled with the old or new method. The
NordnUploadAction is configured to process both kind of tasks. Once the tasks
scheduled for the old tasks are all processed, we can start using the
new method exclusively.

See: go/registry-pull-queue-redesign
2023-02-28 12:57:27 -05:00
Lai Jiang
33f9bc30b7 Ignore invalid old CRL when performing update. (#1946)
There is no point comparing the old CRL to the new ones when the old one
is invalid. This could happen when the CA cert rotates, after which the
old CRL stop being valid as it fails signature verification against the
new cert.

This change will allow us to keep updating the CRL after a CA rotation without
having to manually delete the old CRL from the database.

See b/270983553.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1946)
<!-- Reviewable:end -->
2023-02-28 10:00:18 -05:00
Ben McIlwain
b9045cce86 Remove "letter G with stroke" from Extended Latin IDN table (#1938)
ICANN doesn't like this character because it's confusable with a normal G (the
stroke tends to get lost in the visual clutter of the descender), and .com's
Extended Latin table doesn't use it either. Best to get rid of it.
2023-02-23 16:27:15 -05:00
Lai Jiang
bac9a25800 Change default beam job region (#1937)
For reasons that I cannot explain, the same expand recurring billing
event pipeline would fail in us-east1 but succeed in us-central1.

See:

https://pantheon.corp.google.com/dataflow/jobs/us-central1/2023-02-09_14_52_24-162498476138221714;graphView=0?project=domain-registry

https://pantheon.corp.google.com/dataflow/jobs/us-east1/2023-02-09_14_26_07-4564782062878841960;graphView=1?project=domain-registry

Also improved how the accuracy of the metrics:

It is observed that both counters are consistently higher for the same
start and end times when running in dry run mode. There is no way to
test for consistency when not running in dry run, for obviously reasons.

I can make the recurrings in scope counter consistent by not updating it
in a side-effect-causing transaction, but there is no way around the
other counter. It can only be trusted when running in dry run mode,
unfortunately.
2023-02-13 15:57:32 -05:00
gbrodman
b709a0ce48 Add a frontend endpoint for retrieving a domain in JSON form (#1916)
We might (likely will) modify some of the fiddly bits around this (maybe
the GSON serialization, where we do the actual authorization, etc) but
this should be a decent basic shell structure for endpoints that the new
registrar console can call to retrieve JSON results.
2023-02-09 15:09:42 -05:00
Lai Jiang
3532213d4c Update TMCH root certificate (#1918)
See b/260945047.

Also refactored the corresponding tests, which should future updates easier.

This change should be deployed at or around 2023-02-15T16:00:00Z.
2023-02-06 22:39:54 -05:00
Lai Jiang
706cf69b7f Fix expand recurring billing event pipeline (#1928) 2023-02-06 11:33:57 -05:00
sarahcaseybot
c6e77276b6 Modify DomainCreateFlow to check for an applicable defaultPromoToken (#1904)
* Modify DomainCreateFlow to check for an applicable defaultPromoToken

* Add handling for deleted tokens

* Change cache to allocation token cache

* Abstract away cache methods

* Use AllocationToken.getAll in create flow

* Filter out empty tokens
2023-02-01 14:53:51 -05:00
sarahcaseybot
f1cbab258f Prevent ending package tokens with active domains (#1919)
* Prevent ending package tokens with active domains

* Fix bad formatting in comments

* Fix lots of nits
2023-01-30 16:13:23 -05:00
Lai Jiang
946b683906 Remove @IdAllocation annotation from repoId (#1923)
This annotation only works for Long or long field.
2023-01-30 15:40:40 -05:00
gbrodman
bd66b3cef1 Use the Proxy-Authorization header when using nomulus + IAP (#1921) 2023-01-26 15:16:32 -05:00
Lai Jiang
ab146c4bf8 Add fields needed to implement pull queue alternative (#1915) 2023-01-25 15:26:00 -05:00
Pavlo Tkach
5a3aa9fa52 Add a condition update precaution to validateNewState (#1920) 2023-01-25 14:53:12 -05:00
Weimin Yu
c71a3312ab Upgrade to Gradle 7.3.2 (#1922)
This is an 'easy' upgrade that requires a minor change in
common/build.gradle and the removal of an unnecessary import in buildSrc.

Gradle 7.4 and above has breaking changes that break the latest nebula lint plugin. We may have to wait a while.
2023-01-25 12:47:35 -05:00
Lai Jiang
8088efb25c Update ExpandRecurringBillingEventsAction to use the beam pipeline (#1907)
Due to the way the beam pipeline is designed, it will expand an
recurring billing event when its event time is in scope for expansion,
instead of billing time. This means that the one time will be generated
45 days earlier. This would negate the need to check if the expansion is
finished when generating monthly invoices.

We will need to backfill the past 45 days of onetimes before the new
code is deployed. As an illustration, with the old code, a cursor time
of 2023-01-17 means that all auto-renewals whose billing time is before
2023-01-17 were created, which corresponds to an effective cursor time
of 2022-12-03 (45 days before 2023-01-17) for event time. This cursor
will need to be brought to 2023-01-17 to ensure that there is no gap in
generated event times when switching to use the new code.
2023-01-23 19:08:04 -05:00
Pavlo Tkach
5210ac0419 Remove not null constraint request_log_id column (#1917) 2023-01-23 09:37:20 -05:00