Commit graph

4642 commits

Author SHA1 Message Date
Lai Jiang
597f63a603
Fix URL parameter to the DNS refresh fanout job (#1997)
<!-- Reviewable:start -->
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1997)
<!-- Reviewable:end -->
2023-04-19 15:32:41 -04:00
Lai Jiang
5ec73f3809
Refactor contact history PII wipeout logic into a Beam pipeline (#1994)
Because we need to check if a contact history is the most recent for its
underlying contact resource, the query-wipe out-repeat loop no longer works
ideally due to the added overhead with the query.

Instead, we refactor the logic into a Beam pipeline where the query only
needs to be performed once and history entries eligible for wipe out are
handled individually in their own transforms. Because history entries
are otherwise immutable, we can run the pipeline in relatively relaxed
repeatable read isolation level. We also do not worry about batching for
performance, as we do not anticipate this operation to put a lot of
strains on the particular table.
2023-04-19 13:04:45 -04:00
Ben McIlwain
b474e50e87
Update IDN tables with latest approved by ICANN (#1995)
This also adds README files to explain the two different IDN table locations
(which have different purposes). See http://b/278565478 for more information.
2023-04-18 17:23:12 -04:00
sarahcaseybot
6f3d062c32
Change Registry class name to Tld (#1991)
* Change Registry class name to Tld

* Fix merge conflict

* Some capitalization fixes
2023-04-18 12:26:14 -04:00
gbrodman
371d83b4cc
Add a command to update Recurrence objects' behavior (#1987)
We want to basically be able to change the renewal behavior, either
setting the behavior type (e.g. NONPREMIUM) or the specified renewal
price.
2023-04-17 11:36:12 -04:00
Lai Jiang
e1f29a8103
Add routing for ReadDnsRefreshRequestsAction (#1990)
It looks like we forgot this crucial part to actually add the necessary
routing the new action...

Also fixes a linter warning.
2023-04-12 15:17:21 -04:00
Pavlo Tkach
055a52f67e
Trim cloud scheduler config url value before submitting (#1988) 2023-04-10 19:05:32 -04:00
sarahcaseybot
d17678959c
Add tool commands to modify TTLs on a TLD (#1985)
* Add tool commands to modify TTLs on a TLD

* Small changes

* Add an example to the parameter description
2023-04-10 14:43:56 -04:00
Lai Jiang
79ba1b94c4
Add SQL-based DNS refresh processing mechanism (#1971) 2023-04-07 17:31:28 -04:00
gbrodman
33a771b13e
Add Java code for storing and using IDN tables per-TLD (#1977)
This includes changes to make sure that we use the proper per-TLD IDN
tables as well as setting/updating/removing them via the Create/Update
TLD commands.
2023-04-06 17:33:23 -04:00
gbrodman
bd65c6eee6
Allow a credit of 0 when deleting a domain during a grace period (#1984)
There can be situations (anchor tenants, test tokens, other ways of
getting a domain to cost $0) where we may want to delete a domain during
the add grace period but the credit applied is 0. We should not fail on
those cases.

See b/277115241 for an example.
2023-04-06 15:58:53 -04:00
Ben McIlwain
20c673840e
Add a new Unconfusable Latin table (#1981)
This new table has just been approved by ICANN. It is the same as our existing
Extended Latin table, except with the removal of some lesser-used characters
with diacritic marks that are confusable variants.

The filenames for the IDN tables are made explicit to improve code readability.

And this reverses the removal of G with stroke from the existing Extended Latin
table (see PR #1938), so that that table continues to accurately reflect the
state of our previously launched TLDs.

This is the full list of removed characters:

U+00E1                         # LATIN SMALL LETTER A WITH ACUTE
U+0101                         # LATIN SMALL LETTER A WITH MACRON
U+01CE                         # LATIN SMALL LETTER A WITH CARON
U+010B                         # LATIN SMALL LETTER C WITH DOT ABOVE
U+01E7                         # LATIN SMALL LETTER G WITH CARON
U+0123                         # LATIN SMALL LETTER G WITH CEDILLA
U+01E5                         # LATIN SMALL LETTER G WITH STROKE
U+0131                         # LATIN SMALL LETTER DOTLESS I
U+00ED                         # LATIN SMALL LETTER I WITH ACUTE
U+00EF                         # LATIN SMALL LETTER I WITH DIAERESIS
U+01D0                         # LATIN SMALL LETTER I WITH CARON
U+0144                         # LATIN SMALL LETTER N WITH ACUTE
U+014B                         # LATIN SMALL LETTER ENG
U+00F3                         # LATIN SMALL LETTER O WITH ACUTE
U+014D                         # LATIN SMALL LETTER O WITH MACRON
U+01D2                         # LATIN SMALL LETTER O WITH CARON
U+0157                         # LATIN SMALL LETTER R WITH CEDILLA
U+0163                         # LATIN SMALL LETTER T WITH CEDILLA
U+00FA                         # LATIN SMALL LETTER U WITH ACUTE
U+00FC                         # LATIN SMALL LETTER U WITH DIAERESIS
U+01D4                         # LATIN SMALL LETTER U WITH CARON
U+1E83                         # LATIN SMALL LETTER W WITH ACUTE
U+1E81                         # LATIN SMALL LETTER W WITH GRAVE
U+1E85                         # LATIN SMALL LETTER W WITH DIAERESIS
U+1EF3                         # LATIN SMALL LETTER Y WITH GRAVE
U+017C                         # LATIN SMALL LETTER Z WITH DOT ABOVE
2023-04-06 15:49:36 -04:00
Lai Jiang
11c60b8c8f
Temporarily disable contact history wipeout (#1982)
Makes the next run at the first Monday of December, which should give us
plenty of time to fix the issue with it wiping out PII in the most recent
contact history.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1982)
<!-- Reviewable:end -->
2023-04-06 13:41:51 -04:00
Lai Jiang
e330fd1c66
Remove cron.xml from sandbox (#1979)
It is somehow missed in #1965.
2023-04-06 11:30:07 -04:00
Pavlo Tkach
57c17042b6
Transaction manager to not retry inner transactions (#1974) 2023-04-05 16:46:36 -04:00
sarahcaseybot
8623fce119
Check for default tokens in the renew flow (#1978)
* Check for default tokens in the renew flow

* Remove extra check

* Add allowed action
2023-04-05 12:25:09 -04:00
Lai Jiang
7243575433
Remove unused GAE dependencies from NordnUploadAction (#1980) 2023-04-04 16:53:35 -04:00
sarahcaseybot
8eab43d371
Check allowedEppActions when validating tokens (#1972)
* Check allowedEppActions when validating tokens

* Reflect failed tokens in the fee check

* Add tests for domainCheckFlow

* Add hyphens to fee class name

* Add clarifying comment to catch block

* Add specific exception types
2023-04-04 14:29:50 -04:00
sarahcaseybot
34d329c158
Add tool changes to modify allowedEppActions on allocation tokens (#1970)
* Add tool changes to modify allowedEppActions on allocation tokens

* Change enum value error message

* Remove unnecessary variable

* Prevent UNKNOWN command

* Check command name instead of string
2023-03-31 14:37:19 -04:00
Pavlo Tkach
425ecdcd87
Add disable_runner_v2 to pipeline options (#1976) 2023-03-30 17:10:37 -04:00
gbrodman
77ee124374
Add SQL change for per-TLD IDN tables (#1975) 2023-03-28 17:03:22 -04:00
Lai Jiang
b9742adc0b
Delete cron.xml (#1965)
We've successfully migrated to using Cloud Scheduler.
2023-03-23 14:29:06 -04:00
sarahcaseybot
d4cd25c4ae
Add pricing logic for allocation tokens in domain renew (#1961)
* Add pricing logic for allocation tokens in domain renew

* Add clarifying comment

* Several fixes

* Add test for renewalPriceBehavior not changing
2023-03-23 14:00:36 -04:00
sarahcaseybot
8b7e938ed6
Add TTL configs to Registry object (#1968)
* Add TTL configs to Registry object

* Change A and AAAA records TTL field name
2023-03-22 13:56:11 -04:00
Pavlo Tkach
c216c874b4
Remove app engine deps from Lock flyway change (#1911) 2023-03-20 12:25:12 -04:00
Pavlo Tkach
0ab9471c8d
Make cloud scheduler deployment part of gradle deploy (alpha, qa and crash only) (#1969) 2023-03-20 11:10:00 -04:00
sarahcaseybot
d482754f66
Implement default tokens for the fee extension in domain check flow (#1950)
* Implement default tokens for the fee extension in domain check

* Add test for expired token

* Add test for alloc token and default token

* Fix formatting

* Always check for default tokens

* Change transaction time to passed in DateTime
2023-03-17 15:41:17 -04:00
sarahcaseybot
fe086b43f5
Add TTL columns to the Tld table (#1964)
* Add TTL columns to Tld table

* Change A and AAAA records column name
2023-03-17 11:54:14 -04:00
Lai Jiang
95f1bca3fb
Remove Nordn pull queue code (#1966)
The SQL-based flow is verified to work on production.
2023-03-16 17:37:48 -04:00
sarahcaseybot
178a2323d9
Add allowedEppActions to AllocationToken Java classes (#1958)
* Add allowedEppActions field to AllocationToken Java class and converter

* Add getter and setter
2023-03-16 15:45:34 -04:00
Lai Jiang
a44aa1378f
Create a DnsRefreshRequest entity backed by the corresponding table (#1941)
Also adds a DnsUtils class to deal with adding, polling, and removing
DNS refresh requests (only adding is implemented for now). The class
also takes care of choosing which mechanism to use (pull queue vs. SQL)
based on the current time and the database migration schedule map.
2023-03-16 13:02:20 -04:00
Pavlo Tkach
d0f625f70e
angular version update 15.1.0 -> 15.2.2 (#1967) 2023-03-16 11:56:38 -04:00
gbrodman
fb59874234
Allow for multiple service accounts in authentication (#1963)
When submitting tasks to Cloud Tasks, we will use the built-in OIDC
authentication which runs under the default service account (not the
cloud scheduler service account). We want either to work for app-level
auth.
2023-03-15 10:20:58 -04:00
gbrodman
b6083e227f
Move CloudTasksUtils to core/ project (#1956)
This does nothing for now, but in the future this will allow us to refer
to the RegistryConfig and/or Service objects from the core project. This
will be necessary when changing CloudTasksUtils to not use the AppEngine
built-in connection (it will need to use a standard HTTP request
instead).
2023-03-14 15:15:05 -04:00
Lai Jiang
5805b6859e
Rename process_time column in DnsRefreshRequest (#1962)
Make it explicit that this is the last process time, not a scheduled
future process time.
2023-03-14 14:03:12 -04:00
Pavlo Tkach
3108e8a871
Use builder image as a base for schema-deployer and schema-verifier (#1955) 2023-03-13 15:37:02 -04:00
Pavlo Tkach
ec142caf9c
Expand ID Token Auth verifier to catch all exceptions (#1960) 2023-03-13 12:12:47 -04:00
Pavlo Tkach
e60ad58098
Restore resaveAllEppResourcesPipeline as a cloud task (#1953) 2023-03-13 10:44:25 -04:00
sarahcaseybot
83e9e7fb5c
Add allowedEppActions field to AllocationToken (#1957) 2023-03-10 14:14:47 -05:00
Pavlo Tkach
438c523fcb
Remove app engine deps from Lock (#1910) 2023-03-09 10:47:48 -05:00
Lai Jiang
025a2faff2
Drop the indexs and columns for dns_refresh_request_time (#1949) 2023-03-09 10:29:31 -05:00
gbrodman
fd822dd333
Add create/delete/update commands for User objects (#1936)
This also includes the change of allowing the Java User object to have a
null GAIA ID (when creating user objects, we don't know what the GAIA ID
is).
2023-03-07 17:18:48 -05:00
Ben McIlwain
9b93749d43
Double the number of frontend instances from 12 to 24 (#1954)
It seems like we're hitting App Engine capacity issues resulting in actual pages
now (for whatever reason, but likely one customer), and we obviously don't want
that.
2023-03-06 16:04:28 -05:00
Pavlo Tkach
71a8579ece
Move App Engine cron jobs to cloud scheduler (#1939) 2023-03-01 13:40:56 -05:00
Lai Jiang
cda51f13dc
Remove dnsRefreshRequestTime from EppResources (#1943)
We have decided to use a separate table (#1940) to track DNS refresh requests
due to performance reasons.

See: go/registry-pull-queue-redesign
2023-03-01 13:40:30 -05:00
Lai Jiang
1de5b5dcc1
Add a process time column to DnsRefreshRequest (#1948)
The value of the column would be set to START_OF_TIME for new entries.
Every time a row is read, the value is updated to the current time. This
allows concurrent reads to not repeatedly read the same entry that has the
earliest request time, because they would only look for rows that have a value
of process time that is before current time - some padding time.

This basically fulfills the same function that LEASE_PADDING gives us
when using a pull queue, whereas a task would be leased for a certain
time, during which time they would not be leased by anyone else.

See: https://cs.opensource.google/nomulus/nomulus/+/master:core/src/main/java/google/registry/dns/ReadDnsQueueAction.java;l=99?q=readdnsqueue&ss=nomulus%2Fnomulus
2023-02-28 16:52:02 -05:00
sarahcaseybot
32279e42e4
Allow incorrect fee extensions on domain creates with default tokens (#1927)
* Modify fee extension to accept larger costs on creates with default tokens

* Add tests

* Add some comments to tests
2023-02-28 14:24:03 -05:00
Lai Jiang
ba0f90bdaf
Add support for Nordn upload without using pull queues. (#1925)
This PR adds an alternative method to upload Lordn to Nordn server without
using App Engine pull queue. A new database migration stage is added to control
whether a new task is scheduled with the old or new method. The
NordnUploadAction is configured to process both kind of tasks. Once the tasks
scheduled for the old tasks are all processed, we can start using the
new method exclusively.

See: go/registry-pull-queue-redesign
2023-02-28 12:57:27 -05:00
Lai Jiang
85308eb975
Ignore invalid old CRL when performing update. (#1946)
There is no point comparing the old CRL to the new ones when the old one
is invalid. This could happen when the CA cert rotates, after which the
old CRL stop being valid as it fails signature verification against the
new cert.

This change will allow us to keep updating the CRL after a CA rotation without
having to manually delete the old CRL from the database.

See b/270983553.

<!-- Reviewable:start -->
- - -
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/google/nomulus/1946)
<!-- Reviewable:end -->
2023-02-28 10:00:18 -05:00
Lai Jiang
ed62f27a4a
Update kythe vnames mapping (#1944) 2023-02-27 17:09:57 -05:00