Async tasks will now re-enqueue themselves after completion if there are
additional pending future actions. This allows all parts of domain delete flows
to be successfully re-saved as the parts happen, without going past the maximum
allowed 30 day task ETA limit. The first task runs at 30 days out when the
redemption grace period ends, and that task then enqueues another task to run 5
more days in the future, when the deletion is final and the pending delete
status gets removed.
No data migration plan is necessary because future resaves defaults to empty,
and indeed will always be empty on transfers. So previously enqueued tasks will
still be valid.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202949677
This removes the following unnecessary imports:
//third_party/java_src/gtld/java/google/registry/bigquery
//third_party/java_src/gtld/java/google/registry/config
//third_party/java_src/gtld/java/google/registry/cron
//third_party/java_src/gtld/java/google/registry/dns
//third_party/java_src/gtld/java/google/registry/gcs
//third_party/java_src/gtld/java/google/registry/mapreduce
//third_party/java_src/gtld/java/google/registry/model
//third_party/java_src/gtld/java/google/registry/module/backend
//third_party/java_src/gtld/java/google/registry/module/frontend
//third_party/java_src/gtld/java/google/registry/module/pubapi
//third_party/java_src/gtld/java/google/registry/module/tools
//third_party/java_src/gtld/java/google/registry/request
//third_party/java_src/gtld/java/google/registry/security
//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers
//third_party/java_src/gtld/java/google/registry/util
//third_party/java_src/gtld/java/google/registry/xjc
//third_party/java_src/gtld/javatests/google/registry/model
//third_party/java_src/gtld/javatests/google/registry/testing
//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java_src/gtld/java/google/registry/bigquery,//third_party/java_src/gtld/java/google/registry/config,//third_party/java_src/gtld/java/google/registry/cron,//third_party/java_src/gtld/java/google/registry/dns,//third_party/java_src/gtld/java/google/registry/gcs,//third_party/java_src/gtld/java/google/registry/mapreduce,//third_party/java_src/gtld/java/google/registry/model,//third_party/java_src/gtld/java/google/registry/module/backend,//third_party/java_src/gtld/java/google/registry/module/frontend,//third_party/java_src/gtld/java/google/registry/module/pubapi,//third_party/java_src/gtld/java/google/registry/module/tools,//third_party/java_src/gtld/java/google/registry/request,//third_party/java_src/gtld/java/google/registry/security,//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers,//third_party/java_src/gtld/java/google/registry/util,//third_party/java_src/gtld/java/google/registry/xjc,//third_party/java_src/gtld/javatests/google/registry/model,//third_party/java_src/gtld/javatests/google/registry/testing,//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202652421
Also adds Guy to list of CONTRIBUTORS and removes Xooglers (who definitely won't be contributing more code).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202476218
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This removes the following unnecessary imports:
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='[],[],[],[],[],[],[],[],[],[],[],[],[],[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202325520
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
First step of RDE encoding refactoring.
Creates a single InputStream (OutputStream) to decode (encode) Ghostryde files.
This replaces the 3 InputStreams (OutputStreams) that were needed before.
Also removes a lot of classes, and removes the "injection" of the Ghostryde
class. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value. It's just a buffer size - it doesn't actually affect much. There
are much more "important" fields that weren't configured (such as the
compression algorithm and whether or not to do integrity checks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202319102
It only needs the error page HTML files; everything else isn't used by endpoints
served by the backend service and only serves to increase build times
(especially compiling all that JS).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202229514
We'll continue to use injected clocks for the rest of our tests that use signed marks files, so that they don't all fail after the current validity period. The new test TmchTestDataExpirationTest will let us know when the files are expired, so we can update them.
All updated test data files come from https://newgtlds.icann.org/en/about/trademark-clearinghouse/registries-registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202208196
RdeUpload works on alpha and sandbox by sending the data to a google internal
server. It is still running (and succeeding) after each RdeStaging, but the
lack of the cron job means it is currently lagging behind.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202181148
Two main changes:
- Replaced getClass().getSimpleName() in the logs with a constructor-given name. Right now what we have is a lot of identical classes with slightly different names so that the logs would be different. With this change - we can later get rid of a lot of these classes and replace them with simple wrappers.
- Removed the "expected" feature. Only Tar uses it - and it can override onClose to do that (that's what it's there for!)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202129563
There's no reason to have to create 2,500 simulated database entities for a single test (which makes it take a really long time). Better to just set the relevant limit to be lower for testing purposes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202120722
The signature has expired on Mon Jun 25 23:59:59 UTC 2018.
Instead of fixing all the test files with a new signed mark, we inject the
clock used to validate the signature.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202116721
In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.
This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
We've migrated everything over to using the standard SQL view now. The legacy
version is just causing confusion and costing us resources.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201966352
We haven't used this in years and I doubt it's still even functional. It's using
the legacy BigQuery tables and the pre-Registry 2.0 schema, for starters.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201960046
This is one last hanging piece of work left over from last year's Java 8
migration. There's no functionality changes in this CL, just refactoring.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201947600
It doesn't entirely make semantic sense, since the actual state of the
SystemClock isn't being preserved, but it makes injection into serializable
classes (e.g. mapreduces) much simpler, so it's worth doing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201755949
ICANN's Consistent Labeling & Display policy requires that these fields be
present for all domains. We're currently tracking down the last of the data on
our registrars so that we can display these, but if we don't have the data we
should still at least be displaying the field names to make it more clear that
we do support the field, we just don't have the data for this domain yet.
The two fields in question are:
Registrar Abuse Contact Email:
Registrar Abuse Contact Phone:
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201730290
SystemClock isn't Serializable (for obvious reasons), whereas LockHandlerImpl is
used as a field on some Serializable mapreduce classes. So mark it transient and
then re-generate it on first use following de-serialization when it happens to
be null.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201707209
New upload tasks are created every 4 hours, so if we're waiting on a 2 hour SFTP cooldown or some other long-running dependency like generating the RDE report, just delete this task and let it re-run at the next 4 hour period. No need to let these tasks continue gumming up the queue.
Note that this method of throwing NoContentException to abort the task without enqueuing it for retry is already being used by RdeReportAction for the same purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201372808
They're all in the same entity group anyway (the cross-TLD one), so they can be
loaded in a single call instead of individually.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201364854
This is used in the domain transfer and delete flows, both of which are
asynchronous flows that have implicit default actions that will be taken at some
point in the future. This CL adds scheduled re-saves to take place soon after
those default actions would become effective, so that they can be re-saved
quickly if so.
Unfortunately the redemption grace period on our TLDs is 35 days, which exceeds
the 30 day maximum task ETA in App Engine, so these won't actually fire. That's
fine though; the deletion is actually effective as of 5 days, and this is just
removing the grace period.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201345274
We're still limiting to a maximum of 5 concurrent uploads, but when we get backed up (i.e. because we broke RDE like we did recently), it makes sense to burn through the backlog faster once tasks are succeeding again. As I'm going through the backlog now, 5/m isn't fast enough; 10/m seems right.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201284990
We're currently facing a large backlog of RDE upload tasks, most of which won't
have anything to do when they execute (because the RDE deposit in question has
been successfully uploaded). And we're also facing the occasional >30 minute
timeout even though most uploads are succeeding in around a minute.
So this CL just lets more run simultaneously so that the backlog can be cleared
out faster.
Note that we still enforce locking on a per-TLD basis, so it won't be possible
for uploads to stomp over each other.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201257679
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
It came up during the review of [] that it doesn't make a lot of sense
for encrypt() and decrypt() to not throw the same kinds of Exceptions,
especially not for the same kind of problem, just because one happens to use a
Retrier in its internal implementation and the other doesn't.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201054057
Rather than just logging a generic TimeoutException, this will say what action
timed out and how long it had been executing for.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201049025
*** Reason for rollback ***
We suspect that this is breaking RDE more, so we're going to rollout a cherrypick of this reversion.
*** Original change description ***
Upload to GCS before uploading to FTP
Currently we encode and upload the deposite to GCS and the FTP server at the
same time. This makes debugging harder as there are many possible points of
failure, some of which are external and some internal.
In this CL we start by encoding + uploading the deposit to GCS, and once
that's done we copy the data from GCS to the FTP server. This will (hopefully)
allow us to distinguish between errors on the FTP server and errors with the
GCS connection.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201005260
These are now handled by the pubapi service and all publicly facing sites that
were using these APIs have already been migrated over.
For documentation on the newly added dispatch.xml file, see:
https://cloud.google.com/appengine/docs/standard/java/config/dispatchref
Note that the --auto_update_dispatch parameter needs to be passed to the
`appcfg update` command in order to apply this new XML file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200441580
Also explicitly state that contacts missing GAE-UserId can't access the
registrar console
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402953
An admin that isn't associated with any clientId, will default to the
registryAdminClientId.
However, if we set the registryAdminClientId as the session's
CLIENT_ID_ATTRIBUTE, the next time we access the server we have a client-id
attribute we aren't associated with - which returns a "403 Registrar Console
access revoked" error (the assumption is - we were associated with it before
but aren't anymore)
To fix this - we just add all admins as "hasAccessTo" registryAdminClientId, even if it's not in the contacts. This will let admins stay on the admin registrar, without affecting where they log-in initially if they are also contacts in different registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402856
It was hitting lease timeouts at just 20 minutes in larger environments.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200126822
This will allow us to check in actual SUNRISE billing policies per launch (15% discount), instead of relying on ad-hoc timestamps.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200077926
Currently the input XML to an EPP flow is logged twice, once
in FlowRunner and once in FlowReporter.
The log by FlowReporter was used by reporting but this is no
longer the case.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200057153
It is better to store it ASCII armored so that it can be easily diffed to see
if a file has changed
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200045488
This allows for the creation of records like epp-canary.registr.google.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199850436
This limit did not exist prior to [] which added the ability to limit
the size of the list. I didn't think that we needed to be able to query more
than 30 TLDs at any one time so I got rid of batching, but it turns out we do
need this ability for domain_watcher. So I'm re-adding batching, which is a
little bit more complicated now that we're also limiting and sorting by creation
time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199826414
Also prevents signed marks from being used in non-sunrise TldStates.
Currently, we send out a Lordn update only when there's a ClaimNotice, or if
we're in end-date sunrise.
But EPPs can contain a SignedMark instead of a ClaimsNotice for trademarked
domains - in which case we aren't sending out Lordn update. This also applies
to start-date sunrises.
We also change the SignedMark behavior for superusers. Currently, if a
mismatched signed mark is given as superuser, we accept it. That causes
problems when we want to send the Lordn update.
Instead - we no longer allow superusers to give a mismatched SignedMark (just
as we don't allow users to give a bad ClaimNotice). A super user can still
create a domain WITHOUT a signed mark - but if one is provided, it must match.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199783411
New metrics are necessary because the new API no longer wraps
an EPP flow, therefore does not get metrics for free.
Metrics include
- An EventMetric for processing time
- An IncrementableMetric for request count, with
availability (available/reserved/registered) and
pricing (standard/premium) fields
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199708592
