Also goes back up to 100 max instances.
Hopefully this'll work better this time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204783809
Lists used as accumulators were being updated individually for each domain
without starting over from a fresh list each time, so the number of changes
would grow for each additional domain and potentially be wrong if the previous
domains were set up differently.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204526006
This means that, when writing new tests that are failing, you get much more
useful logs that show the actual XML in a more comprehensible format that is
suitable for pasting back into the golden file in the test (if the change was
intended).
This requires outputting the standalone parameter in the XML transformer, and
some minor changes to some tests as a result that were relying on it being
stripped out.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204513690
This also introduces a production canary environment, similar to sandbox canary. The docker tags are changed to "live" and "sandbox" respectively, to reflect the fact that different images may be used for prod and sandbox.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204343530
This is a part of a large-scale change: [] . All .bzl files are being formatted with buildifier.
To format a file manually run `buildifier path/to/file.bzl`. Integration with `g4 fix` will be available later, but you can try using `g4 fix --format=bzl`.
Tested:
tap_presubmit
Some tests failed; test failures are believed to be unrelated to this CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204137752
The code stopped working after a recent soy-to-js compiler change.
Impacts include:
- The /registrar#domain page only shows part of the contents.
- RegistrarConsoleScreenshotTest times out.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204135349
This is a part of a large-scale change: [] . All .bzl files are being formatted with buildifier.
To format a file manually run `buildifier path/to/file.bzl`. Integration with `g4 fix` will be available later, but you can try using `g4 fix --format=bzl`.
Tested:
tap_presubmit
Some tests failed; test failures are believed to be unrelated to this CL
BEGIN_PUBLIC
Format .bzl files with buildifier
END_PUBLIC
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=203461813
The action isn't routable (i.e. it doesn't work) unless this is specified.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=203020503
We never launched this, don't planning on launching it now anyway, and it's rotted over the past two years anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202993577
Async tasks will now re-enqueue themselves after completion if there are
additional pending future actions. This allows all parts of domain delete flows
to be successfully re-saved as the parts happen, without going past the maximum
allowed 30 day task ETA limit. The first task runs at 30 days out when the
redemption grace period ends, and that task then enqueues another task to run 5
more days in the future, when the deletion is final and the pending delete
status gets removed.
No data migration plan is necessary because future resaves defaults to empty,
and indeed will always be empty on transfers. So previously enqueued tasks will
still be valid.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202949677
This removes the following unnecessary imports:
//third_party/java_src/gtld/java/google/registry/bigquery
//third_party/java_src/gtld/java/google/registry/config
//third_party/java_src/gtld/java/google/registry/cron
//third_party/java_src/gtld/java/google/registry/dns
//third_party/java_src/gtld/java/google/registry/gcs
//third_party/java_src/gtld/java/google/registry/mapreduce
//third_party/java_src/gtld/java/google/registry/model
//third_party/java_src/gtld/java/google/registry/module/backend
//third_party/java_src/gtld/java/google/registry/module/frontend
//third_party/java_src/gtld/java/google/registry/module/pubapi
//third_party/java_src/gtld/java/google/registry/module/tools
//third_party/java_src/gtld/java/google/registry/request
//third_party/java_src/gtld/java/google/registry/security
//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers
//third_party/java_src/gtld/java/google/registry/util
//third_party/java_src/gtld/java/google/registry/xjc
//third_party/java_src/gtld/javatests/google/registry/model
//third_party/java_src/gtld/javatests/google/registry/testing
//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java_src/gtld/java/google/registry/bigquery,//third_party/java_src/gtld/java/google/registry/config,//third_party/java_src/gtld/java/google/registry/cron,//third_party/java_src/gtld/java/google/registry/dns,//third_party/java_src/gtld/java/google/registry/gcs,//third_party/java_src/gtld/java/google/registry/mapreduce,//third_party/java_src/gtld/java/google/registry/model,//third_party/java_src/gtld/java/google/registry/module/backend,//third_party/java_src/gtld/java/google/registry/module/frontend,//third_party/java_src/gtld/java/google/registry/module/pubapi,//third_party/java_src/gtld/java/google/registry/module/tools,//third_party/java_src/gtld/java/google/registry/request,//third_party/java_src/gtld/java/google/registry/security,//third_party/java_src/gtld/java/google/registry/ui/soy/registrar:soy_java_wrappers,//third_party/java_src/gtld/java/google/registry/util,//third_party/java_src/gtld/java/google/registry/xjc,//third_party/java_src/gtld/javatests/google/registry/model,//third_party/java_src/gtld/javatests/google/registry/testing,//third_party/java_src/gtld/javatests/google/registry/testing/mapreduce'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202652421
Also adds Guy to list of CONTRIBUTORS and removes Xooglers (who definitely won't be contributing more code).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202476218
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This removes the following unnecessary imports:
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='[],[],[],[],[],[],[],[],[],[],[],[],[],[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202325520
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
First step of RDE encoding refactoring.
Creates a single InputStream (OutputStream) to decode (encode) Ghostryde files.
This replaces the 3 InputStreams (OutputStreams) that were needed before.
Also removes a lot of classes, and removes the "injection" of the Ghostryde
class. It's an encoding, there's no point in injecting it.
Finally, removed the buffer-size configuration and replaced with a static final
const value. It's just a buffer size - it doesn't actually affect much. There
are much more "important" fields that weren't configured (such as the
compression algorithm and whether or not to do integrity checks)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202319102
It only needs the error page HTML files; everything else isn't used by endpoints
served by the backend service and only serves to increase build times
(especially compiling all that JS).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202229514
We'll continue to use injected clocks for the rest of our tests that use signed marks files, so that they don't all fail after the current validity period. The new test TmchTestDataExpirationTest will let us know when the files are expired, so we can update them.
All updated test data files come from https://newgtlds.icann.org/en/about/trademark-clearinghouse/registries-registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202208196
RdeUpload works on alpha and sandbox by sending the data to a google internal
server. It is still running (and succeeding) after each RdeStaging, but the
lack of the cron job means it is currently lagging behind.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202181148
Two main changes:
- Replaced getClass().getSimpleName() in the logs with a constructor-given name. Right now what we have is a lot of identical classes with slightly different names so that the logs would be different. With this change - we can later get rid of a lot of these classes and replace them with simple wrappers.
- Removed the "expected" feature. Only Tar uses it - and it can override onClose to do that (that's what it's there for!)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202129563
There's no reason to have to create 2,500 simulated database entities for a single test (which makes it take a really long time). Better to just set the relevant limit to be lower for testing purposes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202120722
The signature has expired on Mon Jun 25 23:59:59 UTC 2018.
Instead of fixing all the test files with a new signed mark, we inject the
clock used to validate the signature.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202116721
In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.
This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
We've migrated everything over to using the standard SQL view now. The legacy
version is just causing confusion and costing us resources.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201966352
We haven't used this in years and I doubt it's still even functional. It's using
the legacy BigQuery tables and the pre-Registry 2.0 schema, for starters.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201960046
This is one last hanging piece of work left over from last year's Java 8
migration. There's no functionality changes in this CL, just refactoring.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201947600
It doesn't entirely make semantic sense, since the actual state of the
SystemClock isn't being preserved, but it makes injection into serializable
classes (e.g. mapreduces) much simpler, so it's worth doing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201755949
ICANN's Consistent Labeling & Display policy requires that these fields be
present for all domains. We're currently tracking down the last of the data on
our registrars so that we can display these, but if we don't have the data we
should still at least be displaying the field names to make it more clear that
we do support the field, we just don't have the data for this domain yet.
The two fields in question are:
Registrar Abuse Contact Email:
Registrar Abuse Contact Phone:
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201730290
SystemClock isn't Serializable (for obvious reasons), whereas LockHandlerImpl is
used as a field on some Serializable mapreduce classes. So mark it transient and
then re-generate it on first use following de-serialization when it happens to
be null.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201707209
New upload tasks are created every 4 hours, so if we're waiting on a 2 hour SFTP cooldown or some other long-running dependency like generating the RDE report, just delete this task and let it re-run at the next 4 hour period. No need to let these tasks continue gumming up the queue.
Note that this method of throwing NoContentException to abort the task without enqueuing it for retry is already being used by RdeReportAction for the same purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201372808
They're all in the same entity group anyway (the cross-TLD one), so they can be
loaded in a single call instead of individually.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201364854
This is used in the domain transfer and delete flows, both of which are
asynchronous flows that have implicit default actions that will be taken at some
point in the future. This CL adds scheduled re-saves to take place soon after
those default actions would become effective, so that they can be re-saved
quickly if so.
Unfortunately the redemption grace period on our TLDs is 35 days, which exceeds
the 30 day maximum task ETA in App Engine, so these won't actually fire. That's
fine though; the deletion is actually effective as of 5 days, and this is just
removing the grace period.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201345274
We're still limiting to a maximum of 5 concurrent uploads, but when we get backed up (i.e. because we broke RDE like we did recently), it makes sense to burn through the backlog faster once tasks are succeeding again. As I'm going through the backlog now, 5/m isn't fast enough; 10/m seems right.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201284990
We're currently facing a large backlog of RDE upload tasks, most of which won't
have anything to do when they execute (because the RDE deposit in question has
been successfully uploaded). And we're also facing the occasional >30 minute
timeout even though most uploads are succeeding in around a minute.
So this CL just lets more run simultaneously so that the backlog can be cleared
out faster.
Note that we still enforce locking on a per-TLD basis, so it won't be possible
for uploads to stomp over each other.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201257679
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
It came up during the review of [] that it doesn't make a lot of sense
for encrypt() and decrypt() to not throw the same kinds of Exceptions,
especially not for the same kind of problem, just because one happens to use a
Retrier in its internal implementation and the other doesn't.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201054057
Rather than just logging a generic TimeoutException, this will say what action
timed out and how long it had been executing for.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201049025
*** Reason for rollback ***
We suspect that this is breaking RDE more, so we're going to rollout a cherrypick of this reversion.
*** Original change description ***
Upload to GCS before uploading to FTP
Currently we encode and upload the deposite to GCS and the FTP server at the
same time. This makes debugging harder as there are many possible points of
failure, some of which are external and some internal.
In this CL we start by encoding + uploading the deposit to GCS, and once
that's done we copy the data from GCS to the FTP server. This will (hopefully)
allow us to distinguish between errors on the FTP server and errors with the
GCS connection.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201005260
These are now handled by the pubapi service and all publicly facing sites that
were using these APIs have already been migrated over.
For documentation on the newly added dispatch.xml file, see:
https://cloud.google.com/appengine/docs/standard/java/config/dispatchref
Note that the --auto_update_dispatch parameter needs to be passed to the
`appcfg update` command in order to apply this new XML file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200441580
