We got 500 Internal Server Error when calling GAE RemoteApi in Nomulus
tool. The root cause is we missed the dependency described in this
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=242166362
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.
It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.
This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
This is currently erroring out on entities that fail to load properly through Objectify (e.g. because their entity type is no longer registered). The proper thing to do is to catch the error, log it, and fall back to the raw Datastore operation, which will succeed.
The exact Exception this is designed to catch is:
com.google.apphosting.runtime.jetty9.JettyLogger warn: /_dr/admin/deleteEntity (JettyLogger.java:29)
java.lang.IllegalStateException: No registered subclass for discriminator 'DomainApplication'
at com.googlecode.objectify.v4.impl.PolymorphicEntityMetadata.getConcrete(PolymorphicEntityMetadata.java:133)
at com.googlecode.objectify.v4.impl.PolymorphicEntityMetadata.load(PolymorphicEntityMetadata.java:164)
at com.googlecode.objectify.v4.impl.LoadEngine.load(LoadEngine.java:220)
at com.googlecode.objectify.v4.impl.LoadEngine$1.nowUncached(LoadEngine.java:178)
at com.googlecode.objectify.v4.impl.LoadEngine$1.nowUncached(LoadEngine.java:164)
at com.googlecode.objectify.v4.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.v4.impl.Round$1.nowUncached(Round.java:73)
at com.googlecode.objectify.v4.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.v4.LoadResult.now(LoadResult.java:25)
at google.registry.tools.server.DeleteEntityAction.loadOfyEntity(DeleteEntityAction.java:103)
at google.registry.tools.server.DeleteEntityAction.run(DeleteEntityAction.java:74)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230737553
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.
Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
Our goal is to be able to address every Action by looking at the class itself, and to make it clearer at a glance what you need to access the Action's endpoint
Currently, we can know from the @Action annotation:
- the endpoint path
- the Method needed
- the authentication level needed
This CL adds the service where the Action is hosted, which also translates to the URL.
NOTE - currently we don't have any Action hosted on multiple services. I don't think we will ever need it (since they do the same thing no matter which service they are on, so why host it twice?), but if we do we'll have to update the code to allow it.
The next step after this is to make sure all the @Parameters are defined on the Action itself, and then we will be able to craft access to the endpoint programatically (or at least verify at run-time we crafted a correct URL)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229375735
All current tests that use caches with custom data expiry values
now restore the default config when teardown. We need to prevent
new unsafe uses from being introduced.
Restoration code have also been added to a few other tests that modifies
static fields.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228888041
The link was previously being sent using a JS redirect, which doesn't work
because the endpoints that trigger mapreduces can only be hit from the command
line (because they require auth). This commit switches the link to be in
plaintext and renders the full URL instead of just the path, so that clicking it
directly from the terminal works.
This also improves how these links are sent from callsites by using a fluent
style.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228764606
This also deletes the associated commands and domain application specific
entities.
We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.
Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
This is expected as part of the normal way that this mapreduce runs, so it's not
worth outputting an error.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228553579
This makes the deletion mapreduce more resilient in the face of data integrity
violations (which exist on sandbox but hopefully not in production). Even when
the domain application index doesn't exist, we still want to delete the domain
application itself, as its continuing presence will cause problems after the
code for domain applications is deleted.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228521794
This also deletes associated entities including indexes and history entries.
This needs to run as a prerequisite to [] which deletes all domain
application code entirely. The entities themselves need to be deleted first so
that loading DomainBases in the future doesn't accidentally get applications
which the code no longer knows how to handle.
This deletion is safe to perform because the only remaining applications in our
system are historical and we no longer refer to them. Backups will be retained
in BigQuery.
This mapreduce will be deleted at the same time that the DomainApplication code
is.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227738528
This does not change existing functionality but will allow us to use this common code in the yet-to-be-created web console action as well.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227159346
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
This temporarily disables the ability to download service account credentials. We'd like this to go in to next week's release so that we do not expose this functionality to users who otherwise do not have the ability to download the service account credential, while we research on a more secure solution.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223397754
As previously written, it loaded up all history entries into memory and then
processed them. This was OOMing for some registrars on sandbox who had performed
a large number of testing actions, most of them long OT&E was passed.
This commit changes the verify OT&E action to stream the history entries in
batches, ordered by when they were made, and then terminates once all tests have
passed. This prevents OOMing because only a single batch of history entries need
reside in memory at once.
This does necessitate the creation of a new composite Datastore index on
HistoryEntry, so we'll need to run the ResaveAllHistoryEntriesAction in sandbox
after this change is deployed before the new verify OT&E code will work.
Note that the "history viewer" is long dead, but that the pre-existing index
on HistoryEntries is still used for many other purposes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223163337
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
This is one last hanging piece of work left over from last year's Java 8
migration. There's no functionality changes in this CL, just refactoring.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201947600
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
This limit did not exist prior to [] which added the ability to limit
the size of the list. I didn't think that we needed to be able to query more
than 30 TLDs at any one time so I got rid of batching, but it turns out we do
need this ability for domain_watcher. So I'm re-adding batching, which is a
little bit more complicated now that we're also limiting and sorting by creation
time.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199826414
Currently, we have two different ways to parse a "set" parameter:
key=value1&key=value2&key=value3...
and
keys=value1,value2,value3
This is error prone for several reasons:
- different parts of the code must be "synchronized" to use the same style (the
place that creates the request, and the place that parses the request)
- for the key=value1&key=value2, we often use the same key name for the single
value and the set value. This can result in subtle bugs where part of the
code will successfully read the key assuming there's only one key (and will
get the first key=value1, ignoring the rest)
Here we transition everything to the keys=value1,value2,value3 method. This one
was chosen because:
- it's shorter
- it's more intuitive for users
- the key name is plural, differentiating it from the singular key=value that
other requests might need
-----------------------------------
To make sure there are not "transition issues", we will continue to support
(with warnings) the key=value1&key=value2 parameter parsing until we're sure we
haven't forgotten to update any part of the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198810681
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.
No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.
Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
This is a 'red' Flogger migration CL. Red CLs contain changes which are
likely not to work without manual intervention.
Note that it may not even be possible to directly migrate the logger
usage in this CL to the Flogger API and some additional refactoring may
be required. If this is the case, please note that it should be safe to
submit any outstanding 'green' and 'yellow' CLs prior to tackling this.
If you feel that your use case is not covered by the existing Flogger API
please raise a feature request at []and
revert this CL.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198463651
You don't want to use the cache when loading them for the purposes of updating
them, but you definitely do still want to use the cache when checking the
price of individual domains.
In [] the cache clearing of premium lists on update was removed. This
is a good thing in aggregate because the cache is per-instance and thus
misleading, but it also caused us to not be able to update the same premium
list twice within an hour because the second update would hit a "PremiumList
was concurrently edited" exception, owing to first loading the stale version
from the cache for the purposes of updating it. Now we bypass the cache for
that purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197768142
This allows list_domains to continue working for large TLDs.
TESTED=Deploys to alpha and it works to list the most recently created domains even
on a TLD with a huge number of domains on it (much more than .app has currently).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196717389
It doesn't make sense to log all 10K+ lines of a premium list every time it's
updated, and indeed that seems to hurt performance, yet that's precisely what
we were doing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194449836
Before this change the output looks like this:
registrar1 - Num actions: 93 - Reqs passed: 16/16 - Overall: PASS
registrar2 - Num actions: 47 - Reqs passed: 6/16 - Overall: FAIL
After this change the output looks like this:
registrar1 - # actions: 93 - Reqs: [----------------] 16/16 - Overall: PASS
registrar2 - # actions: 47 - Reqs: [...--.-...-...--] 6/16 - Overall: FAIL
The status of each test is displayed as a hyphen (passing) or a period (failing),
and the tests are always displayed in the same order so it's easier to get an overall
view of whether registrars are struggling with the same tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190776935
We already have verifications that a domain application was created in sunrise
- which checks for end-date sunrise. Start-date sunrise has checks that a
domain (not application) was created. There's no need to specifically check for
a signed mark, since a successful domain create during sunrise must have a
signed mark in it.
Also removed the requirement for end-date sunrise / landrush testing.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190551080
This fixes up the following problems:
1. Using string concatenation instead of the formatting variant methods.
2. Logging or swallowing exception messages without logging the exception
itself (this swallows the stack trace).
3. Unnecessary logging on re-thrown exceptions.
4. Unnecessary use of formatting variant methods when not necessary.
5. Complicated logging statements involving significant processing not being
wrapped inside of a logging level check.
6. Redundant logging both of an exception itself and its message (this is
unnecessary duplication).
7. Use of the base Logger class instead of our FormattingLogger class.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182419837
Stream.concat only accepts 2 parameters. Streams.concat on the other hand
accepts any number of parameters.
Moving to Streams.concat for all uses (2 or more) makes sense for uniformity
and convenience reasons.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179716648
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
