This should have been deleted in [] when the underlying entity was
deleted, but it was missed. It's been a no-op since then. This is just cleanup.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231293033
This code was never finished or fully working anyway. It would require
substantial reworking for the Registry 3.0 migration because it's closely tied
to the Datastore model and App Engine MapReduce framework, both of which will be
going away. We can bring back some of these deleted test files as necessary
if/when we rewrite RDE import for the new schema.
On the plus side, in a relational database, RDE import will be much simpler.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231265578
They are passed around in the format username:password, whereas just saying
"login" implies it's just a username and not necessarily also a secret
password. Putting password in the variable name makes it obvious what this is
and reduces the likelihood of anyone ever logging it or otherwise using it
inappropriately.
Note that this does not require data migrations as the actual key used to store
the data in KMS remains unchanged.
This is a follow-up to []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231253964
This will help us to debug the current MarksDB issue. This also throws an explicit error earlier when attempting to connect to MarksDB without login credentials being specified, which we know will fail.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231236317
We now display the results of each check in addition to the overall result.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231051913
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
Most common:
- Unnecessary parentheses and operator precedence clarify (self-explanatory)
- Reference equality--there were a few instances of using == or != improperly
- Qualification of Builder (and similar) imports so that it's clear which type of Builder we're referring to
- Marking some immutable classes with @Immutable since EP desires that all enums be deeply immutable
- String.split() having "surprising behavior"
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230971531
This view used to point to the latest datastore backup
using the deprecated export mechanism, which has been
disabled. We will point this view to the new backups.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930872
This eliminates the use of Objectify polymorphism for EPP resources entirely
(yay!), which makes the Registry 3.0 database migration easier.
It is unfortunate that the naming parallelism of EppResources is lost between
ContactResource, HostResource, and DomainResource, but the actual type as far as
Datastore was concerned was DomainBase all along, and it would be a much more
substantial data migration to allow us to continue using the class name
DomainResource now that we're no longer using Objectify polymorphism. This
simply isn't worth it.
This also removes the polymorphic Datastore indexes (which will no longer
function as of this change). The non-polymorphic replacement indexes were added
in []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230930546
This is currently erroring out on entities that fail to load properly through Objectify (e.g. because their entity type is no longer registered). The proper thing to do is to catch the error, log it, and fall back to the raw Datastore operation, which will succeed.
The exact Exception this is designed to catch is:
com.google.apphosting.runtime.jetty9.JettyLogger warn: /_dr/admin/deleteEntity (JettyLogger.java:29)
java.lang.IllegalStateException: No registered subclass for discriminator 'DomainApplication'
at com.googlecode.objectify.v4.impl.PolymorphicEntityMetadata.getConcrete(PolymorphicEntityMetadata.java:133)
at com.googlecode.objectify.v4.impl.PolymorphicEntityMetadata.load(PolymorphicEntityMetadata.java:164)
at com.googlecode.objectify.v4.impl.LoadEngine.load(LoadEngine.java:220)
at com.googlecode.objectify.v4.impl.LoadEngine$1.nowUncached(LoadEngine.java:178)
at com.googlecode.objectify.v4.impl.LoadEngine$1.nowUncached(LoadEngine.java:164)
at com.googlecode.objectify.v4.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.v4.impl.Round$1.nowUncached(Round.java:73)
at com.googlecode.objectify.v4.util.ResultCache.now(ResultCache.java:30)
at com.googlecode.objectify.v4.LoadResult.now(LoadResult.java:25)
at google.registry.tools.server.DeleteEntityAction.loadOfyEntity(DeleteEntityAction.java:103)
at google.registry.tools.server.DeleteEntityAction.run(DeleteEntityAction.java:74)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230737553
These indxes do not include the ^i field like the current ones do.
We'll need these so that we can properly index into the DomainBase entities once
DomainResource is merged into them (which will cause us to lose the value of the
^i field).
The migration plan is to push next week's release with these new indexes, then
run the re-save all EPP resources action to populate them, and then eventually
remove the ^i versions of the indexes, retaining only these newly added ones,
once the DomainBase/DomainResource merge is accomplished.
TESTED=Deployed to alpha, ran the re-save all EPP resources mapreduce, and
verified that the new indexes were populated.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229992464
For now, it only displays a status of "Passed: true|false" or an error message in simple text. In further work we will make the UI nicer.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229971564
Remove the GoogleInternal designator on the premium/reserved list tests, move
the lists themselves under the config/files directory, and fix all of the
surrounding infrastructure.
The lists do not get exported. Only the "example.txt" list does, allowing the
test to function on the open source build.
TESTED:
Verified that the gradle build works and that only the example.txt file is exported.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229940659
There was no reason to have several different modules all providing a single
thing. This approach, which creates a single UtilsModule for everything in the
util package, is cleaner. This also removes provisioning of Random and
StringGenerator objects in RegistryConfig.ConfigModule, which don't belong
there because they aren't configuration options.
This also removes insecure random entirely; it was only used in a
single place to generate 24 bytes a couple times per day. We can live with the
lower speed if it means we don't have to worry about multiple types of Random,
or possibly using an insecure random accidentally in a place that security
actually does matter.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229751915
This console is only to be used by Admins (either GAE admins for this project, or Support accounts). It is for "internal" use only, not for use by the registrars themselves.
To prevent abuse, the registrar is created in a non-functional PENDING state and can only be made functional from the nomulus shell tool.
While in "PENDING" state, the registrar can be updated from the registrar-console by admins.
Also - moving all the web consoles to the same directory (moving the otesetup/* files into registrar/)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229681011
Currently, the TLD is a "hidden" part of the metric - we record it, but by looking at the call site you can't see that we record it.
Also, it's injected from the query / POST parameter, so we might not even be aware of what the value is and it might not be the value we wanted.
Instead, making it explicit in the Metric call. That way it's also more similar to the "logging" statements that record the same data but have to explicitly output the TLD.
It also makes the tests better, as we test that we indeed record the correct TLD
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229550115
This removes the configuration ability on both Registry and Registrar entities
to allow operations on premium domains to succeed without acking the fees using
the fee extension. We only ever used this ability during the minna launch, and
it was a fiasco. We have no intention of ever allowing creation, renewal,
transfer, restoring, etc. of premium domains without acking the fees ever again,
and haven't done so since 2013, so removing this ability allows us to simplify
our code, data model, and tests.
Note that all TLDs in our production system currently require price ACKing
anyway, so from an external partner perspective this commit is a noop.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423650
The "lenient" bit must be the same between RdeStagingMapper and
RdeStagingReducer, but this is hidden by the Reducer receiving the bit in a
completely different way than the mapper.
There are 2 ways to do this:
- add a "setLenient" function to RdeStagingReducer that we MUST call, or else
get a runtime error. This is the simplest solution
- have a RdeStagingReducerBuilder you can inject, and that requires the
"lenient" value to actually build the RdeStagingReducer. This prevents bugs
at compile-time but is "more complicated"
I'm going with the second one here, but feel free to ask for the first one.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229423590
Our goal is to be able to address every Action by looking at the class itself, and to make it clearer at a glance what you need to access the Action's endpoint
Currently, we can know from the @Action annotation:
- the endpoint path
- the Method needed
- the authentication level needed
This CL adds the service where the Action is hosted, which also translates to the URL.
NOTE - currently we don't have any Action hosted on multiple services. I don't think we will ever need it (since they do the same thing no matter which service they are on, so why host it twice?), but if we do we'll have to update the code to allow it.
The next step after this is to make sure all the @Parameters are defined on the Action itself, and then we will be able to craft access to the endpoint programatically (or at least verify at run-time we crafted a correct URL)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229375735
Command would be useful when checking the status of daily
backup jobs.
Only two modes are supported: list all operations, or list those
that started within the past period as specified by command line
option.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229239783
Allows correct service names (i.e. lowercased, as they appear in App Engine
configuration files and the GCP console), and adds single letter parameters for
common flags.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229194479
This makes the validation rules much simpler, thus placing less cognitive load on the users of the console and nomulus tool. The changes are:
1. Don't allow hyphens. No real registrars use hyphens in their client IDs, and it's better to reserve these solely as the delimiter between the base client ID and the number representing the environment.
2. Allow the first character to be a number. This has affected multiple real registrars, causing their OT&E and production client IDs to be different. There's no reason for this restriction, as the only reason motivating it was that there are no TLDs that start with a number. However, the OT&E TLDs are created only in sandbox and never have DNS syncing enabled, so this restriction is purposeless.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229187198
Also using Service instead of serviceId String - because it helps readability.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228952033
Uncaught exception skips the call to metricBuilder.setNumResults()
and triggers an AutoValue builder exception from the finally block.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228936029
Takes advantage of the fact that the default state of a TLD created in tests is GENERAL_AVAILABILITY.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228916164
Remove those that call the /_dr/task/exportSnapshot endpoint.
The new export mechanism through /_dr/task/backupDatastore
has been running successfully in production.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228908704
For each registrar, the daily email will only include threats that did not appear
in the prior run's email.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228889972
All current tests that use caches with custom data expiry values
now restore the default config when teardown. We need to prevent
new unsafe uses from being introduced.
Restoration code have also been added to a few other tests that modifies
static fields.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228888041
This backs out most of [] fixes the external build (which wasn't
finding Apache Commons correctly), and makes miscellaneous tweaks and fixes,
including better handling representing the default case of decrypting to stdout.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228877090
Added a separator between the fields, and marked required fields as "required", so you can't submit without them
Also - changed from base64 to base58 in for the auto-generated password. It's conceivable that someone might need to read it outloud to someone else - and not having "visually similar" characters (like O and 0) can be helpful.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228810158
The link was previously being sent using a JS redirect, which doesn't work
because the endpoints that trigger mapreduces can only be hit from the command
line (because they require auth). This commit switches the link to be in
plaintext and renders the full URL instead of just the path, so that clicking it
directly from the terminal works.
This also improves how these links are sent from callsites by using a fluent
style.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228764606
The Visibility field isn't in public Guava yet, so just remove it.
This fixes the breakage caused by []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228759870
This also deletes the associated commands and domain application specific
entities.
We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.
Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
Generated code is now also covered by @VisibleForTesting, including Dagger @Inject
This CL is a cleanup of auto-generated code by ghm@ from the Error Prone team
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228748874
Certain flows need to launch batched jobs. Logically this would mean that flows
depend on batch.
However, the current state of dependency was the other way around, and the
reason for that was ResourceFlowUtils.java that had in it some utility
functions that weren't used in the flows and were needed in the batch jobs.
This CL removes these utility functions from the /flows/ directory, letting us
reverse the dependency edge between flows/ and batch/
Part of this was moving the flows/async/ code into batch/ - which also makes sense because flows/async/ just "enqueued" tasks that would then be run by actions in batch/
It makes sense that the code that enqueues the tasks and the code that dequeues the tasks sit in the same library.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228698761
This is expected as part of the normal way that this mapreduce runs, so it's not
worth outputting an error.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228553579
This makes the deletion mapreduce more resilient in the face of data integrity
violations (which exist on sandbox but hopefully not in production). Even when
the domain application index doesn't exist, we still want to delete the domain
application itself, as its continuing presence will cause problems after the
code for domain applications is deleted.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228521794
Two commands are being added:
- ImportDatastoreCommand starts an async import operation.
User may choose to wait until import completes or quit
immediately.
- GetOperationStatusCommand checks the status of an operation.
It may be used to check the status of an operation started by
ImportDatastoreCommand.
Both commands communicate with Datastore admin api directly, without
going through the Registry server.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228400059
Files.copy() attempts to delete the file if it already exists, which obviously
won't work very well for /dev/stdout. Instead copy directly from the decoder
to standard output.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228384575
Files.copy() attempts to delete the file if it already exists, which obviously
won't work very well for /dev/stdout. Instead copy directly from the decoder
to standard output.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228384575
This uses the new backup implementation, and starts after
the old exportSnapshot task has completed. The old task
will be removed later.
Daily backups in alpha has been running successfully.
Manually triggered exports in both environments also completed
successfully.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227908077
It was saying it was deleting tokens it wasn't, because it was outputting the
raw input list of tokens rather than the list that filtered out redeemed or
domain-specific tokens.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227769266
