Note that even though the nomulus command line tool now supports multiple
DNS writers for all subcommands, this still won't work quite yet because
the DNS task queue format migration from [] is still in progress.
After next week's push that migration will be complete and we can remove
the final restriction against only having one DNS writer per TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162490399
It should not be multiline, as registrar client ids are single short-ish identifiers with no spaces allowed. There's no way for them to span multiple lines.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162389442
I know the query that finds commit logs already should ignore commit logs
that are too young, but this adds an explicit sanity check for safety's
sake, so we don't have to depend solely on an indexed query for safety.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162386413
The dry run does all the steps except the deletion. All the counters will
return the same values they would have returned on an actual run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162379609
This change is motivated by the sandbox run where we saw the backend instances overwhelmed by the 100 default shards to the point where they couldn't even answer a simple status request.
Production has 50 backend instances, so 20 will leave a lot of spare for other tasks.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162357857
This is written in such a way that it can safely handle task items in the
old format so long as the DNS writer to use for the given TLD is unambiguous
(which it is for now, until we allow multiple DNS writers to be configured).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162293412
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
Also updates Truth version to 0.34 where the replacement method was added.
More information: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161970305
This allows us to have a modular view of all tables used in activity reporting, to facilitate generating reports in BigQuery.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161849007
After this point all data is migrated to use the new canonical
plural version, and subsequent code changes can be made that use
multiple writers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161673486
This makes it take a lot less time to run (roughly a 10X speedup).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161666391
Now instead of deleting "all logs older than X", we delete "all logs older than
X that don't have any EppResource.getRevision()" pointing to them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161455827
This is the first step in a multi-step data migration to allow multiple
DNS writers per TLD. The overall process looks like this:
1. Add a plural DNS writers field with backfill (this commit).
2. Deploy it.
3. Run the ResaveEnvironmentEntitiesCommand to populate this new field
on all entities.
4. Update the code to use the new field everywhere.
5. Deploy it.
6. Delete the now-unreferenced, old deprecated singular value field.
This process is rollback-safe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161253436
The billing account map will be serialized in the following format:
{currency1=id1, currency2=id2, ...}
In order for the output to be deterministic, the billing account map is stored as a sorted map.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161075814
We're already storing this as a metric, but on a registry of our
scale these operations tend to only happen on a daily-ish basis,
for which seeing results in logs is easier to deal with than metrics
(and also still very light-weight).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160552291
This is the first step in moving the current []cron-Python reporting scripts
into App Engine, as an official part of the Nomulus package. This copies the
structure of RDE uploads, with a few changes specific to monthly reporting.
I've left some TODOs related to actually testing it on the ICANN endpoint, as we're still not sure how files to be uploaded will be staged, and whether we can actually ping their endpoint on valid ports (80 or 443).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160408703
This will allow us to migrate one TLD at a time by refreshing all zones
on the specified TLD after dual-writing is enabled.
Note that the TLDs parameter is required, which seems reasonable given
that almost all imagined use cases would be on a by-TLD basis.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160294546
I'm moving it out of the scrap folder too because there's nothing else
in there and we do want to retain this indefinitely because it's a useful
tool for performing DNS writer migrations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160168902
Up to now, our search wildcard rules have been that there must be an initial string of at least two characters. If a wildcard is present after that, it can optionally be followed by a suffix specifying the TLD (for domains) or domain (for nameservers). So domain queries can look like:
example.tld
ex*
ex*.tld
and nameserver queries can look like:
ns1.example.tld
ns*.example.tld
ns*
But you can't do a domain query for *.tld, nor a nameserver query for *.example.tld. It would be nice to support such queries, and the presence of a valid TLD or domain makes them relatively efficient. This CL relaxes the restrictions to allow wildcards with no initial string if the suffix is present. For nameservers, the suffix must be a valid domain in the system, to avoid having to loop through all nameservers.
A side effect of the changes is to fix a shortcoming in the logic which caused wildcard nameserver searches to fail if the specified domain suffix referred to an external domain.
Entity searches are not affected, since they do not support suffixes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159856563
I'm making this change ONLY for packages with targets that I'll be removing the appengine constraint from. default_compatible_with is fine in general; I just want to separate the default_compatible_with->compatible_with CLs from the CLs that remove compatible_with.
This is part of the cleanup after []
Tested:
blaze build --nobuild
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159713533
The affected actions have been changed to check that the user is logged in by [] so this attribute is no longer needed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159572365
private="true" has never done very much unfortunately. There is an alternate
way to mark templates as private by setting 'visibility="private"' which does
get enforced both by the soy compiler and by custom per-language strategies in
the various backends.
Unfortunately, it isn't possible to safely migrate from one to the other since
users may be calling these templates from server side soy which enforces
visibility via a runtime exception. So they are just being deleted.
LSC: https://docs.google.com/document/d/1aOM_tBmanDQolF8mAuB0y29yB76cmYS5qOVYBAzFzyw/edit#
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159565458
We are going to remove the requireLogin attribute from the action attribute, because it is specific to the UserService API. This is used by four actions:
ConsoleUIAction
RegistrarSettingsAction
RegistrarPaymentSetupAction
RegistrarPaymentAction
Instead, these four actions will now check the login status directly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159562335
Now that the registration period has been added to DomainApplication, we
can remove this @OnLoad that was populating it for objects that were
missing the period.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159464438
I happened to notice that one domain affected by b/33289763 had existing transfer data, so in that case the data got overwritten to look like a server cancellation (a year after the transfer request, so obviously wrong) rather than creating invalid transfer data, and hence wasn't picked up in the previous fixes. See b/33289763#comment32 for more details.
This CL renames the RemoveDomainTransferDataCommand used to fix the errors to FixDomainTransferDataCommand and changes its behavior to instead reset the TransferData to a server-approval result. It also changes it to load domains by ROID because loadByForeignKey() fails for deleted domains.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159273225
When doing update_registrar, it is now possible to only specify the currencies and the account ids that need updating.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159262119
The --superuser command in the nomulus command-line tool should be
bypassing checks on whether the passed-in registrar client ID has access
to the TLD in question, but currently it is not.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158974462
We currently only have one method for fine level logging. Add another three so that fine level logging can be called just like other levels.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158760803
