This also brings the SQL template parameters in-line with the anticipated Bigquery dataset format, and switches from DateTime to the more appropriate LocalDate (since we only need monthly granularity).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162747692
Note that even though the nomulus command line tool now supports multiple
DNS writers for all subcommands, this still won't work quite yet because
the DNS task queue format migration from [] is still in progress.
After next week's push that migration will be complete and we can remove
the final restriction against only having one DNS writer per TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162490399
It should not be multiline, as registrar client ids are single short-ish identifiers with no spaces allowed. There's no way for them to span multiple lines.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162389442
I know the query that finds commit logs already should ignore commit logs
that are too young, but this adds an explicit sanity check for safety's
sake, so we don't have to depend solely on an indexed query for safety.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162386413
The dry run does all the steps except the deletion. All the counters will
return the same values they would have returned on an actual run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162379609
This change is motivated by the sandbox run where we saw the backend instances overwhelmed by the 100 default shards to the point where they couldn't even answer a simple status request.
Production has 50 backend instances, so 20 will leave a lot of spare for other tasks.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162357857
This is written in such a way that it can safely handle task items in the
old format so long as the DNS writer to use for the given TLD is unambiguous
(which it is for now, until we allow multiple DNS writers to be configured).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162293412
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
Also updates Truth version to 0.34 where the replacement method was added.
More information: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161970305
This allows us to have a modular view of all tables used in activity reporting, to facilitate generating reports in BigQuery.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161849007
This standardizes use of annotations/inheritance/formatting across
tests, to make the code more legible and consistent.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161810734
This paves the way for [] which expands the set of classes Blaze will check for possible test methods that are not properly annotated.
For more details and FAQs please see: []
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161795590
After this point all data is migrated to use the new canonical
plural version, and subsequent code changes can be made that use
multiple writers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161673486
This makes it take a lot less time to run (roughly a 10X speedup).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161666391
Now instead of deleting "all logs older than X", we delete "all logs older than
X that don't have any EppResource.getRevision()" pointing to them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161455827
This is the first step in a multi-step data migration to allow multiple
DNS writers per TLD. The overall process looks like this:
1. Add a plural DNS writers field with backfill (this commit).
2. Deploy it.
3. Run the ResaveEnvironmentEntitiesCommand to populate this new field
on all entities.
4. Update the code to use the new field everywhere.
5. Deploy it.
6. Delete the now-unreferenced, old deprecated singular value field.
This process is rollback-safe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161253436
The billing account map will be serialized in the following format:
{currency1=id1, currency2=id2, ...}
In order for the output to be deterministic, the billing account map is stored as a sorted map.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161075814
We're already storing this as a metric, but on a registry of our
scale these operations tend to only happen on a daily-ish basis,
for which seeing results in logs is easier to deal with than metrics
(and also still very light-weight).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160552291
This is the first step in moving the current []cron-Python reporting scripts
into App Engine, as an official part of the Nomulus package. This copies the
structure of RDE uploads, with a few changes specific to monthly reporting.
I've left some TODOs related to actually testing it on the ICANN endpoint, as we're still not sure how files to be uploaded will be staged, and whether we can actually ping their endpoint on valid ports (80 or 443).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160408703
This fixes non-determinism in the order of the generated classdocs, which
appears to be causing test failures for us now, presumably due to some subtle
environmental change.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160296878
This will allow us to migrate one TLD at a time by refreshing all zones
on the specified TLD after dual-writing is enabled.
Note that the TLDs parameter is required, which seems reasonable given
that almost all imagined use cases would be on a by-TLD basis.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160294546
I'm moving it out of the scrap folder too because there's nothing else
in there and we do want to retain this indefinitely because it's a useful
tool for performing DNS writer migrations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160168902
Up to now, our search wildcard rules have been that there must be an initial string of at least two characters. If a wildcard is present after that, it can optionally be followed by a suffix specifying the TLD (for domains) or domain (for nameservers). So domain queries can look like:
example.tld
ex*
ex*.tld
and nameserver queries can look like:
ns1.example.tld
ns*.example.tld
ns*
But you can't do a domain query for *.tld, nor a nameserver query for *.example.tld. It would be nice to support such queries, and the presence of a valid TLD or domain makes them relatively efficient. This CL relaxes the restrictions to allow wildcards with no initial string if the suffix is present. For nameservers, the suffix must be a valid domain in the system, to avoid having to loop through all nameservers.
A side effect of the changes is to fix a shortcoming in the logic which caused wildcard nameserver searches to fail if the specified domain suffix referred to an external domain.
Entity searches are not affected, since they do not support suffixes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159856563
