Per discussions here:
https://groups.google.com/forum/#!topic/nomulus-discuss/ylDW2PblL60
Any use of keyring in the FOSS build would result in crashes because KMS is not configured. We should use the dummy keyring instead so that a vanilla FOSS deployment to GAE can run. Of course users would still need to configure their keyrings (and revert back to KMS keyring module) when they actually use any of the keys.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175868399
They can be inferred correctly even in Java 7, and display as
compiler warnings in IntelliJ.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173451087
The code was not checking the value returned by getTrid() to make sure it was
not null.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172384499
This will result is a prompt like
Creating TLD with:
projectId= domain-registry-alpha
description= some description
dnsName= mytld.
name= mytld.
nameServerSet= cloud-dns-registry-test
Perform this command? (y/N):
before actually performing the command, and adds a --force flag to bypass the
prompt.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172369944
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
Automated g4 rollback of changelist 171011304. Please see history below, the original CL was [] Rolling back and re-mailing
so I can get a readability review on it.
*** Reason for rollback ***
Re-opening for readability review.
*** Original change description ***
Automated g4 rollback of changelist 170906329.
*** Reason for rollback ***
Forgot to send this for readability review.
*** Original change description ***
Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171161371
*** Reason for rollback ***
Forgot to send this for readability review.
*** Original change description ***
Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
b/67367533 created to clean this up, along with...
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171011304
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
b/67367533 created to clean this up, along with related warts in the codebase.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170906329
Strict autoescaping is the default so they serve no purpose.
Design doc: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170905337
Specifically, this prevents suspended registrars from creating domains or applications. Pending registrars already can't perform these actions because they get an error message when attempting to log in.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170481338
It makes sense for all mapreduces to run in backend, especially onces
that are scheduled regularly to run in cron like this one now. We don't
have many instances configured for the tools service anymore on some
of our environments, so backend is the friendliest place for a mapreduce
to run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882122
We're planning on adding a cronjob to run this mapreduce monthly, so
we may as well also project the resources being re-saved to the present
time so as to handle pending transfers, grace periods, and such. This will
make the BigQuery exports more useful.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168831056
Void tags (e.g. <img>) couldn't have a closing tag (e.g. </img> is invalid). Non-void tags (e.g. <div>) couldn't be self closing (e.g. <div/> is invalid) and must be closed explicitly (e.g. with </div>). This CL fixes the tags which also prepares the templates for stricthtml which enforces it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168829028
This pattern will mainly be used for data migrations, i.e. updating all
HistoryEntries' DomainTransactionRecords to the new schema.
TESTED=Ran in alpha, the underlying data dropped non-Objectify fields as
expected.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168684356
Implement toplevel class that reads in two database backups and displays
diffs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168592124
RecordAccumulator builds a set of datastore Entity records from a set of
leveldb logfiles in a directory tree (which is how we receive them for
database backup/restore testing).
This CL also refactors some of the logfile test code out of
LevelDbLogReaderTest so that we can reuse it for building test logs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167313553
ComparableEntity allows us to compare Entity objects by both key and
properties, regardless of the order of those properties in the protobufs.
This is necessary for comparing database backups, where we evaluate the
differences in the set of entities in one backup with those of another.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167288489
This is the first in a series of CLs containing code from an old CL of Dai's that had never been completed, which compares zone data between Datastore and DNS. I had written a script to do this by calling two nomulus commands, but maybe it can be done directly in Java, which would be convenient.
This CL is just the plumbing to check on the status of a Mapreduce. We will need this to know that we can proceed with the next step of comparing the output to the DNS data.
Cloned from CL 134295050 by 'g4 patch'.
Original change by dxy@dxy:zoneman-reader:1939:citc on 2016/09/26 10:34:22.
Add a command for comparing zone data between DNS and datastore
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167188979
To log autorenews, we currently run a mapreduce daily that creates synthetic
billing events for each recurring event past its due time. These are all
parented under the original recurring event, which allows these synthetic events to incorrectly stack on the original mutating entry.
We now explicitly create a new HistoryEntry of type DOMAIN_AUTORENEW to log
autorenews alongside other mutating EPP flows. These also parent DomainTransactionRecords for the NET_RENEWS_1_YEAR field, with the reporting time equal to the billing time (which accounts for the autorenew grace period).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166379700
In standard DNS format, the first thing on an A, NS or DS definition line is a domain label relative to the zone, which in our case is a TLD. However, the generate_zone_files command prints out fully qualified host and domain names, resulting in a discrepancy when compared to the contents of the DNS subsystem. This CL removes the TLD suffix, which should remove one preprocessing step before file comparison.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166103705
Previously, GenerateZoneFilesAction mapreduced its way through all domains and hosts for the specified TLD(s), emitting information for each matching domain and host (subject to constraints like not being deleted and so on). This resulted in host information (aka glue records) for all hosts subordinate to domains in the specified TLD(s). This is incorrect. DNS glue records should only be present for hosts which act as nameservers for their superordinate domains.
The new version of the mapreduce iterates only over domains. When a matching domain is found, a check is made to see whether any subordinate hosts are also nameservers for the domain, in which case host information is generated.
The test was updated to reflect the new reality, and check for a couple additional nuances.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165766472
This is a first step towards porting our database backup comparison tool
(which was written in Crack and subsequently in Python) to Java so that we can
run it in a reasonable amount of time. Ideally, this functionality should be
available in Java, however the only instance of it that I can find is
currently deprecated without a replacement.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165433766
Attending to this old bug will improve our ability to perform zone comparisons between Datastore and the DNS provider. Right now, zone comparison finds some bogus differences, because the TTL we send to the DNS subsystem doesn't match the TTL we use when generating our local dump files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164635557
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).
This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
Note that even though the nomulus command line tool now supports multiple
DNS writers for all subcommands, this still won't work quite yet because
the DNS task queue format migration from [] is still in progress.
After next week's push that migration will be complete and we can remove
the final restriction against only having one DNS writer per TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162490399
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
After this point all data is migrated to use the new canonical
plural version, and subsequent code changes can be made that use
multiple writers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161673486
This makes it take a lot less time to run (roughly a 10X speedup).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161666391
This will allow us to migrate one TLD at a time by refreshing all zones
on the specified TLD after dual-writing is enabled.
Note that the TLDs parameter is required, which seems reasonable given
that almost all imagined use cases would be on a by-TLD basis.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160294546
I'm moving it out of the scrap folder too because there's nothing else
in there and we do want to retain this indefinitely because it's a useful
tool for performing DNS writer migrations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160168902
I happened to notice that one domain affected by b/33289763 had existing transfer data, so in that case the data got overwritten to look like a server cancellation (a year after the transfer request, so obviously wrong) rather than creating invalid transfer data, and hence wasn't picked up in the previous fixes. See b/33289763#comment32 for more details.
This CL renames the RemoveDomainTransferDataCommand used to fix the errors to FixDomainTransferDataCommand and changes its behavior to instead reset the TransferData to a server-approval result. It also changes it to load domains by ROID because loadByForeignKey() fails for deleted domains.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159273225
When doing update_registrar, it is now possible to only specify the currencies and the account ids that need updating.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159262119
Memcache is already off but now it's not in the code anymore.
This includes removing domain creation failfast, since that is actually
slower now than just running the flow - all you gain is a non-transactional
read over a transactional read, but the cost is that you always pay that
read, which is going to drive up latency.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158183506
The command was set up such that an update without any contact types specified would clear out the list, instead of leaving them unchanged, as it should.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157766429
This brings the affected actions into line with the settings on other similar actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157259842
