Automated g4 rollback of changelist 171011304. Please see history below, the original CL was [] Rolling back and re-mailing
so I can get a readability review on it.
*** Reason for rollback ***
Re-opening for readability review.
*** Original change description ***
Automated g4 rollback of changelist 170906329.
*** Reason for rollback ***
Forgot to send this for readability review.
*** Original change description ***
Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171161371
Also fixed a bug discovered as a result of the more complete testing!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171156304
Also fixes the issue that dry run EPP commands were incorrectly being
reported on.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171062984
This CL changes the domain and contact transfer flows to check the entire
TransferData on the post-transfer resource, rather than just spot-checking
certain fields. This approach provides much better code coverage - in
particular, it checks that the non-request flows (approve, cancel, reject)
don't modify the fields that they shouldn't be modifying, and that they do
actually clear out the transfer server-approve entities fields written by
the transfer request flow. It's slightly orthogonal, but I also added
testing that the server-approve entities fields are actually set in the
request flows, which was previously untested.
This is pre-work for introducing an exDate-storing field into TransferData,
by making it easier to test everywhere that exDate is set *and* unset only
in the correct places.
As part of this CL, I've introduced a TransferData.copyConstantFieldsToBuilder()
method that is like asBuilder() but instead of copying all the fields to the new
builder, it only copies the logically constant ones: losing/gaining client IDs,
the request time and TRID, and transferPeriod. This is useful both in tests but
is also used in the resolvingPendingTransfer() helper that centralizes the core
transfer resolution logic (as of [] That method has its own tests,
and in the process I removed a bunch of crufty defunct TransferData tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171053454
The concrete implementation of a Metric is not of importance when asserting on the values it contains. Therefore this CL removes Metric<T> as a type parameter of AbstractMetricSubject. As a result the two implementations of the abstract subject can be used on any Metric<Long> and Metric<Distribution>, respectively.
Also migrate to Subject.Factory from deprecated SubjectFactory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171012012
*** Reason for rollback ***
Forgot to send this for readability review.
*** Original change description ***
Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
b/67367533 created to clean this up, along with...
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171011304
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.
Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login. Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.
b/67367533 created to clean this up, along with related warts in the codebase.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170906329
Strict autoescaping is the default so they serve no purpose.
Design doc: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170905337
Tools inheriting from MutatingCommand print out the change they are going to
make and then ask the user to confirm that this is indeed what they wanted to
do.
The change is outputted as a list of updated values in the form
key -> [oldValue, newValue]
e.g.
dnsPaused -> [true, false]
This CL will change the output to be clearer:
key: oldValue -> newValue
e.g.
dnsPaused: true -> false
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170853745
In the great flow flattening, ResourceFlowUtils grew a couple nice helpers
for rebuilding transferrable resources (Domains and Contacts) upon the
resolution of a transfer - approvePendingTransfer() and denyPendingTransfer().
Most transfer-resolving callsites use one of these two helpers, but for legacy
reasons the deletion flows (DomainDeleteFlow and DeleteContactsAndHostsAction)
were instead using the "manual" resolvePendingTransfer() method or its even more
low-level createResolvedTransferData() helper instead of denyPendingTransfer().
It's simpler to just have two options - approve and deny - so this CL inlines
createResolvedTransferData() into resolvePendingTransfer() and makes the latter
a private helper for the approve/denyPendingTransfer() public helpers.
This CL also adds sanity checks that approve/denyPendingTransfer() are called
only with the logically appropriate values of TransferStatus.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170819358
This CL adds the functionality for contact and registrar searches. A future CL will handle domains and entities.
Support is also added for filtering results by registrar.
Deleted items can only be seen by admins, and by registrars viewing their own deleted items.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170734664
Strict autoescaping is the default so they serve no purpose.
Design doc: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170725633
Normally, if a domain is in the auto-renew grace period, a transfer will cancel the auto-renew billing event. In the event of a transfer with no change to registration end date, the auto-renew billing event should not be cancelled and the gaining registrar should not be charged for the transfer.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170576726
We're going to need to switch away from Guava's Functions and Predicates for
everything and replace them with the java.util versions. Unfortunately there
does not appear to be an automated tool to do this all at once. Refaster got
close but doesn't seem to care about these particular types of mismatch (I
suspect we're using a different version of the JDK than the outside world;
ours is OK with Guava classes).
This also bumps up Guava to 0.23, which is needed for some new functionality
used in combination with Java 8 features.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170531539
It's bad practice in terms of data model consistency to use DateTime.now(UTC) for cloneProjectedAtTime() within a transactional context - instead, we should always use ofy().getTransactionTime() when it's available. The ofy transaction time is frozen at the start of the transaction attempt, whereas DateTime.now(UTC) will continue marching ahead. So using it for a cloneProjectedAtTime() could mean materializing the outcome of logical-time-dependent events (like autorenews, or transfers, or grace period expirations) at a logical time (i.e. commit log time) prior to when they should actually have occurred.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170501277
Users of the client library should be responsible for providing the bindings required. By removing the injectable constructors and named bindings in their parameters, the client library is no longer forcing its users to provide bindings with specific named annotations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170490834
Specifically, this prevents suspended registrars from creating domains or applications. Pending registrars already can't perform these actions because they get an error message when attempting to log in.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170481338
Add cloudDns.{rootUrl, servicePath} to allow us to point an environment at the
Cloud DNS staging API for testing. Make sandbox and alpha point to staging.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170340859
This removes the ramp-up and pre-ramp-up registrar fields, and lists 'CZDS' as
the zfa-passwords field, in accordance to the registry agreement changes listed
on the associated bug.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170337426
It occurs to me that we can't have this setting different between sandbox
and production, otherwise we can end up with a situation where we push code
that works on sandbox but then fails only when it is pushed to production.
Sandbox and production need to always be set up similarly for this reason.
We'll just have to pay a greater amount of attention to the release process
next week than normal, and continue playing around in alpha for the mean
time with a fully Java 8 build.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170197703
Since HistoryEntry always represents a read-only log of mutation to a core resource, that mutation should always happen in a transaction, and the HistoryEntry should be saved in that transaction. As such, it's always more accurate to use ofy().getTransactionTime() for the modificationTime of the HistoryEntry rather than just DateTime.now(UTC).
In addition, having these be the exact same timestamp makes it possible to align HistoryEntries with commit log manifests using modificationTime = transactionTime, which is useful for recovery and analysis purposes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170136957
This CL adds the functionality for nameserver searches. Future CLs will handle domains and entities.
Deleted items can only be seen by admins, and by registrars viewing their own deleted items.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170106014
The existing CloudDnsWriter code uses ImmutableMap.Builder to construct the
map of DNS records to update. This has been seen to fail on alpha, presumably
in a cases where host records and domain records produce duplicate updates for
a host.
Convert the Builder to a HashMap, allowing us to safely overwrite existing
records in the case of duplicates.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170103421
No reason for these not to work out of the box, right? The default-config.yaml already has a couple other real icann.org URLs for RDE testing and the TMCH CRL. This way Nomulus users don't have to worry about changing this setting (since other than testing it's hard to imagine what other value would be used; this is literally ICANN reporting so it always goes to ICANN).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170093954
*** Reason for rollback ***
Rolling back to fix []
*** Original change description ***
Remove @Inject annotation in StackDriverWriter
StackDriverWriter is provided by java.google.registry.monitoring.whitebox.StackdriverModule. The @Inject annotation in its constructor is not used. It also uses outdated named binding like @Named(stackdriverGcpProject) which causes confusion because that name is not provided anywhere in the code.
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169931047
Mostly based on the original PR, but with some tweaking by nfelt@, in particular to add some support for autorenew grace period subsumption.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169922894
StackDriverWriter is provided by java.google.registry.monitoring.whitebox.StackdriverModule. The @Inject annotation in its constructor is not used. It also uses outdated named binding like @Named(stackdriverGcpProject) which causes confusion because that name is not provided anywhere in the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169922096
Based on the original pull request below with some modifications for code drift
over time (including adding handling/testing for the case where superordinate
domains are in pending delete, and thus can't have hosts linked to them).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169698157
We can easily end up enlisting too many entity groups (separate
DomainApplications) in a TransactionalFlow when loading all applications
tracked by the DomainApplicationIndex. This makes the load operation
transactionless, to avoid overenlisting.
Potential problems:
1. We fail to prevent landrush applications, if a single sunrise application
exists. This is likely fine, except for a brief moment in Sunrush when a
sunrise application is made immediately prior to a landrush application. The
result is we accept an invalid application- which can be mediated manually.
2. We fail to prevent a domain create for a domain with an open application.
This is a little more sinister, but also unlikely unless someone submits an
application immediately before someone tries to create the same domain (sans
application?)
3. We return an invalid DomainCheck response (instead of 'pending allocation').
Not the worst outcome.
4. We reduce the AuctionStatusCommand and GetApplicationIdsCommand to
eventual consistency. Since they're internal tools, that's not too big a deal.
A better solution:
DomainApplications really should just be normalized under a virtualEntityGroup
by fullyQualifiedDomainName, or a hash-bucket like EppResources are. The
DomainApplication -> DomainBase -> EppResource hierarchy seems to be purely for
code reuse, at the cost of Datastore consistency. This would, however, require
quite some refactoring, and a custom resave operation across all
DomainApplications.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169395586
This moves us from the oudated google/data XML api to the OnePlatform REST/JSON api, finally silencing the deprecation warnings we've been seeing.
The synchronization algorithm diffs the spreadsheet's current values with its internally sourced values, adding the row to a batch update request if there's a discrepancy. Additional internal data are added as an append operation to the end of the sheet, and any extraneous spreadsheet data is cleared from the spreadsheet.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169273590
LogsExportCursor was only used by ExportLogsTaskServlet, which we removed a long time ago. It's just dead code. The PersistedRangeLong type was only written for use by LogsExportCursor, and since it hasn't picked up new users in 3+ years I don't think we need to keep it around.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169264994
There was a TODO to consider making RDAP queries look up domains by foreign key rather than directly to the domain itself. But since the TLD is a possible search filter, it makes more sense to stick with direct lookup, so we can take advantage of the index on TLD.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169240318
When searching for domains by nameserver name or IP address, we fetch the matching nameserver keys, then search for domains by those keys. We limit the number of nameserver keys returned, to avoid arbitrarily large domain queries. This CL adds a warning to the RDAP response if we retrieved the maximum number of nameservers. This may indicate that we have not found all the domains.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168885124
It makes sense for all mapreduces to run in backend, especially onces
that are scheduled regularly to run in cron like this one now. We don't
have many instances configured for the tools service anymore on some
of our environments, so backend is the friendliest place for a mapreduce
to run.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882122
The lock is used to prevent dual writing of the same output file, and to
prevent us from generating the same data twice (if RdeStaging was run twice).
However, at times when we're creating both RDE and BRDA reports - we use the
same lock for both even though they write to different files and create
different reports.
This causes one of the two to almost always fail (when both are created, which
is once a week I think) delaying that report by several hours.
Note that the "prefix" part of the filenames has the mode in it, so different
modes will not have the same files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882007
Sometimes requests "die" suddenly, without going through catch/finally blocks.
If this happens, any lock they own will remain locked until it times out (which
can take hours in some cases).
This cl implicitly unlocks any lock if the owner of the lock isn't running
anymore.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168880938
Also adds a "resave all epp" cron job that's needed for the delete to work correctly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168879965
We had been allowing lookups by nameserver name using a wildcard and suffix if the suffix was a domain name. That's ok if the domain name is one we manage, but doesn't work efficiently otherwise. A lookup of ns*.zombo.com would require us to search for all nameservers beginning with ns (which could be almost all of them), then loop through until we found those ending with .zombo.com. So we are going to prohibit suffixes after the TLD unless the suffix is an in-bailiwick domain.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168835732
We're planning on adding a cronjob to run this mapreduce monthly, so
we may as well also project the resources being re-saved to the present
time so as to handle pending transfers, grace periods, and such. This will
make the BigQuery exports more useful.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168831056
Void tags (e.g. <img>) couldn't have a closing tag (e.g. </img> is invalid). Non-void tags (e.g. <div>) couldn't be self closing (e.g. <div/> is invalid) and must be closed explicitly (e.g. with </div>). This CL fixes the tags which also prepares the templates for stricthtml which enforces it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168829028
