Commit graph

1177 commits

Author SHA1 Message Date
mmuller
bf818a0139 Translate multi-part TLD zone names
Convert periods to hyphens in multi-part TLDs when using them as a zone name
(cloud-dns doesn't allow periods in zone names).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172007089
2017-10-24 16:53:47 -04:00
mountford
7e85ae9964 Fix problem with RDAP help information
Soon this will be rewritten with much more complete information. But this fixes
an obviously wrong bit in the meantime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172004672
2017-10-24 16:53:47 -04:00
mcilwain
05f6ad80ab Format some imports using fix_imports.py
Unfortunately this tool isn't smart enough to deal with the assertThat
situation (which has two static imports of a function with the same name).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172000753
2017-10-24 16:53:47 -04:00
mountford
326cf698e0 Don't validate RDAP nameserver names using validateDomainName
The nameserver may be external, in which case its TLD will not appear in our
list of valid TLDs, and the search will be rejected erroneously.

Tests for letter case canonicalizations also added at reviewer's suggestion.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171985702
2017-10-24 16:53:47 -04:00
mountford
9d1eb0d429 Set content type for RDAP responses before setting the payload
Also sets the charset to UTF8 explicitly, to avoid confusion, and removes apparently unneeded Truth8 include.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171963981
2017-10-24 16:53:47 -04:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
nickfelt
184b2b56ac Persist transferredRegistrationExpirationTime (exDate) on TransferData
This CL adds transferredRegistrationExpirationTime as a TransferData field
persisted to Datastore.  It's only relevant for domains, and it represents the
registration expiration time resulting from the approval of the most recent
transfer request.  For pending transfers, we assume the transfer will be
server-approved, and thus in DomainTransferRequestFlow we set this field to the
existing computed value serverApproveNewExpirationTime, which is what we use
for setting up the server-approve autorenew billing event and poll message.
In DomainTransferApproveFlow we overwrite this field with the freshly computed
newExpirationTime, whereas in DomainTransferCancel/RejectFlow (and in the
implicit cancel of DomainDeleteFlow during a pending transfer) we null it out.

There are two key benefits to having this field, which are described in more
detail in b/36405140.

1) b/25084229 - it allows storage of a frozen value to back the "exDate" field
   of DomainTransferResponse, which we can use to fix various errors with how
   exDate display currently works.

2) b/36354434 - it allows DomainResource.cloneProjectedAtTime() to just directly
   set the registrationExpirationTime to this value, without computing it de
   novo, which reduces duplicated logic and ensures that the new expiration time
   matches the autorenew child objects.

This CL only starts writing the field on TransferData as persisted directly on
the DomainResource itself.  We'll then want to backfill the field for at
least pending transfers, whether expired or not (so we can do (2) above), but
I think we might as well backfill it for all pending and approved transfers
so that we also fix (1) even for historical transfers.  And then we can start
actually reading the field for both purposes.  (Note that for (1), this will
only fix synchronous transfer responses served via DomainTransferQueryFlow,
not async transfer responses served via poll messages, since these have already
been persisted with a potentially bad exDate, but I don't think it's worth a
backfill for those).

One last naming note: I chose the verbose transferredRegistrationExpirationTime
rather than the extendedRegistrationExpirationTime of DomainTransferResponse
because (as is the case in autorenew grace, or for a superuser transfer) the
new registration time isn't necessarily extended at all; it may be the same as
the pre-transfer expiration time.  Also, including "registration" helps clarify
w.r.t. pendingTransferExpirationTime which refers confusingly to the expiry of
the transfer itself, rather than the domain registration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171858083
2017-10-24 16:53:47 -04:00
bbilbo
9a3877fdb2 Add a log message indicating that the restore is complete
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171719014
2017-10-24 16:53:47 -04:00
guyben
840d53c819 Allow EventSample.record to accept numSamples=0
There's really no reason not to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037754
2017-10-24 16:50:30 -04:00
mmuller
69d33e928a Use a direct reference to jsch 0.1.53
Use a direct reference to the jsch 0.1.53 package instead of going through the
target link, which doesn't exist after MOE transformation.  Also change the
target's visibility (since the whitelist referenced by the current rule isn't
preserved by the moe transform).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171554085
2017-10-10 12:09:42 -04:00
bbilbo
6d80482875 Remove all jar_urls pointing to maven.ibiblio.org and add missing mirrors
com_google_template_soy wasn't sorted, so I fixed that as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171347611
2017-10-10 12:09:42 -04:00
mmuller
a790ec27db Upgrade domain-registry to jsch 0.1.53
TESTED=Verified on Alpha with an updated build using Java 8, Verified on alpha
against the latest java 8 based build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171297613
2017-10-10 12:09:42 -04:00
bbilbo
24d58bf505 Add cron/commitLogCheckpoint to cron.xml for the crash environment
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171297448
2017-10-10 12:09:41 -04:00
mcilwain
13408532b3 Convert Registrar entity class to use more Java 8 features
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171207262
2017-10-10 12:09:41 -04:00
mcilwain
5edb7935ed Run automatic Java 8 conversion over codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171174380
2017-10-10 12:09:41 -04:00
mmuller
44df5da771 Add "create_cdns_tld" command to nomulus
Automated g4 rollback of changelist 171011304.  Please see history below, the original CL was []  Rolling back and re-mailing
so I can get a readability review on it.

*** Reason for rollback ***

Re-opening for readability review.

*** Original change description ***

Automated g4 rollback of changelist 170906329.

*** Reason for rollback ***

Forgot to send this for readability review.

*** Original change description ***

Add "create_cdns_tld" command to nomulus

Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171161371
2017-10-10 12:09:41 -04:00
mountford
07e5536988 Revise RDAP domain action tests to check results more completely
Also fixed a bug discovered as a result of the more complete testing!

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171156304
2017-10-05 11:51:29 -04:00
jianglai
373e174deb Upgrade to Truth 0.36
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171079195
2017-10-05 11:48:54 -04:00
mcilwain
302a27f0db Record a version of EPP metrics with TLD for domain commands
Also fixes the issue that dry run EPP commands were incorrectly being
reported on.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171062984
2017-10-05 11:47:31 -04:00
nickfelt
7e68ffa16a Change transfer flow tests to assert on entire TransferData contents
This CL changes the domain and contact transfer flows to check the entire
TransferData on the post-transfer resource, rather than just spot-checking
certain fields.  This approach provides much better code coverage - in
particular, it checks that the non-request flows (approve, cancel, reject)
don't modify the fields that they shouldn't be modifying, and that they do
actually clear out the transfer server-approve entities fields written by
the transfer request flow.  It's slightly orthogonal, but I also added
testing that the server-approve entities fields are actually set in the
request flows, which was previously untested.

This is pre-work for introducing an exDate-storing field into TransferData,
by making it easier to test everywhere that exDate is set *and* unset only
in the correct places.

As part of this CL, I've introduced a TransferData.copyConstantFieldsToBuilder()
method that is like asBuilder() but instead of copying all the fields to the new
builder, it only copies the logically constant ones: losing/gaining client IDs,
the request time and TRID, and transferPeriod.  This is useful both in tests but
is also used in the resolvingPendingTransfer() helper that centralizes the core
transfer resolution logic (as of []  That method has its own tests,
and in the process I removed a bunch of crufty defunct TransferData tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171053454
2017-10-05 11:44:46 -04:00
bbilbo
3584bbde53 Allow restore commit logs action in the crash environment
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171042730
2017-10-05 11:43:23 -04:00
guyben
0b5b16e97c Allow EventSample.record to accept numSamples=0
There's really no reason not to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037754
2017-10-05 11:41:59 -04:00
bbilbo
6b113603db Add nocron ear and deployer rules for the crash environment
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171027505
2017-10-04 16:16:45 -04:00
jianglai
9d8e48cf24 Refector metrics truth subject
The concrete implementation of a Metric is not of importance when asserting on the values it contains. Therefore this CL removes Metric<T> as a type parameter of AbstractMetricSubject. As a result the two implementations of the abstract subject can be used on any Metric<Long> and Metric<Distribution>, respectively.

Also migrate to Subject.Factory from deprecated SubjectFactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171012012
2017-10-04 16:16:45 -04:00
mmuller
a9ecccf672 Automated g4 rollback of changelist 170906329.
*** Reason for rollback ***

Forgot to send this for readability review.

*** Original change description ***

Add "create_cdns_tld" command to nomulus

Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

b/67367533 created to clean this up, along with...

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171011304
2017-10-04 16:16:45 -04:00
mmuller
6e84401271 Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

b/67367533 created to clean this up, along with related warts in the codebase.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170906329
2017-10-04 16:16:45 -04:00
jakubvrana
939c18bbce Remove autoescape="strict" attributes from Soy templates.
Strict autoescaping is the default so they serve no purpose.

Design doc: []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170905337
2017-10-04 16:16:45 -04:00
mcilwain
51b04f7340 Add TODO for better handling of public suffix lists for hosts
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170854701
2017-10-04 16:16:45 -04:00
guyben
01591ff88e Clarify diff display of MutatingCommand
Tools inheriting from MutatingCommand print out the change they are going to
make and then ask the user to confirm that this is indeed what they wanted to
do.

The change is outputted as a list of updated values in the form

key -> [oldValue, newValue]

e.g.

dnsPaused -> [true, false]

This CL will change the output to be clearer:

key: oldValue -> newValue

e.g.

dnsPaused: true -> false

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170853745
2017-10-04 16:16:45 -04:00
nickfelt
fd62f4a74e Streamline transfer resolving helpers in ResourceFlowUtils
In the great flow flattening, ResourceFlowUtils grew a couple nice helpers
for rebuilding transferrable resources (Domains and Contacts) upon the
resolution of a transfer - approvePendingTransfer() and denyPendingTransfer().

Most transfer-resolving callsites use one of these two helpers, but for legacy
reasons the deletion flows (DomainDeleteFlow and DeleteContactsAndHostsAction)
were instead using the "manual" resolvePendingTransfer() method or its even more
low-level createResolvedTransferData() helper instead of denyPendingTransfer().
It's simpler to just have two options - approve and deny - so this CL inlines
createResolvedTransferData() into resolvePendingTransfer() and makes the latter
a private helper for the approve/denyPendingTransfer() public helpers.

This CL also adds sanity checks that approve/denyPendingTransfer() are called
only with the logically appropriate values of TransferStatus.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170819358
2017-10-04 16:16:45 -04:00
mountford
a5c931a152 Add RDAP support for deleted contacts and registrars
This CL adds the functionality for contact and registrar searches. A future CL will handle domains and entities.

Support is also added for filtering results by registrar.

Deleted items can only be seen by admins, and by registrars viewing their own deleted items.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170734664
2017-10-04 16:16:45 -04:00
jakubvrana
6740e9270f Remove autoescape="strict" attributes from Soy templates.
Strict autoescaping is the default so they serve no purpose.

Design doc: []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170725633
2017-10-04 16:16:45 -04:00
bbilbo
7aa5629517 Allow domain transfers with 0 period and in auto-renew grace period
Normally, if a domain is in the auto-renew grace period, a transfer will cancel the auto-renew billing event. In the event of a transfer with no change to registration end date, the auto-renew billing event should not be cancelled and the gaining registrar should not be charged for the transfer.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170576726
2017-10-04 16:16:45 -04:00
mcilwain
a50ef39c04 Fix mismatch in types of Predicates being used
We're going to need to switch away from Guava's Functions and Predicates for
everything and replace them with the java.util versions. Unfortunately there
does not appear to be an automated tool to do this all at once. Refaster got
close but doesn't seem to care about these particular types of mismatch (I
suspect we're using a different version of the JDK than the outside world;
ours is OK with Guava classes).

This also bumps up Guava to 0.23, which is needed for some new functionality
used in combination with Java 8 features.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170531539
2017-10-04 16:16:45 -04:00
nickfelt
447b83f7db Fix typo in DomainTransferResponse.Builder method namename
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170529902
2017-10-04 16:16:45 -04:00
nickfelt
bc02b701b5 Use transaction time when cloneProjectedAtTime()-ing within transaction
It's bad practice in terms of data model consistency to use DateTime.now(UTC) for cloneProjectedAtTime() within a transactional context - instead, we should always use ofy().getTransactionTime() when it's available.  The ofy transaction time is frozen at the start of the transaction attempt, whereas DateTime.now(UTC) will continue marching ahead.  So using it for a cloneProjectedAtTime() could mean materializing the outcome of logical-time-dependent events (like autorenews, or transfers, or grace period expirations) at a logical time (i.e. commit log time) prior to when they should actually have occurred.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170501277
2017-10-04 16:16:45 -04:00
jianglai
dc41d7923e Remove injectable constructors in metrics library
Users of the client library should be responsible for providing the bindings required. By removing the injectable constructors and named bindings in their parameters, the client library is no longer forcing its users to provide bindings with specific named annotations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170490834
2017-10-04 16:16:45 -04:00
jianglai
ac0718079d Fix typo
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170487675
2017-10-04 16:16:45 -04:00
mcilwain
f70a7e6342 Upgrade Nomulus to Java8
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170486775
2017-10-04 16:16:45 -04:00
mcilwain
1c4e79f99e Don't allow non-active registrars to create domains or applications
Specifically, this prevents suspended registrars from creating domains or applications. Pending registrars already can't perform these actions because they get an error message when attempting to log in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170481338
2017-10-04 16:16:45 -04:00
mmuller
d09bd89629 Add config parameters to point at us to use cloud-dns staging
Add cloudDns.{rootUrl, servicePath} to allow us to point an environment at the
Cloud DNS staging API for testing.  Make sandbox and alpha point to staging.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170340859
2017-10-04 16:16:45 -04:00
larryruili
d4ab6fe90b Update activity reports to comply with agreement changes
This removes the ramp-up and pre-ramp-up registrar fields, and lists 'CZDS' as
the zfa-passwords field, in accordance to the registry agreement changes listed
on the associated bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170337426
2017-10-04 16:16:45 -04:00
mcilwain
c69b409b30 Add java8 runtime option to production appengine-web.xml files
It occurs to me that we can't have this setting different between sandbox
and production, otherwise we can end up with a situation where we push code
that works on sandbox but then fails only when it is pushed to production.
Sandbox and production need to always be set up similarly for this reason.

We'll just have to pay a greater amount of attention to the release process
next week than normal, and continue playing around in alpha for the mean
time with a fully Java 8 build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170197703
2017-10-04 16:16:45 -04:00
nickfelt
2702c3a7f3 Fix code to always set HistoryEntry modification time using txn time
Since HistoryEntry always represents a read-only log of mutation to a core resource, that mutation should always happen in a transaction, and the HistoryEntry should be saved in that transaction.  As such, it's always more accurate to use ofy().getTransactionTime() for the modificationTime of the HistoryEntry rather than just DateTime.now(UTC).

In addition, having these be the exact same timestamp makes it possible to align HistoryEntries with commit log manifests using modificationTime = transactionTime, which is useful for recovery and analysis purposes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170136957
2017-10-04 16:16:45 -04:00
jianglai
ceadfea11f Remove @Inject annotation in StackDriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170111976
2017-10-04 16:16:45 -04:00
mountford
c13c2f403a Add RDAP support for deleted nameservers and filtering by registrar
This CL adds the functionality for nameserver searches. Future CLs will handle domains and entities.

Deleted items can only be seen by admins, and by registrars viewing their own deleted items.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170106014
2017-10-04 16:16:45 -04:00
mmuller
0c8b5bc8bf Build DNS changes with HashMap instead of Builder
The existing CloudDnsWriter code uses ImmutableMap.Builder to construct the
map of DNS records to update.  This has been seen to fail on alpha, presumably
in a cases where host records and domain records produce duplicate updates for
a host.

Convert the Builder to a HashMap, allowing us to safely overwrite existing
records in the case of duplicates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170103421
2017-10-04 16:16:45 -04:00
mcilwain
8908686f23 Add java8 runtime option to all non-production appengine-web*.xml files
Java 8 is go!

https://cloudplatform.googleblog.com/2017/09/Java-8-on-App-Engine-Standard-environment-is-now-generally-available.html

We will add this option to the production files next week.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170101056
2017-10-04 16:16:45 -04:00
nickfelt
498f390e54 Make ICANN reporting URLs default to real icann.org URLs
No reason for these not to work out of the box, right?  The default-config.yaml already has a couple other real icann.org URLs for RDE testing and the TMCH CRL.  This way Nomulus users don't have to worry about changing this setting (since other than testing it's hard to imagine what other value would be used; this is literally ICANN reporting so it always goes to ICANN).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170093954
2017-10-04 16:16:45 -04:00
shikhman
f27dd70c9e Automated g4 rollback of changelist 169922096.
*** Reason for rollback ***

Rolling back to fix []
*** Original change description ***

Remove @Inject annotation in StackDriverWriter

StackDriverWriter is provided by java.google.registry.monitoring.whitebox.StackdriverModule. The @Inject annotation in its constructor is not used. It also uses outdated named binding like @Named(stackdriverGcpProject) which causes confusion because that name is not provided anywhere in the code.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169931047
2017-10-04 16:16:45 -04:00