When we set the payload, it is converted to bytes using the response's
character set. Changing the MIME type later has no effect on the conversion
to bytes, even though it does change the returned MIME type. This results in
bytes that were encoded using one character set while the response reports a
different character set.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196258752
The 'referralUrl' Datastore field is filled with mostly junk data, whereas
'url' contains real registrar web addresses. This makes the long needed fix to
display the proper url in WHOIS.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194398003
Per EPP RFC 5730, the <clTRID> element is optional. However, we weren't handling
it not being specified in asynchronous contact/host deletions because we were
adding it directly as a parameter value on a task, which does not allow null and
thus threw a NullPointerException.
This fixes handling for nulls (the parameter isn't set at all) and adds a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194123259
TestDataHelper is build exactly to prevent direct reads of resources. It caches
the resources and makes sure they are in the correct directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191785004
This is in preparation for automatic EPP consolidation.
The assertion will now look like:
assertThatCommand("file.xml"[, substitution])
.atTime(time)
.hasResponse("file2.xml"[, substitution]);
Also, added convenience functions for often reused commands (login and logout)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190617505
We are no longer using Eclipse internally and therefore stopped maintaining
stuff related to it. We cannot guarantee that any pertinent information remains correct
and relevant in the future.
Users are advised to use IntelliJ (Community Edition is fine) with Bazel plugin
if they want IDE support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189586127
With https://github.com/bazelbuild/bazel/issues/4376, bazel 0.10.0 now supports accessing system TMPDIR in its sandbox. Use this instead of hardcoding /tmp in BUILD rules to get around the gpg-agent path length restriction.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=186010932
The intention (from []is:
- actualAsString() is the method that people call.
- actualCustomStringRepresentation() is the method that people override.
Fortunately, no one actually calls actualCustomStringRepresentation(), aside from some tests that call it to test a subject's implementation. That's easy enough to work around by extracting a method.
(Arguably @ForOverride should permit calls from tests in some cases (now that Error Prone knows how to identify test code). But it's not entirely clear, since, e.g., people shouldn't be testing Converter.doForward(null) because the method can never be invoked that way. Some discussion here: []Tested:
global TAP
[]
RELNOTES=Marked `actualCustomStringRepresentation()` as `@ForOverride`. To retrieve the string representation, call `actualAsString()`.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185672328
The task-queue API only allows reading 1000 tasks at a time, hence the original reason for this limit. We get over that limit by reading (and processing) items from the queue in a loop - 1000 at a time.
This is important because the 1000 dns-updates are shared among all TLDs,
meaning that a TLD with >1000 waiting updates can affect the update latency of
other TLDs.
In addition, partially fixes the bug where if there are more than 1000 updates to paused
/ non-existing TLDs, we completely block all updated to all TLDs.
By partially fixed, I mean "if we have around 1000 updates to paused TLDs, we will read them every time ReadDnsUpdates is called, ignore then, and only then get to the actual updates we want to process".
This works for a number of 1000 updates waiting - but if paused TLDs have tens or hundreds of thousands of updates waiting - this might still choke up other TLDs (not to mention we keep reading / updating 10s or 100s of thousands of tasks in the queue, that's... bad.)
A more thorough fix will come in a future CL, as it requires a more thorough change in the code.
Note that the queue lease command supports a maximum of 10 QPS. Any more than
that - and we get errors / empty results. Hence we limit our QPS to 9 to be on
the safe side.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185218684
This fixes up the following problems:
1. Using string concatenation instead of the formatting variant methods.
2. Logging or swallowing exception messages without logging the exception
itself (this swallows the stack trace).
3. Unnecessary logging on re-thrown exceptions.
4. Unnecessary use of formatting variant methods when not necessary.
5. Complicated logging statements involving significant processing not being
wrapped inside of a logging level check.
6. Redundant logging both of an exception itself and its message (this is
unnecessary duplication).
7. Use of the base Logger class instead of our FormattingLogger class.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182419837
This creates a specified number of tokens of a given schema, with a dryrun option
to not persist them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181403775
ArrayList is more performant and there's no reason to use a LinkedList here.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179717525
In Truth8, we can do assertThat(stream) directly. It's less verbose and clearer
in most cases.
Note that for the "finishers" (e.g. "containsExactyElementsIn") - streams are
still not allowed. So when there is:
assertThat(stream.map(someTransformation).collect(toList()))
.containsExactlyElementsIn(expecteStream.map(someTransformation).collect(toList()));
I kept the .collect in the assertThat to preserve the symmetry with the
finisher.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179697587
This also incorporates general improvements and additions to the existing EPP
lifecycle tests around domain deletion. As a refresher: There is a 5 day
add grace period (AGP) following domain creation. Domains that are deleted
during that period have their create costs (but not EAP costs) refunded. This
deletion takes place immediately. Refunds are implemented by issuing a
Cancellation for the associated OneTime billing event.
Domains that are deleted after AGP ends first go through a 30 day redemption
grace period followed by a 5 day pending deletion period. No create fees are
refunded in this case.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179597874
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.
Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
This fixes a bug where collections of incompatible types are being tested for equality to each other (e.g.: Set<Foo> equals Set<Integer> should never return true unless both sets are empty, a bit of a vacuous assertion).
This change is necessary to unblock future improvements to the static analysis capabilities of the java compiler.
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178798071
The only remaining methods on ExceptionRule after this are methods that
also exist on ExpectedException, which will allow us to, in the next CL,
swap out the one for the other and then run the automated refactoring to
turn it all into assertThrows/expectThrows.
Note that there were some assertions about root causes that couldn't
easily be turned into ExpectedException invocations, so I simply
converted them directly to usages of assertThrows/expectThrows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178623431
We were running FOSS tests on an old version of Ubuntu (14.04) which comes with a rather old gpg version 1.4.16 compared to the current version 2.1.22 (https://lists.gnupg.org/pipermail/gnupg-announce/2017q3/000411.html). As a result some default settings have changed between these versions, leading to test failures when tests are run on newer platforms. In this CL several of the settings are made explicit, no longer depending on default values, which makes them work on either platform.
1. "--no-mdc-error" is set. We do not have mdc integrity protection for the test keys, which results in a non-zero return value for newer versions of GPG. Setting this flag makes return value zero again.
2. "--keyid-format" long is set. GPG key IDs are the last 16 (long key id) or 8 (short key id) octets of the key fingerprint (https://security.stackexchange.com/questions/84280/short-openpgp-key-ids-are-insecure-how-to-configure-gnupg-to-use-long-key-ids-i). Older version uses the short id as default, whereas newer versions defaults to long id. Also change the expected key ID to 16 bytes accordingly.
3. Output stderr in GpgSystemCommandRule when failure occurs during key import. This help debug key import failure due to too long gpg root path. Note: this failure itself is not fixed and still would occur on newer Debian or macOS systems.
4. Set gpg root folder permission correctly to 700, otherwise the newer version gpg will return non-zero value. Previously used method set it to 755.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178163297
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
The scheme is:
- loadBytes: returns a ByteSource of the data
- loadFile: returns a string using UTF8 encoding, optionally applying
substitutions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177606406
This removes some qualifiers that aren't necessary (e.g. public/abstract on interfaces, private on enum constructors, final on private methods, static on nested interfaces/enums), uses Java 8 lambdas and features where that's an improvement
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177182945
There is a big mix of different "load Resources" from different libraries
depending on where you were and what type of resource you want. Now there is a
clear hirarchy:
ResourceUtils:
for use in actual (non-test) code, reads a file from a context directory
TestDataHelper (uses ResourceUtils):
for use in tests, reads a file from a context directory + "/testdata". Also
caches the resource so calling it multiple times with the same file will
not read the file multiple times.
Library specific helpers (e.g. ToolsTestData) (uses TestDataHelper):
for use in that library's tests only, reads from a specific testdata directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177027533
This command is used by registry operators to apply registry locks to
domain names.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176549874
Also changed truth.SubjectFactory to truth.Subject.Factory (plain renaming) and use method reference instead of anonymous class to create the factory when applicable.
FailureMetadata, an opaque object to its users, is introduced to replace FailureStrategy in in custom Subject in order to resolve some existing flaws of FailureStrategy as well as enable new features to be added to Truth.
New API is available in Truth-0.36, if there is a build/pom.xml, it's also updated to use this version.
More information: See []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176110788
JUnit 4.13 isn't released yet, but these functions are essential to being
able to write good test assertions about thrown exceptions. Rather than
not using them until JUnit 4.13 comes out (which might be awhile, as JUnit
4.12 came out almost three years ago), we're making the same decision that
Google made internally, which is to backport them. Indeed, the only reason
this commit is necessary is to fix breakage in the Nomulus build, as the
existing code worked fine internally where the backports are already in
place.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173435579
I could've sworn we were already doing this, but apparently not? Anyway,
ROID suffixes have a number of requirements on them that weren't being
enforced, so this enforces them. All existing production data is compliant
with these requirements; the only existing bad data we have is in alpha and
sandbox.
ROID suffixes are now required to match the regex ^[A-Z0-9_]{1,8}$
See also https://tools.ietf.org/html/rfc5730
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173400001
We no longer find contacts by name if the request is not authorized to see the name.
Several changes cascade from this. Previously, the code assumed that deleted contacts might still have full names, and therefore be searchable. This is not possible in all cases, because Datastore doesn't have the right index to find deleted contacts by name with a matching registrar. However, luckily, this situation can never occur, because contacts always have their name fields nulled out when they are deleted.
So instead, we simply ignore deleted records when searching by name, knowing that none can ever match.
The tests were then changed so that deleted records look the way the really will, meaning devoid of personal information.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172776926
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
This CL adds transferredRegistrationExpirationTime as a TransferData field
persisted to Datastore. It's only relevant for domains, and it represents the
registration expiration time resulting from the approval of the most recent
transfer request. For pending transfers, we assume the transfer will be
server-approved, and thus in DomainTransferRequestFlow we set this field to the
existing computed value serverApproveNewExpirationTime, which is what we use
for setting up the server-approve autorenew billing event and poll message.
In DomainTransferApproveFlow we overwrite this field with the freshly computed
newExpirationTime, whereas in DomainTransferCancel/RejectFlow (and in the
implicit cancel of DomainDeleteFlow during a pending transfer) we null it out.
There are two key benefits to having this field, which are described in more
detail in b/36405140.
1) b/25084229 - it allows storage of a frozen value to back the "exDate" field
of DomainTransferResponse, which we can use to fix various errors with how
exDate display currently works.
2) b/36354434 - it allows DomainResource.cloneProjectedAtTime() to just directly
set the registrationExpirationTime to this value, without computing it de
novo, which reduces duplicated logic and ensures that the new expiration time
matches the autorenew child objects.
This CL only starts writing the field on TransferData as persisted directly on
the DomainResource itself. We'll then want to backfill the field for at
least pending transfers, whether expired or not (so we can do (2) above), but
I think we might as well backfill it for all pending and approved transfers
so that we also fix (1) even for historical transfers. And then we can start
actually reading the field for both purposes. (Note that for (1), this will
only fix synchronous transfer responses served via DomainTransferQueryFlow,
not async transfer responses served via poll messages, since these have already
been persisted with a potentially bad exDate, but I don't think it's worth a
backfill for those).
One last naming note: I chose the verbose transferredRegistrationExpirationTime
rather than the extendedRegistrationExpirationTime of DomainTransferResponse
because (as is the case in autorenew grace, or for a superuser transfer) the
new registration time isn't necessarily extended at all; it may be the same as
the pre-transfer expiration time. Also, including "registration" helps clarify
w.r.t. pendingTransferExpirationTime which refers confusingly to the expiry of
the transfer itself, rather than the domain registration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171858083
Also fixed a bug discovered as a result of the more complete testing!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171156304
Also fixes the issue that dry run EPP commands were incorrectly being
reported on.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171062984
This CL changes the domain and contact transfer flows to check the entire
TransferData on the post-transfer resource, rather than just spot-checking
certain fields. This approach provides much better code coverage - in
particular, it checks that the non-request flows (approve, cancel, reject)
don't modify the fields that they shouldn't be modifying, and that they do
actually clear out the transfer server-approve entities fields written by
the transfer request flow. It's slightly orthogonal, but I also added
testing that the server-approve entities fields are actually set in the
request flows, which was previously untested.
This is pre-work for introducing an exDate-storing field into TransferData,
by making it easier to test everywhere that exDate is set *and* unset only
in the correct places.
As part of this CL, I've introduced a TransferData.copyConstantFieldsToBuilder()
method that is like asBuilder() but instead of copying all the fields to the new
builder, it only copies the logically constant ones: losing/gaining client IDs,
the request time and TRID, and transferPeriod. This is useful both in tests but
is also used in the resolvingPendingTransfer() helper that centralizes the core
transfer resolution logic (as of [] That method has its own tests,
and in the process I removed a bunch of crufty defunct TransferData tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171053454
This CL adds the functionality for contact and registrar searches. A future CL will handle domains and entities.
Support is also added for filtering results by registrar.
Deleted items can only be seen by admins, and by registrars viewing their own deleted items.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170734664
This migration does not change any behavior.
More information: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168248439
This CL changes the RDAP responses. When the requester asks for information about a domain, and is not logged in as the owning registrar, no contact information is shown. When the requester asks for information about a contact, and is not logged in as the owner registrar, the existence of the contact is shown, but not any personal data (the existence is shown to make things easier to test).
The login uses the same functionality as the registrar console.
For the most part, this CL does not include the necessary tests to make sure that data is not returned when not logged in. The CL is so large that I didn't want to burden it further. Those tests will be added in a follow-on CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168022034
We create an injectable LockHandler that just calls the static
Lock.executeWithLocks function.
I'm not sure what's the correct place to put the LockHandler. I think
model/server is only appropriate for the actual datastore lock. This is a "per request" lock, so maybe request/lock?
-----------------------------
This is the initial step in adding the "lock implicitly released on request death" feature, but it's also useful on its own - easier to test Actions when we can use a fake lock.
To keep this CL simple, we keep using the old Lock as is in most places. We just choose a single example to convert to LockHandler to showcase it. Converting all other uses will be in a subsequent CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167357564
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).
This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
Also updates Truth version to 0.34 where the replacement method was added.
More information: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161970305
