It's better that it always takes a non-null ImmutableSet, which may either be
empty or contain elements. That way the ugliness of nullness is contained
just to the entity class itself, and all other code that interacts with it
can always be assured of having a real set to deal with.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132066238
This is one of a series of CLs which will refactor EppMetrics into a value type
and Metrics into a stateless class which will have an export(EppMetrics requestDetails)
method to export EPP metrics in a stateless way. Once EppMetrics is a value
type, I will create a new StackdriverEppMetrics that will also accept the value
type via an incrementRequests(EppMetrics requestDetails), allowing us to
monitor EPP via BigQuery and Stackdriver with minimum code duplication.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131973288
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.
Our best practices doc says to use Keys not Refs because:
* The .get() method obscures what's actually going on
- Much harder to visually audit the code for datastore loads
- Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
* Implicit ofy().load() offers much less control
- Antipattern for ultimate goal of making Ofy injectable
- Can't control cache use or batch loading without making ofy() explicit anyway
* Serialization behavior is surprising and could be quite dangerous/incorrect
- Can lead to serialization errors. If it actually worked "as intended",
it would lead to a Ref<> on a serialized object being replaced upon
deserialization with a stale copy of the old value, which could potentially
break all kinds of transactional expectations
* Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
- E.g. helper methods all need to have Ref and Key overloads, or you need to
call .key() to get the Key<T> for every Ref<T> you want to pass in
- Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
create directly from kind+ID/name, raw Key, websafe key string, etc.
(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)
The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
It seems the Bazel sandbox has become stricter at HEAD (which is
AWESOME) which caused this error to materialize from the shadows:
9) testExpiredTransfer_subordinateHost(google.registry.model.host.HostResourceTest)
java.net.UnknownHostException: sandbox: sandbox: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at google.registry.model.host.HostResourceTest.setUp(HostResourceTest.java:82)
...
Caused by: java.net.UnknownHostException: sandbox: unknown error
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130700624
This CL contains the non-open source extra flow logic for EPP update commands.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130427820
See b/30806813 for more context. Copied from there: This appears to be happening if we get an EPP domain create command that is missing any contacts (but has a registrant; with no registrant we exercise a different codepath). In that case, JAXB leaves the contacts field on the Create null, and we try to pass it into Sets.union() as a result of Corey's refactoring in [] that changed contact loading to load the contacts and registrant all at once. The fix is just to apply nullToEmpty() first.
Note that it's always an error to try to create a domain without any non-registrant contacts, but that's supposed to happen later on in BaseDomainCreateFlow.verifyCreateIsAllowed() via validateRequiredContactsPresent(), which will produce a nice error message.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130309019
This CL adds the hooks necessary to implement TLD-specific flow info and update flow logic. Usage of the hooks follows in a separate CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130108832
This fixes a significant memory consumption issue when running tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129482832
For the .app discounting logic, we need a new extension which will let
registrars set, clear and query custom flags on a domain. Hopefully
this will be reusable for other custom TLDs later. This CL adds the
XSD, the associated classes for marshalling and unmarshalling, and some
marshalling tests, and links the classes into the system-wide extension
lists.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128178999
We want to support multiple versions of the fee extension, to allow new features while maintaining backward compatibility. This CL extends the framework and adds one new version, 0.11 (spec version 7), to the existing version 0.6 (spec version 3). A follow-on CL will add version 0.12 (spec version 8).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127849044
The presubmits are warning that toUpperCase() and toLowerCase() are locale-specific, and advise using Ascii.toUpperCase() and Ascii.toLowerCase() as a local-invariant alternative.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127583677
This adds a home in our opensource repo for python libraries and binaries,
under a top-level "python" directory. Future CLs will relocate ICANN
reporting bits and pieces to new homes under this directory, and will use
the MOE configuration and python_directory_import rule defined here.
This approach is roughly modeled on the protobuf Bazel opensource project,
which also uses a top-level directory for various languages, and also uses
the "imports" parameter to exclude that directory in python module names:
https://github.com/google/protobuf/blob/v3.0.0-beta-3/BUILD#L568
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127459882
If a TLD has a whitelist on nameservers, domains in such TLD must have
at least one nameserver. Therefore creating domains with empty nameserver
is forbidden, as well as deleting the last nameserver on a domain. We
enforce this policy by checking the number of nameservers for the new resource
to makesure it is not zero if a whitelist exists.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127318320
When updating a filed other than the registrant for a domain, a null
registratantContactId is passed to the registrant whitelist validator, causing
an excpetion because null is not an element in the whitelist (if it exists).
Added logic to handle null registrantContactId.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=126817310
We've been using the very fragile newline-delimited legacy logging
statement in FlowRunner for ICANN reporting for a long time. While
this is bad in a few ways, the worst is that the parsing of this
logging statement is extremely fragile (e.g. adding/removing fields
can easily break the parsing). This is in fact part of what broke the
ExportLogsServlet parsing last fall ([] and forced us to
recover by manually parsing the log statement (and its XML) in
BigQuery. It also broke again in [] where we were relying
on matching the logging classname, since matching on 'EPP Command'
was considered insufficiently narrow.
This introduces a new JSON-format logging statement to FlowRunner
that fixes both of these problems:
1) it replaces the newline-delimited "format" with a JSON-based
format, so that we can add new fields much more easily and
reliably support logging more structured data
2) it replaces the short 'EPP Command' signature with a much more
targeted 'EPP-REPORTING-LOG-SIGNATURE' signature so that we can
use that alone for matching, rather than relying on the class
name in the log message
What this doesn't fix is the fact that we still need to parse the
XML in BigQuery; we should fix this by logging the parts of the XML
that ICANN reporting needs explicitly, but that'll be a subsequent
change, since while the existing approach is gross, it's actually
much less fragile than just matching the log statement itself.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125902976
This changelist makes the newly added v1_2_0 the default and fixes incompatibilities
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125609347
This is needed because IRDB contains information on Android apps that
aren't in the Play Store, and differentiates them by production
certificate SHA256 hash fingerprint. So we need to know the value of
this hash in order to ask IRDB about links to/from a specific app,
which we can fortunately get by querying the Play Store.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125566907
Daggerizes all of the EPP flows. This does not change anything yet
about the flows themselves, just how they are invoked, but after
this CL it's safe to @Inject things into flow classes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125382478
Despite the comment, DomainAllocateFlow is absolutely registered in
FlowPicker. It gets picked if there's a domain create epp command that
also specifies the allocate extension. Remove the explicit setting of
flowClass, and remove two tests that now fail because DomainCreateFlow
gets loaded - which is the desired behavior.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125339191
The "SessionSource" has nothing to do with sessions (and it's often
used in sessionless contexts). What it does indicate is the endpoint
used to make the request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125295224
Superuser should only be settable via the tool (see []
which is merged in here but not diffbased, and which removes
the implicit superuser for CharlestonRoad). It is a property
of the request, not of the session (there are no sessions in the tool).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125204707
dryRun is only available via the (sessionless!) tool, and is not
a property of the session.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125203026
TransportCredentials are per-request, not per-session, and
there's no reason to carry them within SessionMetadata.
While I'm in here, get rid of "null" credentials.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125202213
This introduces Actions and Dagger up until FlowRunner. The changes
to the servlets are relatively simple, but the required changes to
the tests, as well as to auxillary EPP endpoints (such as the http
check api and the load test servlet) were vast. I've added some
comments in critique to make the review easier that don't really
make sense as in-code comments for the future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=124593546
This cleans up some of the tests, and helps with
future injection CLs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=124208164
For domain checks in GA and quiet period, show domains with pending applications as unavailable.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=124134193
Before this CL, each contact and host was independently
loaded via the ReferenceUnion adapter. Since fields are
processed serially by JAXB, this means worst-case there
were 17 loads, best case 3 (the 3 required contacts) and
usual case 5-6 (some hosts). This CL reduces that to 1
datastore roundtrip in all cases.
A side effect of this CL is the further hollowing-out of
ReferenceUnion, since it no longer is involved in
marshalling at all.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=123712842
This completes the command extensions for the regType 0.2 extension.
Up next will be the response extensions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=123322887
Note that it doesn't do anything yet beyond basic XML validation
because the default registry system doesn't use registration types,
but this serves as a template for the other domain commands using
registration types and provides a method that TLDs implementing custom
logic can use. This also explicitly doesn't yet handle the response
extensions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=123245388
This is so we can associate history records with all mutations when doing
database maintenance.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=123209304
This also renames the existing FlowRegistry to FlowPicker to avoid
overloaded uses of the word "registry". Absent this renaming, the new
package would've been google.registry.flows.registry, which gives
entirely the wrong impression as it makes it sound like the home for
flows that affect TLDs.
This is a preparatory CL for adding flow picker engines that will
allow customized flows to run on a per-TLD basis.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=122671260
RFC5731 leaves the registrant field as optional. We
currently don't show it on un-authed info, but we do
show it in WHOIS, so there's no strong reason not to
show it here. By doing so, we maintain the invariant
that registrant must always be present on a
DomainResource object.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=122645356
