Also consolidates the DNS refresh functionality in AsyncFlowUtils that was
being used by HostUpdateFlow into AsyncFlowEnqueuer.
TESTED=I threw together some batch scripts to create dozens of contacts on
alpha and then request their deletion, and the [] ran fine and
successfully deleted them in batches.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133714691
Currently EapFee is a separate class that has no inheritance from either
BaseFee and Fee. With this CL its functionality is merged into the Fee class
and the type of the fee can be identified by the FeeType enum in the Fee class.
Future custom fees can follow the same pattern.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133627570
Added support to specify custom effective date to run the fee check command.
Such functionality is useful for TLDs with creation price as a function of
time, such as those with EAP. However the implementation is not limited EAP or
create price check. Any fee check can specify a date, as long as its XML schema
supports a date.
Currently conforming to fee extension v12, subject to schema changes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133420255
There's so little meat here that there's not much
reason to break this cl up any further
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133171754
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1. An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once. This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
This allows us to inject an optional once, in FlowRunner, and
inject a non-null value in the flows (not done yet, after this
goes in).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133130485
There was very little meat in the contact hierarchy and it
flattened quiet easily.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133080191
Getting rid of builder boilerplate makes my heart sing. Since we can no
longer @Inject the Builder() constructor, this change adds a provider
in WhiteboxModule that calls a special builderForRequest() factory method,
which gets passed a request ID and Clock and preserves the existing
EppMetric magic that sets the start and end time for you.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132714432
This disentangles GaeUserCredentials and UserService, which lets us remove a
bunch of hacky and brittle code from LoginFlowViaConsoleTest.
Previously, GaeUserCredentials was constructed for a user, but then was still
directly calling UserService to check if the user was an admin. UserService
can be adjusted in tests (via AppEngineRule / LocalServiceTestHelper) but it's
a pain, especially to do dynamically within a single test file. The hacky
code in LoginFlowViaConsoleTest was working around that restriction.
With this CL, you can pass into GaeUserCredentials whether the user is an
admin or not (for testing) or construct one directly from a UserService object
(for production, and for convenience in tests using an AppEngineRule user).
Note that I also changed EppConsoleAction to @Inject UserService.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132696391
Followup to [] Mocking shouldn't be used when you can use the real
implementation just as easily (and more robustly) - in particular, you almost
never want to mock a value type.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132586826
While working on an implementation of TLD-specific logic, it was realized that the extra logic methods would need access to the flow's HistoryEntry, so that things like poll messages could be parented properly.
Also, the update flow had not been fixed to perform the fee check.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132561527
This CL enhances various domain flows (check, create, delete, renew, restore, transfer, update) so that they invoke the appropriate methods on the object implementing the TLD's RegistryExtraFlowLogic (if any). TldSpecificLogicProxy is also updated to invoke RegistryExtraFlowLogic proxy (if any) to fetch the appropriate price. The tests use a made-up extra flow logic object which can be attached to a test TLD to make sure that the proper routines are being invoked.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132486734
This change refactors EppMetrics from the mutable self-exporting thing that it
was into a real value type EppMetric, and delegates exporting functionality to the
BigQueryMetricsEnqueuer.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132387660
When EAP is involed we current have one billing event for domain create that
has the create fee and EAP fee lumped together. Change it to record two
separate billing events for each.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132335349
It's better that it always takes a non-null ImmutableSet, which may either be
empty or contain elements. That way the ugliness of nullness is contained
just to the entity class itself, and all other code that interacts with it
can always be assured of having a real set to deal with.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132066238
This is one of a series of CLs which will refactor EppMetrics into a value type
and Metrics into a stateless class which will have an export(EppMetrics requestDetails)
method to export EPP metrics in a stateless way. Once EppMetrics is a value
type, I will create a new StackdriverEppMetrics that will also accept the value
type via an incrementRequests(EppMetrics requestDetails), allowing us to
monitor EPP via BigQuery and Stackdriver with minimum code duplication.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131973288
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.
Our best practices doc says to use Keys not Refs because:
* The .get() method obscures what's actually going on
- Much harder to visually audit the code for datastore loads
- Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
* Implicit ofy().load() offers much less control
- Antipattern for ultimate goal of making Ofy injectable
- Can't control cache use or batch loading without making ofy() explicit anyway
* Serialization behavior is surprising and could be quite dangerous/incorrect
- Can lead to serialization errors. If it actually worked "as intended",
it would lead to a Ref<> on a serialized object being replaced upon
deserialization with a stale copy of the old value, which could potentially
break all kinds of transactional expectations
* Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
- E.g. helper methods all need to have Ref and Key overloads, or you need to
call .key() to get the Key<T> for every Ref<T> you want to pass in
- Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
create directly from kind+ID/name, raw Key, websafe key string, etc.
(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)
The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
It seems the Bazel sandbox has become stricter at HEAD (which is
AWESOME) which caused this error to materialize from the shadows:
9) testExpiredTransfer_subordinateHost(google.registry.model.host.HostResourceTest)
java.net.UnknownHostException: sandbox: sandbox: unknown error
at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
at google.registry.model.host.HostResourceTest.setUp(HostResourceTest.java:82)
...
Caused by: java.net.UnknownHostException: sandbox: unknown error
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getLocalHost(InetAddress.java:1500)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130700624
This CL contains the non-open source extra flow logic for EPP update commands.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130427820
See b/30806813 for more context. Copied from there: This appears to be happening if we get an EPP domain create command that is missing any contacts (but has a registrant; with no registrant we exercise a different codepath). In that case, JAXB leaves the contacts field on the Create null, and we try to pass it into Sets.union() as a result of Corey's refactoring in [] that changed contact loading to load the contacts and registrant all at once. The fix is just to apply nullToEmpty() first.
Note that it's always an error to try to create a domain without any non-registrant contacts, but that's supposed to happen later on in BaseDomainCreateFlow.verifyCreateIsAllowed() via validateRequiredContactsPresent(), which will produce a nice error message.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130309019
This CL adds the hooks necessary to implement TLD-specific flow info and update flow logic. Usage of the hooks follows in a separate CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130108832
This fixes a significant memory consumption issue when running tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129482832
For the .app discounting logic, we need a new extension which will let
registrars set, clear and query custom flags on a domain. Hopefully
this will be reusable for other custom TLDs later. This CL adds the
XSD, the associated classes for marshalling and unmarshalling, and some
marshalling tests, and links the classes into the system-wide extension
lists.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=128178999
We want to support multiple versions of the fee extension, to allow new features while maintaining backward compatibility. This CL extends the framework and adds one new version, 0.11 (spec version 7), to the existing version 0.6 (spec version 3). A follow-on CL will add version 0.12 (spec version 8).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127849044
The presubmits are warning that toUpperCase() and toLowerCase() are locale-specific, and advise using Ascii.toUpperCase() and Ascii.toLowerCase() as a local-invariant alternative.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127583677
This adds a home in our opensource repo for python libraries and binaries,
under a top-level "python" directory. Future CLs will relocate ICANN
reporting bits and pieces to new homes under this directory, and will use
the MOE configuration and python_directory_import rule defined here.
This approach is roughly modeled on the protobuf Bazel opensource project,
which also uses a top-level directory for various languages, and also uses
the "imports" parameter to exclude that directory in python module names:
https://github.com/google/protobuf/blob/v3.0.0-beta-3/BUILD#L568
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127459882
If a TLD has a whitelist on nameservers, domains in such TLD must have
at least one nameserver. Therefore creating domains with empty nameserver
is forbidden, as well as deleting the last nameserver on a domain. We
enforce this policy by checking the number of nameservers for the new resource
to makesure it is not zero if a whitelist exists.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127318320
When updating a filed other than the registrant for a domain, a null
registratantContactId is passed to the registrant whitelist validator, causing
an excpetion because null is not an element in the whitelist (if it exists).
Added logic to handle null registrantContactId.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=126817310
We've been using the very fragile newline-delimited legacy logging
statement in FlowRunner for ICANN reporting for a long time. While
this is bad in a few ways, the worst is that the parsing of this
logging statement is extremely fragile (e.g. adding/removing fields
can easily break the parsing). This is in fact part of what broke the
ExportLogsServlet parsing last fall ([] and forced us to
recover by manually parsing the log statement (and its XML) in
BigQuery. It also broke again in [] where we were relying
on matching the logging classname, since matching on 'EPP Command'
was considered insufficiently narrow.
This introduces a new JSON-format logging statement to FlowRunner
that fixes both of these problems:
1) it replaces the newline-delimited "format" with a JSON-based
format, so that we can add new fields much more easily and
reliably support logging more structured data
2) it replaces the short 'EPP Command' signature with a much more
targeted 'EPP-REPORTING-LOG-SIGNATURE' signature so that we can
use that alone for matching, rather than relying on the class
name in the log message
What this doesn't fix is the fact that we still need to parse the
XML in BigQuery; we should fix this by logging the parts of the XML
that ICANN reporting needs explicitly, but that'll be a subsequent
change, since while the existing approach is gross, it's actually
much less fragile than just matching the log statement itself.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125902976
This changelist makes the newly added v1_2_0 the default and fixes incompatibilities
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125609347
This is needed because IRDB contains information on Android apps that
aren't in the Play Store, and differentiates them by production
certificate SHA256 hash fingerprint. So we need to know the value of
this hash in order to ask IRDB about links to/from a specific app,
which we can fortunately get by querying the Play Store.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125566907
Daggerizes all of the EPP flows. This does not change anything yet
about the flows themselves, just how they are invoked, but after
this CL it's safe to @Inject things into flow classes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125382478
Despite the comment, DomainAllocateFlow is absolutely registered in
FlowPicker. It gets picked if there's a domain create epp command that
also specifies the allocate extension. Remove the explicit setting of
flowClass, and remove two tests that now fail because DomainCreateFlow
gets loaded - which is the desired behavior.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125339191
The "SessionSource" has nothing to do with sessions (and it's often
used in sessionless contexts). What it does indicate is the endpoint
used to make the request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125295224
