Commit graph

403 commits

Author SHA1 Message Date
Wolfgang Meyers
28eeda189d Add map reduce job for contact import
Some additional changes were made by Ben McIlwain.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133875824
2016-09-22 14:29:20 -04:00
cgoldfeder
aed3c0f0d0 Fix nits on flows:
Rename existingResource flows variable to be specific to EPP resource type and replace some explicit checks with helper methods.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133774229
2016-09-22 14:12:34 -04:00
cgoldfeder
df70da48a2 Minor cleanups in host flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133760258
2016-09-22 14:10:56 -04:00
mcilwain
2dcac3ca68 Cut over to batched async deletion for contacts/hosts
Also consolidates the DNS refresh functionality in AsyncFlowUtils that was
being used by HostUpdateFlow into AsyncFlowEnqueuer.

TESTED=I threw together some batch scripts to create dozens of contacts on
alpha and then request their deletion, and the [] ran fine and
successfully deleted them in batches.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133714691
2016-09-22 14:08:14 -04:00
mcilwain
65ff6b45d1 Add documentation on poll messages and an outline of Code structure
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133713736
2016-09-22 14:06:32 -04:00
jianglai
77571e2063 Refactor Fee class to support EAP fee
Currently EapFee is a separate class that has no inheritance from either
BaseFee and Fee. With this CL its functionality is merged into the Fee class
and the type of the fee can be identified by the FeeType enum in the Fee class.
Future custom fees can follow the same pattern.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133627570
2016-09-22 14:03:01 -04:00
cgoldfeder
0518f63aad []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133597433
2016-09-22 14:01:29 -04:00
mcilwain
b1e1dd0f47 Remove toString() method on Result.Code and fix metrics recording
We almost certainly had intended to record a String representation of the
numeric Code all along, but a bad toString() method on the enum resulted
in the wrong thing happening.  This is more evidence of why overriding
toString() on enums is bad.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133583683
2016-09-19 12:11:44 -04:00
mountford
aa2af68af0 RDAP: Add result set sizing logic for domain name searches
Because we cannot weed out deleted domains in the query itself, the RDAP code must pull all domains with matching names, then throw out the deleted domains. So we don't know how many domains to fetch up front to fill up the desired maximum result set size. This CL adds a loop to attempt to fetch addition domains if the first fetch did not yield enough, while giving up after a while to avoid bogging down the system.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133420297
2016-09-19 12:10:14 -04:00
jianglai
743ff99ca3 Support custom effective date for fee check command
Added support to specify custom effective date to run the fee check command.
Such functionality is useful for TLDs with creation price as a function of
time, such as those with EAP. However the implementation is not limited EAP or
create price check. Any fee check can specify a date, as long as its XML schema
supports a date.

Currently conforming to fee extension v12, subject to schema changes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133420255
2016-09-19 12:08:46 -04:00
mcilwain
aa7c05cb8b Improve pretty-print diffing of Datastore entities
This removes the countless lines of the form "[null, []]" in registry_tool diffs
that are an artifact of the way we handle nulls in Objectify.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133409440
2016-09-19 12:06:54 -04:00
cgoldfeder
841be34d18 Nits in poll flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133345254
2016-09-19 12:05:08 -04:00
mcilwain
c6e9779af1 Fix Result.Code enum values to use UPPER_CAMEL naming
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133324460
2016-09-19 12:02:56 -04:00
mcilwain
823bdc721f Don't override toString() on enums
It is a bad idea to override toString() on enums to return something other than
the actual name of the enum.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133318012
2016-09-19 12:01:21 -04:00
cgoldfeder
4a723576d5 Flatten and inject the poll flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133302791
2016-09-19 11:59:48 -04:00
mcilwain
c35de8144f Improve logging/reduce shard count on async deletion []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133280141
2016-09-19 11:56:37 -04:00
cgoldfeder
949a360f46 Inject clientId into LoggedInFlow
(take two)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133206842
2016-09-19 11:51:11 -04:00
cgoldfeder
516b5663a5 Flatten the hosts flows
There's so little meat here that there's not much
reason to break this cl up any further

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133171754
2016-09-19 11:49:37 -04:00
mcilwain
00ea99960a Improve efficiency of async contact and host deletion with batching
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1.  An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once.  This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
2016-09-19 11:47:55 -04:00
cgoldfeder
75d9268ecd Resolve an old TODO by doing nothing
The RFCs are ambiguous.

5733 (contacts):

3.2.4.  EPP <transfer> Command

   ...the
   <transfer> command MUST contain a <contact:transfer> element that
   identifies the contact namespace.  The <contact:transfer> element
   contains the following child elements:

    ...

   -  A <contact:authInfo> element that contains authorization
      information associated with the contact object.

However, the xsd explicitly marks it as optional:

    <complexType name="authIDType">
      <sequence>
        <element name="id" type="eppcom:clIDType"/>
        <element name="authInfo" type="contact:authInfoType"
         minOccurs="0"/>
      </sequence>
    </complexType>

The language in 5731 (domains) is [] The only example given in both is for a transfer request, which is the one flow that obviously requires the authInfo.

We had decided that for transfer approve and reject, which are done by the losing client, requiring the authInfo is silly because it's available to that registrar from an <info> and there's no extra security in having them present it (although if they do present it we validate it). The question about cancel was whether the gaining client, which had to present the authInfo in the original transfer request, needs it again for cancel.

I can't come up with any reason this would be beneficial, and I'm making the decision: authInfo is not required on transfer cancel.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133168739
2016-09-19 11:46:14 -04:00
mcilwain
0afc83c8f9 Fix build by upgrading to latest version of AutoValue libs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133164502
2016-09-14 16:37:36 -04:00
cgoldfeder
65ef712f35 Inject targetId into contact flows
Other flows to come. This removes the need for
most of the flows to inject the command at all.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133163030
2016-09-14 16:37:36 -04:00
cgoldfeder
01e2e0141d Simplify the use of the fee extension a little
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133149148
2016-09-14 16:11:56 -04:00
cgoldfeder
1ee02108ae Crush out shared code in contact flows, especially transfer
Although the delta implies that this is actually adding code, it's
better than it looks, because some of the stuff in ContactFlowUtils
is duplicating more generic methods in ResourceFlowUtils, which
can be deleted when the domain and host flows are cut over.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133149104
2016-09-14 16:10:18 -04:00
cgoldfeder
3978b4f169 Get rid of @Nullable on the injected client id
This allows us to inject an optional once, in FlowRunner, and
inject a non-null value in the flows (not done yet, after this
goes in).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133130485
2016-09-14 16:08:38 -04:00
cgoldfeder
99af33328d Flatten the contact flows
There was very little meat in the contact hierarchy and it
flattened quiet easily.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133080191
2016-09-14 16:07:02 -04:00
cgoldfeder
bf9a3a0fb2 Add some more configs to ConfigModule and provide it in flow tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133079669
2016-09-14 16:05:24 -04:00
cgoldfeder
4f320232b1 Add some common functions to ResourceFlowUtils to support flat flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133079286
2016-09-14 16:03:52 -04:00
ctingue
75203918a9 Add command for creating LRP tokens
Command allows for both one-off creation and bulk import of assignees via file (the latter will be used for the initial import from Play Store).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133048360
2016-09-14 16:02:08 -04:00
mountford
1a050554fe Move flags extension exceptions to separate classes
The exceptions created for generic problems with the flags extension (invalid flag, etc.) should be in a common location, so they can be used by all interested TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133040831
2016-09-14 15:59:10 -04:00
cgoldfeder
efd3424849 Change to FlowModule to support the new flat flows
This factors out a huge chunk of boilerplate that
would otherwise be in every single flow.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133014837
2016-09-14 15:57:31 -04:00
cgoldfeder
04fd14995e Add a missing test to ContactUpdateFlow
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133000632
2016-09-14 15:55:52 -04:00
cgoldfeder
68cdd04124 Duplicate EppExceptions outside of the flow hierarchy
By duplicating rather than moving them, I can keep both versions
around while I port the flows over to the new flat form.

I could have made these g4 moves from the original sources, but
there's barely anything in these files and it didn't really seem
like it was adding anything useful.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132999387
2016-09-14 15:54:15 -04:00
shikhman
cee08d48f2 Remove unused dns-cron queue
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884314
2016-09-14 15:44:28 -04:00
cgoldfeder
b9b2829f7c Log the class names of unimplemented extensions before
throwing the generic user-visible error.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884249
2016-09-14 15:42:59 -04:00
cgoldfeder
d7443f2eee Fix copy/paste javadoc erroc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132875203
2016-09-14 15:41:32 -04:00
nickfelt
9dffd64dc7 Cleanup minor things in whitebox metrics code
Specifically:
- remove @RequestScope from EppMetric since it's only for components
- fix to call the better overload of toBigqueryTimestamp
- use the same UUID provider for BigQueryMetricsEnqueuer that already exists for
  the VerifyEntityIntegrityStreamer
- minor cleanup in VerifyEntityIntegrityStreamer (inject projectId vs whole env)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132721794
2016-09-14 15:39:51 -04:00
nickfelt
2537e95de5 Change EppMetric.Builder to use @AutoValue.Builder
Getting rid of builder boilerplate makes my heart sing.  Since we can no
longer @Inject the Builder() constructor, this change adds a provider
in WhiteboxModule that calls a special builderForRequest() factory method,
which gets passed a request ID and Clock and preserves the existing
EppMetric magic that sets the start and end time for you.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132714432
2016-09-14 15:38:22 -04:00
nickfelt
ceb5c2117e Decouple GaeUserCredentials from UserService and simplify tests
This disentangles GaeUserCredentials and UserService, which lets us remove a
bunch of hacky and brittle code from LoginFlowViaConsoleTest.

Previously, GaeUserCredentials was constructed for a user, but then was still
directly calling UserService to check if the user was an admin.  UserService
can be adjusted in tests (via AppEngineRule / LocalServiceTestHelper) but it's
a pain, especially to do dynamically within a single test file.  The hacky
code in LoginFlowViaConsoleTest was working around that restriction.

With this CL, you can pass into GaeUserCredentials whether the user is an
admin or not (for testing) or construct one directly from a UserService object
(for production, and for convenience in tests using an AppEngineRule user).

Note that I also changed EppConsoleAction to @Inject UserService.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132696391
2016-09-14 15:36:53 -04:00
shikhman
bd887e857e Fix incorrect field name in EppMetric
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132688882
2016-09-14 15:35:25 -04:00
nickfelt
cb55ab4f5f Convert straggler GAE modules to basic-scaling
This fixes the proximate cause of b/31380927 and makes alpha frontend usable
again.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132675121
2016-09-14 15:33:57 -04:00
shikhman
aa4ca42cdd Add EPP metrics to flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132591518
2016-09-14 15:32:27 -04:00
nickfelt
e478fd09fb Remove straggling JarKeyring link and sort MOE-added imports
Followups from []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132564245
2016-09-14 15:29:30 -04:00
mountford
a63921350b HistoryEntry for extra logic; update fee check
While working on an implementation of TLD-specific logic, it was realized that the extra logic methods would need access to the flow's HistoryEntry, so that things like poll messages could be parented properly.

Also, the update flow had not been fixed to perform the fee check.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132561527
2016-09-14 15:28:03 -04:00
mountford
95cc7ab3d8 Add extra logic for all relevant flows
This CL enhances various domain flows (check, create, delete, renew, restore, transfer, update) so that they invoke the appropriate methods on the object implementing the TLD's RegistryExtraFlowLogic (if any). TldSpecificLogicProxy is also updated to invoke RegistryExtraFlowLogic proxy (if any) to fetch the appropriate price. The tests use a made-up extra flow logic object which can be attached to a test TLD to make sure that the proper routines are being invoked.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132486734
2016-09-14 15:26:36 -04:00
mcilwain
a6db24c8bb Fix NullPointerException in group syncing
We were trying to set the scopes too early in the creation of a
GoogleCredential.  They aren't yet known at this point in the code and should be
an empty collection (and also not null, which yields an NPE).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132477868
2016-09-14 15:25:07 -04:00
mcilwain
2b42118aaf Make boolean operator precedence unambiguous in ClaimsListShard
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132442796
2016-09-14 15:19:15 -04:00
mcilwain
8cfa643598 Switch Optional.ofNullable() to Optional.fromNullable() in EppMetric
The latter is the canonical way to call it; the former is not available in
public Guava (and is just a pass-through to fromNullable anyway).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132441956
2016-09-14 15:17:49 -04:00
mcilwain
cadf9d4af2 Use smaller shard size in ClaimsListShardTest
The default production value of 10,000 was unnecessarily large for testing
purposes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132441792
2016-09-14 15:16:22 -04:00
shikhman
4652688585 Add default DistributionFitter implementation to EventMetric
This DistributionFitter is suitable for tracking the latency of network calls.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132400538
2016-09-07 12:20:46 -04:00