Commit graph

290 commits

Author SHA1 Message Date
mcilwain
00ea99960a Improve efficiency of async contact and host deletion with batching
This allows handling of N asynchronous deletion requests simultaneously instead
of just 1.  An accumulation pull queue is used for deletion requests, and the
async deletion [] is now fired off whenever that pull queue isn't empty,
and processes many tasks at once.  This doesn't particularly take more time,
because the bulk of the cost of the async delete operation is simply iterating
over all DomainBases (which has to happen regardless of how many contacts and
hosts are being deleted).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133169336
2016-09-19 11:47:55 -04:00
cgoldfeder
75d9268ecd Resolve an old TODO by doing nothing
The RFCs are ambiguous.

5733 (contacts):

3.2.4.  EPP <transfer> Command

   ...the
   <transfer> command MUST contain a <contact:transfer> element that
   identifies the contact namespace.  The <contact:transfer> element
   contains the following child elements:

    ...

   -  A <contact:authInfo> element that contains authorization
      information associated with the contact object.

However, the xsd explicitly marks it as optional:

    <complexType name="authIDType">
      <sequence>
        <element name="id" type="eppcom:clIDType"/>
        <element name="authInfo" type="contact:authInfoType"
         minOccurs="0"/>
      </sequence>
    </complexType>

The language in 5731 (domains) is [] The only example given in both is for a transfer request, which is the one flow that obviously requires the authInfo.

We had decided that for transfer approve and reject, which are done by the losing client, requiring the authInfo is silly because it's available to that registrar from an <info> and there's no extra security in having them present it (although if they do present it we validate it). The question about cancel was whether the gaining client, which had to present the authInfo in the original transfer request, needs it again for cancel.

I can't come up with any reason this would be beneficial, and I'm making the decision: authInfo is not required on transfer cancel.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133168739
2016-09-19 11:46:14 -04:00
mcilwain
0afc83c8f9 Fix build by upgrading to latest version of AutoValue libs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133164502
2016-09-14 16:37:36 -04:00
cgoldfeder
65ef712f35 Inject targetId into contact flows
Other flows to come. This removes the need for
most of the flows to inject the command at all.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133163030
2016-09-14 16:37:36 -04:00
cgoldfeder
01e2e0141d Simplify the use of the fee extension a little
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133149148
2016-09-14 16:11:56 -04:00
cgoldfeder
1ee02108ae Crush out shared code in contact flows, especially transfer
Although the delta implies that this is actually adding code, it's
better than it looks, because some of the stuff in ContactFlowUtils
is duplicating more generic methods in ResourceFlowUtils, which
can be deleted when the domain and host flows are cut over.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133149104
2016-09-14 16:10:18 -04:00
cgoldfeder
3978b4f169 Get rid of @Nullable on the injected client id
This allows us to inject an optional once, in FlowRunner, and
inject a non-null value in the flows (not done yet, after this
goes in).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133130485
2016-09-14 16:08:38 -04:00
cgoldfeder
99af33328d Flatten the contact flows
There was very little meat in the contact hierarchy and it
flattened quiet easily.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133080191
2016-09-14 16:07:02 -04:00
cgoldfeder
bf9a3a0fb2 Add some more configs to ConfigModule and provide it in flow tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133079669
2016-09-14 16:05:24 -04:00
cgoldfeder
4f320232b1 Add some common functions to ResourceFlowUtils to support flat flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133079286
2016-09-14 16:03:52 -04:00
ctingue
75203918a9 Add command for creating LRP tokens
Command allows for both one-off creation and bulk import of assignees via file (the latter will be used for the initial import from Play Store).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133048360
2016-09-14 16:02:08 -04:00
mountford
1a050554fe Move flags extension exceptions to separate classes
The exceptions created for generic problems with the flags extension (invalid flag, etc.) should be in a common location, so they can be used by all interested TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133040831
2016-09-14 15:59:10 -04:00
cgoldfeder
efd3424849 Change to FlowModule to support the new flat flows
This factors out a huge chunk of boilerplate that
would otherwise be in every single flow.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133014837
2016-09-14 15:57:31 -04:00
cgoldfeder
04fd14995e Add a missing test to ContactUpdateFlow
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133000632
2016-09-14 15:55:52 -04:00
cgoldfeder
68cdd04124 Duplicate EppExceptions outside of the flow hierarchy
By duplicating rather than moving them, I can keep both versions
around while I port the flows over to the new flat form.

I could have made these g4 moves from the original sources, but
there's barely anything in these files and it didn't really seem
like it was adding anything useful.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132999387
2016-09-14 15:54:15 -04:00
shikhman
cee08d48f2 Remove unused dns-cron queue
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884314
2016-09-14 15:44:28 -04:00
cgoldfeder
b9b2829f7c Log the class names of unimplemented extensions before
throwing the generic user-visible error.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884249
2016-09-14 15:42:59 -04:00
cgoldfeder
d7443f2eee Fix copy/paste javadoc erroc
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132875203
2016-09-14 15:41:32 -04:00
nickfelt
9dffd64dc7 Cleanup minor things in whitebox metrics code
Specifically:
- remove @RequestScope from EppMetric since it's only for components
- fix to call the better overload of toBigqueryTimestamp
- use the same UUID provider for BigQueryMetricsEnqueuer that already exists for
  the VerifyEntityIntegrityStreamer
- minor cleanup in VerifyEntityIntegrityStreamer (inject projectId vs whole env)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132721794
2016-09-14 15:39:51 -04:00
nickfelt
2537e95de5 Change EppMetric.Builder to use @AutoValue.Builder
Getting rid of builder boilerplate makes my heart sing.  Since we can no
longer @Inject the Builder() constructor, this change adds a provider
in WhiteboxModule that calls a special builderForRequest() factory method,
which gets passed a request ID and Clock and preserves the existing
EppMetric magic that sets the start and end time for you.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132714432
2016-09-14 15:38:22 -04:00
nickfelt
ceb5c2117e Decouple GaeUserCredentials from UserService and simplify tests
This disentangles GaeUserCredentials and UserService, which lets us remove a
bunch of hacky and brittle code from LoginFlowViaConsoleTest.

Previously, GaeUserCredentials was constructed for a user, but then was still
directly calling UserService to check if the user was an admin.  UserService
can be adjusted in tests (via AppEngineRule / LocalServiceTestHelper) but it's
a pain, especially to do dynamically within a single test file.  The hacky
code in LoginFlowViaConsoleTest was working around that restriction.

With this CL, you can pass into GaeUserCredentials whether the user is an
admin or not (for testing) or construct one directly from a UserService object
(for production, and for convenience in tests using an AppEngineRule user).

Note that I also changed EppConsoleAction to @Inject UserService.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132696391
2016-09-14 15:36:53 -04:00
shikhman
bd887e857e Fix incorrect field name in EppMetric
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132688882
2016-09-14 15:35:25 -04:00
nickfelt
cb55ab4f5f Convert straggler GAE modules to basic-scaling
This fixes the proximate cause of b/31380927 and makes alpha frontend usable
again.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132675121
2016-09-14 15:33:57 -04:00
shikhman
aa4ca42cdd Add EPP metrics to flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132591518
2016-09-14 15:32:27 -04:00
nickfelt
e478fd09fb Remove straggling JarKeyring link and sort MOE-added imports
Followups from []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132564245
2016-09-14 15:29:30 -04:00
mountford
a63921350b HistoryEntry for extra logic; update fee check
While working on an implementation of TLD-specific logic, it was realized that the extra logic methods would need access to the flow's HistoryEntry, so that things like poll messages could be parented properly.

Also, the update flow had not been fixed to perform the fee check.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132561527
2016-09-14 15:28:03 -04:00
mountford
95cc7ab3d8 Add extra logic for all relevant flows
This CL enhances various domain flows (check, create, delete, renew, restore, transfer, update) so that they invoke the appropriate methods on the object implementing the TLD's RegistryExtraFlowLogic (if any). TldSpecificLogicProxy is also updated to invoke RegistryExtraFlowLogic proxy (if any) to fetch the appropriate price. The tests use a made-up extra flow logic object which can be attached to a test TLD to make sure that the proper routines are being invoked.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132486734
2016-09-14 15:26:36 -04:00
mcilwain
a6db24c8bb Fix NullPointerException in group syncing
We were trying to set the scopes too early in the creation of a
GoogleCredential.  They aren't yet known at this point in the code and should be
an empty collection (and also not null, which yields an NPE).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132477868
2016-09-14 15:25:07 -04:00
mcilwain
2b42118aaf Make boolean operator precedence unambiguous in ClaimsListShard
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132442796
2016-09-14 15:19:15 -04:00
mcilwain
8cfa643598 Switch Optional.ofNullable() to Optional.fromNullable() in EppMetric
The latter is the canonical way to call it; the former is not available in
public Guava (and is just a pass-through to fromNullable anyway).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132441956
2016-09-14 15:17:49 -04:00
mcilwain
cadf9d4af2 Use smaller shard size in ClaimsListShardTest
The default production value of 10,000 was unnecessarily large for testing
purposes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132441792
2016-09-14 15:16:22 -04:00
shikhman
4652688585 Add default DistributionFitter implementation to EventMetric
This DistributionFitter is suitable for tracking the latency of network calls.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132400538
2016-09-07 12:20:46 -04:00
shikhman
42a39b0ddc Refactor EppMetrics into the EppMetric value type
This change refactors EppMetrics from the mutable self-exporting thing that it
was into a real value type EppMetric, and delegates exporting functionality to the
BigQueryMetricsEnqueuer.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132387660
2016-09-07 12:19:22 -04:00
jart
b77ebd1df9 Don't use modules to instantiate classes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132380251
2016-09-07 12:17:58 -04:00
mcilwain
3b4b7a475b Replace VoidKeyring with InMemoryKeyring and a dummy module
VoidKeyring always threw exceptions whenever any of its methods were called,
which caused several parts of the system to fail early (and thus required a
proper Keyring to be implemented almost immediately, early on in the "just
playing around with the system" phase).

I'm swapping this out with an InMemoryKeyring which is supplied by
DummyKeyringModule, which, instead of throwing exceptions, returns dummy
values, delaying the onset of errors to when connecting to external services
is attempted.  This pushes off the required implementation of a real Keyring-
providing module, allowing the system to be played around with more first.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132361157
2016-09-07 12:15:08 -04:00
mcilwain
fea8c96f79 Remove GoogleCredential method call that isn't publicly released yet
It turns out that we don't actually need it for the system to function normally.
When the next version of the OAuth2 API is released and this is included, I'll
add the call back in.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132359891
2016-09-07 12:12:16 -04:00
shikhman
e5a0392994 Make StackdriverWriter#flush() return early when attempting to flush empty
timeseries

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132359115
2016-09-07 12:10:49 -04:00
shikhman
0e8d9e3859 Force endTimestamp to be strictly greater than or equal to startTimestamp
This fixes an error that we've been seeing from Stackdriver.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132358790
2016-09-07 12:09:23 -04:00
ctingue
daca7d65c2 Allow PasswordGenerator to use different alphabets
Per mcilwain's suggestion in the LRP design doc, LRP tokens should use a Base58 alphabet. I'll move PasswordGenerator out of the tools package and into a utils class in a future CL, as we'll want to use this generator in the LrpToken class itself rather than relegate the token definition to a tool.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132358363
2016-09-07 12:07:58 -04:00
shikhman
dbb9ef80c5 Refactor ConcurrentHashMap creation code to a helper method
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132352263
2016-09-07 12:06:26 -04:00
shikhman
0538c1e210 Add method to create and register EventMetrics
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132346375
2016-09-07 12:03:28 -04:00
shikhman
2ba9b01a13 Add the EventMetric metric type
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132345599
2016-09-07 12:01:58 -04:00
jianglai
6641f105b7 Create a separate billing event when EAP is applied
When EAP is involed we current have one billing event for domain create that
has the create fee and EAP fee lumped together. Change it to record two
separate billing events for each.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132335349
2016-09-07 12:00:28 -04:00
shikhman
969d9483ae Add encoding for the Stackdriver Distribution value type to StackdriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132327732
2016-09-07 11:58:58 -04:00
ctingue
6f241c5605 Index redemptionHistoryEntry on LrpToken
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132323443
2016-09-07 11:57:30 -04:00
shikhman
93bf68039b Add lock.unlock() to finally clause in Counter
It is good practice to unlock a lock in a finally clause.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132302413
2016-09-07 11:56:00 -04:00
shikhman
dcb189943b Add the Distribution data type for instrumentation
This is one of a series of CLs adding a new metric type, EventMetric, which
is used for tracking numerical distributions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132103552
2016-09-07 11:54:26 -04:00
shikhman
180240ae04 Wire in MetricsReporter to FrontendServlet
This work is identical to the work done for BackendServlet in
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132100448
2016-09-07 11:52:59 -04:00
ctingue
c75ab37dab Export LrpToken to BigQuery
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132083818
2016-09-07 11:51:28 -04:00
mcilwain
07135f6190 Don't allow null in BillingEvent.setFlags()
It's better that it always takes a non-null ImmutableSet, which may either be
empty or contain elements.  That way the ugliness of nullness is contained
just to the entity class itself, and all other code that interacts with it
can always be assured of having a real set to deal with.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132066238
2016-09-02 13:57:50 -04:00