Commit graph

221 commits

Author SHA1 Message Date
mcilwain
c41c4dbbdc Fix alphabetical ordering in FOSS schema
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132460165
2016-09-14 15:22:06 -04:00
mcilwain
cadf9d4af2 Use smaller shard size in ClaimsListShardTest
The default production value of 10,000 was unnecessarily large for testing
purposes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132441792
2016-09-14 15:16:22 -04:00
shikhman
42a39b0ddc Refactor EppMetrics into the EppMetric value type
This change refactors EppMetrics from the mutable self-exporting thing that it
was into a real value type EppMetric, and delegates exporting functionality to the
BigQueryMetricsEnqueuer.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132387660
2016-09-07 12:19:22 -04:00
mcilwain
697098602e Mark VerifyEntityIntegrityActionTest as a medium test
It was timing out occasionally in the Bazel build on less-powerful hardware.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132378555
2016-09-07 12:16:35 -04:00
mcilwain
27dfe358ce Shard all [] tests
This prevents error messages about not enough tests existing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132360344
2016-09-07 12:13:42 -04:00
ctingue
daca7d65c2 Allow PasswordGenerator to use different alphabets
Per mcilwain's suggestion in the LRP design doc, LRP tokens should use a Base58 alphabet. I'll move PasswordGenerator out of the tools package and into a utils class in a future CL, as we'll want to use this generator in the LrpToken class itself rather than relegate the token definition to a tool.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132358363
2016-09-07 12:07:58 -04:00
shikhman
0538c1e210 Add method to create and register EventMetrics
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132346375
2016-09-07 12:03:28 -04:00
shikhman
2ba9b01a13 Add the EventMetric metric type
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132345599
2016-09-07 12:01:58 -04:00
jianglai
6641f105b7 Create a separate billing event when EAP is applied
When EAP is involed we current have one billing event for domain create that
has the create fee and EAP fee lumped together. Change it to record two
separate billing events for each.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132335349
2016-09-07 12:00:28 -04:00
shikhman
969d9483ae Add encoding for the Stackdriver Distribution value type to StackdriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132327732
2016-09-07 11:58:58 -04:00
ctingue
6f241c5605 Index redemptionHistoryEntry on LrpToken
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132323443
2016-09-07 11:57:30 -04:00
shikhman
dcb189943b Add the Distribution data type for instrumentation
This is one of a series of CLs adding a new metric type, EventMetric, which
is used for tracking numerical distributions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132103552
2016-09-07 11:54:26 -04:00
shikhman
180240ae04 Wire in MetricsReporter to FrontendServlet
This work is identical to the work done for BackendServlet in
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132100448
2016-09-07 11:52:59 -04:00
mcilwain
07135f6190 Don't allow null in BillingEvent.setFlags()
It's better that it always takes a non-null ImmutableSet, which may either be
empty or contain elements.  That way the ugliness of nullness is contained
just to the entity class itself, and all other code that interacts with it
can always be assured of having a real set to deal with.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132066238
2016-09-02 13:57:50 -04:00
shikhman
5ee78c505d Add metric related unit tests to EppControllerTest and FlowRunnerTest
This is one of a series of CLs which will refactor EppMetrics into a value type
and Metrics into a stateless class which will have an export(EppMetrics requestDetails)
method to export EPP metrics in a stateless way. Once EppMetrics is a value
type, I will create a new StackdriverEppMetrics that will also accept the value
type via an incrementRequests(EppMetrics requestDetails), allowing us to
monitor EPP via BigQuery and Stackdriver with minimum code duplication.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131973288
2016-09-02 13:51:48 -04:00
cgoldfeder
5098b03af4 DeReference the codebase
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.

Our best practices doc says to use Keys not Refs because:
 * The .get() method obscures what's actually going on
   - Much harder to visually audit the code for datastore loads
   - Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
 * Implicit ofy().load() offers much less control
   - Antipattern for ultimate goal of making Ofy injectable
   - Can't control cache use or batch loading without making ofy() explicit anyway
 * Serialization behavior is surprising and could be quite dangerous/incorrect
   - Can lead to serialization errors. If it actually worked "as intended",
     it would lead to a Ref<> on a serialized object being replaced upon
     deserialization with a stale copy of the old value, which could potentially
     break all kinds of transactional expectations
 * Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
   - E.g. helper methods all need to have Ref and Key overloads, or you need to
     call .key() to get the Key<T> for every Ref<T> you want to pass in
   - Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
     overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
     create directly from kind+ID/name, raw Key, websafe key string, etc.

(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)

The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
2016-09-02 13:50:20 -04:00
shikhman
c11ac3129f Factor out labelValue length check to abstract base class and small name fix
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131902964
2016-09-02 13:47:21 -04:00
cgoldfeder
8059ab5c90 A @DoNotHydrate annotation for toHydratedString
This is needed for a soon-to-be-submitted CL that changes
all Refs to Keys and therefore removes the logic in
toHydratedString that doesn't expand Keys. We need to be
able to tag types as unexpandable to avoid cycles. It
would be better to tag fields, not types, but that is a
much harder change and not currently needed by the use
case of the following CL, so for now this suffices.

While I am in here, add tests for all of the features
of toHydratedString.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131618634
2016-08-30 14:15:16 -04:00
shikhman
d77dced024 Fix NPE in StackdriverWriter
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131603566
2016-08-30 14:13:50 -04:00
ctingue
942dc58251 Add valid TLD check to LrpToken validation
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131438274
2016-08-30 14:10:54 -04:00
mcilwain
1b3f77a468 Add more tests to external code repo
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131406104
2016-08-30 14:03:24 -04:00
shikhman
1c5b4e16c6 Add metrics to PublishDnsUpdatesAction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131190862
2016-08-26 09:43:30 -04:00
mcilwain
8f0f701ff7 Standardize test methods to be prefixed with "test"
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131176866
2016-08-26 09:41:52 -04:00
mcilwain
1482e413e0 Remove redundant run commmands in CreateDomainCommandTest
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130954171
2016-08-22 14:47:18 -04:00
mcilwain
965cf6edc3 Add registry_tool host_create command
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130953679
2016-08-22 14:45:54 -04:00
nickfelt
6915e35800 Remove unused redirectIfNotLoggedIn() method
It's superseded by RequestHandler's processing of @Action(requireLogin = true), and is no longer used anywhere:
[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130788873
2016-08-22 14:41:41 -04:00
cgoldfeder
5556becd0e Remove duplicate fields hiding the same objects on a parent class
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130778688
2016-08-22 14:37:22 -04:00
jart
f73e1c06b1 Remove InetAddress.getLocalHost() from tests
It seems the Bazel sandbox has become stricter at HEAD (which is
AWESOME) which caused this error to materialize from the shadows:

9) testExpiredTransfer_subordinateHost(google.registry.model.host.HostResourceTest)
java.net.UnknownHostException: sandbox: sandbox: unknown error
  at java.net.InetAddress.getLocalHost(InetAddress.java:1505)
  at google.registry.model.host.HostResourceTest.setUp(HostResourceTest.java:82)
  ...
Caused by: java.net.UnknownHostException: sandbox: unknown error
  at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
  at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
  at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
  at java.net.InetAddress.getLocalHost(InetAddress.java:1500)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130700624
2016-08-22 14:31:45 -04:00
ctingue
1894b2308b Handle LRP tokens in flows
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130679951
2016-08-22 14:30:19 -04:00
cgoldfeder
5ff8b9377c Remove unused request parameter extractor
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130672143
2016-08-22 14:24:44 -04:00
shikhman
b45c2ca2ee Wire in MetricReporter into the backend servlet to publish metrics
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130468842
2016-08-22 14:07:45 -04:00
mountford
4a34807b1d RDAP: Use IANA identifier as the registrar handle
According to the ICAAN operation profile:

2.1.7. Registries MUST support lookup for entities with the registrar role within other objects using the handle (as described in 3.1.5 of RFC7482). The handle of the entity with the registrar role MUST be equal to IANA Registrar ID. The entity with the registrar role in the RDAP response MUST contain a publicIDs member to identify the IANA Registrar ID from the IANA’s Registrar ID registry. The type value of the publicID object MUST be equal to IANA Registrar ID.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130452501
2016-08-22 14:06:07 -04:00
mountford
cfe4eb0fbb Implement extra logic for update flow
This CL contains the non-open source extra flow logic for EPP update commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130427820
2016-08-22 13:58:56 -04:00
mcilwain
467286d79e Remove registrar certificate state dead code
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130418304
2016-08-22 13:55:54 -04:00
mountford
0e794c790a Add additional methods for manipulating timed transition maps
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130311938
2016-08-15 17:13:29 -04:00
nickfelt
d1ea3e3a68 Fix NPE bug in DomainCommand.cloneAndLinkReferences()
See b/30806813 for more context.  Copied from there:  This appears to be happening if we get an EPP domain create command that is missing any contacts (but has a registrant; with no registrant we exercise a different codepath).  In that case, JAXB leaves the contacts field on the Create null, and we try to pass it into Sets.union() as a result of Corey's refactoring in [] that changed contact loading to load the contacts and registrant all at once.  The fix is just to apply nullToEmpty() first.

Note that it's always an error to try to create a domain without any non-registrant contacts, but that's supposed to happen later on in BaseDomainCreateFlow.verifyCreateIsAllowed() via validateRequiredContactsPresent(), which will produce a nice error message.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130309019
2016-08-15 17:13:29 -04:00
mountford
160266f37a RDAP: Implement entity name search
Adds the ability to search for entities (contacts and registrars) by name.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130305930
2016-08-15 17:13:29 -04:00
mountford
19da9a1531 RDAP: Add "last update of RDAP database" event
The ICANN operational profile says:

1.4.12. An entity object within an RDAP response MUST contain an events
member with the following events:
o An event of eventAction type registration.
o An event of eventAction type last changed. The event of eventAction type last changed MUST be omitted if the Contact Object (as defined in RFC5733) has not been updated since it was created.
o An event of eventAction type last update of RDAP database.

It has similar wording for domains and hosts. The registration and last changed events are already being generated. This CL adds a "last update of RDAP database" event for all three object types. It's very redundant, and I have warned ICANN that "last update" doesn't mean what they might expect in the context of an eventually consistent database, but there we are.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130279688
2016-08-15 17:01:29 -04:00
mcilwain
3efe09616c Allow two-letter domain names
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130139005
2016-08-15 16:57:19 -04:00
jianglai
5f26196993 Add create_domain command to gtech_tool
Support creating domain with gtech_tool, instead of creating temporary epp file
and run execute_epp manually. Also changes CreateContactCommand for consistency.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130127280
2016-08-15 16:53:01 -04:00
mountford
0066a03709 Add extra flow logic hooks for info and update
This CL adds the hooks necessary to implement TLD-specific flow info and update flow logic. Usage of the hooks follows in a separate CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130108832
2016-08-15 16:51:26 -04:00
mountford
0cab13e6d8 RDAP: Remove port43 element from JSON output
According to Gustavo Lozano of ICANN: "In the case of the RDAP profile (gTLD space), the “port43” element is not expected to be used, because Whois/43 tcp will be deprecated in the future." So it sounds like we should not include the port43 element for the moment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130017966
2016-08-15 16:42:06 -04:00
ctingue
2e88129c43 Swap id and indexed field in LrpToken
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130014294
2016-08-15 16:39:03 -04:00
ctingue
159528fb6f Add LrpToken to schema-foss
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129758267
2016-08-15 16:35:42 -04:00
Ben McIlwain
92fa0d6781 Reduce mapreduce shards in tests
This fixes a significant memory consumption issue when running tests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129482832
2016-08-05 20:41:53 -04:00
Chris Tingue
1ef8716177 Add LRP TLD states to Registry and *_tld tools
Also had to add an EnumParameter class to support
List<T extends Enum<T>>, as these aren't natively supported by
JCommander (although single Enum parameters are.)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129464699
2016-08-05 20:41:11 -04:00
Greg Shikhman
a620d06239 Add dagger map for injecting DnsWriter implementations
This is one of several CLs in a sequence for allowing per-TLD DNS
implementations.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129445641
2016-08-05 20:38:21 -04:00
Ben McIlwain
770fd35e20 Add LrpToken to export kinds
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129444295
2016-08-05 20:38:04 -04:00
mountford
eb792e89ef RDAP: Document RDAP guidance
Updates the RDAP code to reflect guidance and confirmation received from various mailing lists.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129254894
2016-08-03 17:15:02 -04:00
mcilwain
ea24f6ca31 Use string keys for the multimap of pricing engines for TLDs
This is better than the previous way of using the canonical name of the class,
because the previous way did not allow for refactoring, and also required the
PremiumPricingEngine to live in the model package lest there be circular
dependencies, which does not seem ideal.

Note that, for reasons of backwards compatibility with existing persisted data,
the name of the static premium pricing engine has been set to its canonical
class name, but the class can now be refactored going forward so long as this
string remains unchanged, and any new pricing engine implementations can use
whatever string key they want (it doesn't have to be a canonical class name).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129215185
2016-08-03 11:16:01 -04:00