Adding it to one test is sufficient because we use the same loading logic across all flows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239506003
The most common issues were:
* Arrays.asList() shouldn't be called with a single parameter.
* Broken Javadoc @links.
* Unnecessary casts and type declarations.
* Unnecessary unused variable initializations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230994311
This also deletes the associated commands and domain application specific
entities.
We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.
Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
The main goal of this CL is to make the clock constructor injectable (so that tests do not need to use InjectRule to override the clock). The complication is that the clock is used by two static caches (ROOT_CACHE directly and CRL_CACHE indirectly). The clock is not actually used to construct the lock, but rather to verify that the root certificate is within its validity period.
For ROOT_CACHE we move the verification to its call sites. This adds a bit overhead because the validity check happens every time the cache is called, not just when the cache is built or refreshed. However this check is rather cheap. Also the resources are included in the jar and the cache is valid for 1 year. Given that we deploy every week, there's not much point making it an expiring cache rather than a static map.
For CRL_CACHE we change the key to a tuple of TmchCaMode and X509Certificate. The certificate is no longer provided from the ROOT_CACHE directly and must be verified before it is provided as a cache key. We left the CRL verification inside the cache loader because it (signature verification) is more expensive compared to simple expiration check, and we do not want to do this every time the cache is called.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218385684
These are simply too costly in their current form now that we are handling double-digit QPS, so at a minimum we'd want to refactor these for batched exports using a background thread (like how Stackdriver metrics work). However, upon further review, that work isn't worth doing if this BigQuery table isn't actually being used for anything, and it seems that we aren't using it anymore given that ICANN transaction reporting no longer requires it.
So the simplest thing to do is simply to get rid of this entirely, and just use a combination of Stackdriver metrics and App Engine logs. The eppMetrics BigQuery table is ~1.2 billion rows and takes up 223 GB, so that's not an insignificant GCP billings saving if we can delete it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215905466
In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.
This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
This also incorporates general improvements and additions to the existing EPP
lifecycle tests around domain deletion. As a refresher: There is a 5 day
add grace period (AGP) following domain creation. Domains that are deleted
during that period have their create costs (but not EAP costs) refunded. This
deletion takes place immediately. Refunds are implemented by issuing a
Cancellation for the associated OneTime billing event.
Domains that are deleted after AGP ends first go through a 30 day redemption
grace period followed by a 5 day pending deletion period. No create fees are
refunded in this case.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179597874
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.
Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
The scheme is:
- loadBytes: returns a ByteSource of the data
- loadFile: returns a string using UTF8 encoding, optionally applying
substitutions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177606406
There is a big mix of different "load Resources" from different libraries
depending on where you were and what type of resource you want. Now there is a
clear hirarchy:
ResourceUtils:
for use in actual (non-test) code, reads a file from a context directory
TestDataHelper (uses ResourceUtils):
for use in tests, reads a file from a context directory + "/testdata". Also
caches the resource so calling it multiple times with the same file will
not read the file multiple times.
Library specific helpers (e.g. ToolsTestData) (uses TestDataHelper):
for use in that library's tests only, reads from a specific testdata directory.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177027533
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
Memcache is already off but now it's not in the code anymore.
This includes removing domain creation failfast, since that is actually
slower now than just running the flow - all you gain is a non-transactional
read over a transactional read, but the cost is that you always pay that
read, which is going to drive up latency.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158183506
No tests break because of this, but some tests that were previously
intentionally clearing memcache (for example, the test in
DomainInfoFlow that counts datastore calls) are now doing something
that is not a no-op anymore.
Also fix an incorrect comment about memcache in DatastoreHelper.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154087182
This wasn't being recorded correctly because the clientId is only
set in LoginFlow after the flow succeeds, whereas we were
previously logging the clientId before executing the flow. This
adds special handling for LoginFlow. Note that we only set the
metric label to the clientId for valid registrar logins, to ensure
that metric cardinality doesn't grow unbounded (as it might if we
used every arbitrary string passed in as an attempted login).
This also refactors creation and handling of FakesAndMocksModule so
as to be able to make test assertions about EPP metrics from
integration flow tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154048280
This fixes recording of number of attempts and command name on EPP
flows, which was broken because a separate metric builder was
being injected in two places, EppController and FlowRunner, with the
one injected into FlowRunner being discarded rather than having changes
applied to the same instance as in EppController.
This also adds a test that the metric is created successfully inside
a flow. Note that tests already exist for EppController to ensure that
the metric is recorded correctly.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152306596
This also bypasses signed mark validation during domain creation if
the flow is being executed as superuser.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145435268
Also more narrowly scopes a catch block in TmchCertificateAuthority.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144744847
Now that the flows are flattened, the commitAdditionalLogicChanges() call, which used to come later in the flow to actually save the Datastore objects, is now happening right after the performAdditionalXXXLogic() call. So we can instead just do the saves in performAdditionalXXXLogic(), and get rid of the separate call. As a first step, this CL simply makes commitAdditionalLogicChanges() a private method that gets called internally by the extra logic manager. Later, we can move the saves into their proper position, affecting only the extra logic class itself.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137529991
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.
Our best practices doc says to use Keys not Refs because:
* The .get() method obscures what's actually going on
- Much harder to visually audit the code for datastore loads
- Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
* Implicit ofy().load() offers much less control
- Antipattern for ultimate goal of making Ofy injectable
- Can't control cache use or batch loading without making ofy() explicit anyway
* Serialization behavior is surprising and could be quite dangerous/incorrect
- Can lead to serialization errors. If it actually worked "as intended",
it would lead to a Ref<> on a serialized object being replaced upon
deserialization with a stale copy of the old value, which could potentially
break all kinds of transactional expectations
* Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
- E.g. helper methods all need to have Ref and Key overloads, or you need to
call .key() to get the Key<T> for every Ref<T> you want to pass in
- Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
create directly from kind+ID/name, raw Key, websafe key string, etc.
(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)
The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
This CL adds the hooks necessary to implement TLD-specific flow info and update flow logic. Usage of the hooks follows in a separate CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=130108832
We want to support multiple versions of the fee extension, to allow new features while maintaining backward compatibility. This CL extends the framework and adds one new version, 0.11 (spec version 7), to the existing version 0.6 (spec version 3). A follow-on CL will add version 0.12 (spec version 8).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=127849044
This is needed because IRDB contains information on Android apps that
aren't in the Play Store, and differentiates them by production
certificate SHA256 hash fingerprint. So we need to know the value of
this hash in order to ask IRDB about links to/from a specific app,
which we can fortunately get by querying the Play Store.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125566907
Daggerizes all of the EPP flows. This does not change anything yet
about the flows themselves, just how they are invoked, but after
this CL it's safe to @Inject things into flow classes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125382478
Despite the comment, DomainAllocateFlow is absolutely registered in
FlowPicker. It gets picked if there's a domain create epp command that
also specifies the allocate extension. Remove the explicit setting of
flowClass, and remove two tests that now fail because DomainCreateFlow
gets loaded - which is the desired behavior.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125339191
The "SessionSource" has nothing to do with sessions (and it's often
used in sessionless contexts). What it does indicate is the endpoint
used to make the request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125295224
Superuser should only be settable via the tool (see []
which is merged in here but not diffbased, and which removes
the implicit superuser for CharlestonRoad). It is a property
of the request, not of the session (there are no sessions in the tool).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125204707
dryRun is only available via the (sessionless!) tool, and is not
a property of the session.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125203026
TransportCredentials are per-request, not per-session, and
there's no reason to carry them within SessionMetadata.
While I'm in here, get rid of "null" credentials.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=125202213
