This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.
Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
The only remaining methods on ExceptionRule after this are methods that
also exist on ExpectedException, which will allow us to, in the next CL,
swap out the one for the other and then run the automated refactoring to
turn it all into assertThrows/expectThrows.
Note that there were some assertions about root causes that couldn't
easily be turned into ExpectedException invocations, so I simply
converted them directly to usages of assertThrows/expectThrows.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178623431
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
This originally started as a small change, but quickly grew into a major refactor as I realized the original parameter structure wasn't conducive to a cron task and manual re-runs.
The changes are as follows:
1. Adds DNS metrics to activity reports, thanks to Nick's work with the Zoneman Dremel -> #plx workflow.
2. Surrounds registrar names in transactions reports with quotes, to escape possible commas.
3. Factors out the report generation logic into IcannReportingStager.
4. Assigns default values to the three main parameters
- yearMonth defaults to the previous month
- subdir defaults to "icann/monthly/yearMonth", i.e. "gs://domain-registry-reporting/icann/monthly/yyyy-MM"
- reportType defaults to both reports
5. Adds "Total" row generation logic to transactions reports
- This was a previously overlooked requirement.
6. Adds "MANIFEST.txt" generation and upload logic.
- The MANIFEST lists out which files need to be uploaded in the subdirectory.
7. Increases urlfetch timeout from 5s to 10s in backend tasks.
- Backend tasks should be more latency tolerant anyway, and this reduces the number of incorrect timeouts we see for services like Bigquery which might take some time to respond.
TESTED=Extensive testing in alpha, and ran FOSS test.
TODO: send out an e-mail for report generation and upload, and add reporting to cron.xml
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=172738344
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
Sometimes requests "die" suddenly, without going through catch/finally blocks.
If this happens, any lock they own will remain locked until it times out (which
can take hours in some cases).
This cl implicitly unlocks any lock if the owner of the lock isn't running
anymore.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168880938
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
The affected actions have been changed to check that the user is logged in by [] so this attribute is no longer needed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159572365
The code to authenticate and authorize incoming requests (including via OAuth) has been in the system. This CL actually turns it on, since we are satisfied from logging information that it is not unjustly denying access.
Auth settings are also updated on a few commands missed earlier.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152381820
This is the second step of migrating to our new XSRF token format. The
first step ([] made validate() start accepting new tokens
(basically, dual-read). This step cuts over our "writing" to write the
new token format. The third and final step will drop support for
validating the old token format (back to single-read). We'll do that
in a subsequent push so that we don't invalidate all the current XSRF
tokens that people might have in their browsers.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149790648
A test has been added to RequestHandlerTest, making sure that, while we merely log errors for the time being, the correct dummy AuthResult is being created.
Most actions use the default settings, which have been changed to INTERNAL / APP / IGNORED. Actions with non-default settings are:
INTERNAL/NONE/PUBLIC (non-auth public endpoints)
CheckApiAction
WhoisHttpServer
Rdap*Action
INTERNAL,API/APP/ADMIN (things currently protected by web.xml)
EppTlsAction
EppToolAction
CreateGroupsAction
CreatePremiumListAction
DeleteEntityAction
List*sAction
UpdatePremiumListAction
VerifyOteAction
WhoisServer
INTERNAL,API,LEGACY/USER/PUBLIC (registrar console)
RegistrarPaymentAction
RegistrarPaymentSetupAction
RegistrarSettingsAction
EppConsoleAction
INTERNAL,API,LEGACY/NONE/PUBLIC (registrar console main page)
ConsoleUiAction
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149761652
This is the first step in rolling out the changes so that we can check via logging whether turning on the logic would reject anything it should not.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149050878
This follows up on Brian's work to transition not just to a new format
with an empty scope value, but instead to replace the existing format
entirely with a new one that:
1) includes a version number to support future format migrations
2) doesn't include a field for the scope at all, since scoping the
tokens adds no real security benefit and just makes verification
more difficult
3) replaces the raw SHA-256 hash with a SHA-256 HMAC instead, as a
best practice to avoid length-extension attacks [1], even though
in our particular case they would only be able to extend the
timestamp and would thus be relatively innocuous
The new format will be produced by calling generateToken(), and the
scope-accepting version is renamed to generateLegacyToken() in addition
to its existing deprecation, for maximum clarity.
I changed the validateToken() logic to stop accepting a scope entirely;
when validating a legacy-style token, we'll test it against the two
existing legacy scope values ("admin" and "console") and accept it if
it matches either one.
Note that this means the xsrfScope parameter in @Action is now wholly
obsolete; I'll remove it in a follow-up to avoid bringing extra files
into this CL.
After this CL hits production, the next one will replace all calls to
generateLegacyToken() with generateToken(). Once that CL is deployed,
the last step will be removing the legacy fallback in validateToken().
[1] See https://en.wikipedia.org/wiki/Length_extension_attack
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148936805
This provides a safeguard against using TypeInstantiator to resolve the component class, where if resolution is done incorrectly, you end up with java.lang.Object. Formerly, that would have "succeeded" in generating a Router for Object, which of course has no methods that return @Action classes. Such a router is pretty useless, so it's better to make Router stricter and have it fail if you give it such a class by accident.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148353224
It turns out this type parameter was never necessary. A builder only needs the reflexive second type parameter when you want to have a builder inheritance hierarchy where the descendant builders have methods that the ancestor builder doesn't. In that case, the type param enables the ancestor builder's setter methods to automatically return the correct derived type, so that if you start with a derived builder, you can call a setter method inherited from an ancestor and then continue the chain with setters from the derived builder (e.g. new ContactResource.Builder().setCreationTime(now).setContactId(), which otherwise would have returned an EppResource.Builder from setCreationTime(), at which point the call to setContactId() would not compile).
Even then, it's not strictly necessary to use the type parameter, since you could instead just have each derived type override every inherited method to specify itself as the return type. But that would be a lot of extra boilerplate and brittleness.
Anyway, in this case, there is a builder hierarchy, but RequestComponentBuilder specifies all the methods that we're ever going to want on our builders, so there's never any need to be able to call specific derived builder methods. We only even need the individual builder classes so that Dagger can generate them separately for each component.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148269178
The one-day validity period is also moved from the caller into XsrfTokenManager.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147857716
No actual injection is happening in test, and at least some compilers complain that @Inject qualifiers are not allowed on non-static inner classes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147835985
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072
We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018
A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55
The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.
Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:
- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
This refactors RequestHandler so that it handles the construction of the request
component itself, rather than being handed a pre-built request component
instance constructed by the invoking servlet.
The motivation for this change is so that RequestHandler can be extended in
future CLs to compute authentication results, and can provide those results as
an available binding in the constructed request component. An alternative
approach could have been to compute the authentication results within
RequestModule itself, but I think it's clearer to keep business logic like
that outside of Dagger providers.
This CL makes the following individual changes:
- Adds request component builders, which implement a RequestComponentBuilder
interface so they can all be manipulated by RequestHandler
- Instead of obtaining request components via factory methods on the global
components, one now can have global-scoped bindings just inject the request
component builders (which requires adding a module to each global component
declaring the subcomponent). This follows the recommended approach here:
http://google.github.io/dagger/subcomponents.html
- Instead of exposing request components on the global component interface,
we now expose module-specific subclasses of RequestHandler that @Inject the
appropriate request component builder's provider and pass it to the superclass
(note that inheritance isn't strictly necessary here but saves boilerplate)
- RequestHandler now takes the Provider<RequestComponentBuilder> and builds
the component itself using its own fresh RequestModule instance. This provides
some nice encapsulation but is mainly needed for adding a RequestAuthModule
in future work.
- RequestHandler also takes UserService now, which can be provided via Dagger
by the subclass. Longer-term that will go away in favor of instead providing
AuthStrategy instances, some of which will use UserService internally.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138815648
This change moves the reflective setAccessible() calls on the request component
methods (needed so that they can be invoked reflectively from RequestHandler)
to within Router itself, eliminating the need to manually call this from each
Servlet class and then pass in the resulting Method objects. Instead, we just
pass in the request component class and let Router do the rest.
Old comments say that cross-package reflection is not allowed on AppEngine, but
while it's quite possible this was once the case, I can't reproduce that
limitation, and the documentation seems to contradict any such restriction:
"""
An application is allowed full, unrestricted, reflective access to its own
classes. It can query any private members, call the method
java.lang.reflect.AccessibleObject.setAccessible(), and read/set private
members.
"""
https://cloud.google.com/appengine/docs/java/runtime#reflection
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138693006
Convert to an action and remove ResourceServlet, JsonTransportServlet and
JsonTransportServlet, all of which exist only to support it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137519385
The dark lord Gosling designed the Java package naming system so that
ownership flows from the DNS system. Since we own the domain name
registry.google, it seems only appropriate that we should use
google.registry as our package name.
This change renames directories in preparation for the great package
rename. The repository is now in a broken state because the code
itself hasn't been updated. However this should ensure that git
correctly preserves history for each file.
