Commit graph

139 commits

Author SHA1 Message Date
mcilwain
ec55aa5361 Resolve some Guava 20 TODOs (mostly unnecessary asList() calls)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146799536
2017-02-07 13:27:40 -05:00
nickfelt
2a32f9048e Fix CommandUtilities to fail gracefully if stdin is unavailable
Right now, it just NPEs, which is harder to debug.  Also make it handle end-of-input more cleanly by assuming that means a negative response.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146674937
2017-02-06 16:45:23 -05:00
cgoldfeder
468f1fb0f8 Fix two Gauava 20 deprecations across the tools package
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146162927
2017-02-02 16:54:28 -05:00
mmuller
ecbe2662ac Convert request factory components into a module
Move all of the code to create the request factories into
RequestFactoryModule.  Also add the --force_http_connection flag to allow us
to force the use of HTTP connections instead of HTTPOverRPC for our internal
connections.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146116640
2017-02-02 16:35:36 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mcilwain
f647ea1190 Move registry policy settings and some others into YAML config
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145571850
2017-01-30 15:03:53 -05:00
Justin Graham
bb3a0c78c5 Add framework for customizable WHOIS commands
With some additional changes by Ben McIlwain.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145447136
2017-01-25 12:25:49 -05:00
Ben McIlwain
d16971c34f Migrate away from deprecated Executor method
Among the Futures methods that run user callbacks, those that don't take
an Executor will be deleted.  This CL migrates them to the counterparts
that take MoreExecutors.directExecutor() as such Executor in the
parameter list, exactly the way that the old method works.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145358533
2017-01-25 12:21:13 -05:00
nickfelt
be4c62ccf1 Make BigqueryCommand use application default creds
This fixes a long-standing bug b/26016322 to move BigqueryCommand off of using a service account to access the Bigquery API.  It's now using Application Default Credentials, which can be easily auto-installed on a machine by running 'gcloud auth application-default login' and clicking through the OAuth consent screen.

The old method was a pain because:

1) individual users of the tool each needed to know to download and store a private key for the service account, and specify the key file via a CLI flag
2) BigQuery actions taken via the tool (e.g. load or query jobs) were listed as belonging to the service account, making them harder to find in the UI or for debugging, and difficult to audit (no idea which engineer invoked the tool)
3) within Google, this meant extra whitelisting headaches

The new method also isn't perfect because Application Default Credentials obtained via gcloud are supposed to be used primarily for local testing, and don't support setting any custom scopes.  However, we don't need custom scopes for this, and the smoother flow is worth it.

In the longer term, once the CLI is using OAuth to talk to the app itself, we'll be able to switch to the "best practice" option of also using those credentials for talking to the BigQuery API.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145120770
2017-01-25 12:21:07 -05:00
ctingue
0c2b1d1b32 Filter synthetic OneTimes from billing SQL
Fix [] this filter was in the incorrect place.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144875580
2017-01-25 10:46:09 -05:00
mcilwain
8b61dcc6d0 Remove RoidSuffixes entirely
We no longer care about ROID suffix uniqueness in a post-Registry-2.0-migration
world, and the Registry cache is sufficient for efficiently grabbing the ROID
suffix for TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144483726
2017-01-18 11:08:21 -05:00
ctingue
07010150b0 Remove PurgeSyntheticBillingEventsAction
Effectively a revert of [] now that synthetic billing events have been verified in production.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144473744
2017-01-18 11:06:44 -05:00
ctingue
3ee98988c5 Add TODOs in SQL files for recurring billing expansion
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144133707
2017-01-12 14:05:57 -05:00
ctingue
bca2169c68 Add global cursor functionality to UpdateCursorsCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143965268
2017-01-09 12:15:51 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00
mcilwain
d3397e991e Remove the old RegistryConfig paradigm entirely
We are now ready to begin configuration using YAML, mediated by ConfigModule.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143818507
2017-01-09 12:01:09 -05:00
mcilwain
c05424b947 Daggerize TMCH/signed mark util classes
This allows them to support injectable configuration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143709052
2017-01-09 11:59:04 -05:00
ctingue
a2bc05d1cd Add scrap [] to purge synthetic OneTimes
This is temporary until we verify that recurring billing event expansion is working as expected. I want to have this available in case things go south, though in a perfect world, we won't need this.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143693458
2017-01-09 11:59:04 -05:00
mmuller
270734fd9b Convert AppEngineConnection to use HttpTransport
Make AppEngineConnection use HttpTransport through HttpRequestFactory and
create factory factories for localhost and HTTPOverRPC.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143680257
2017-01-09 11:59:04 -05:00
mcilwain
8252e97dfb Break domain flow TMCH helper methods into separate injected class
This is a necessary prerequisite to subsequently injecting the configuration
dependencies.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143567753
2017-01-09 11:59:04 -05:00
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
ctingue
1c927a48e9 Filter synthetic OneTimes from billing SQL
I intend to configure ExpandRecurringBillingEventsAction in production in the near future, but until I verify that there's a 1:1 match between OneTimes expanded via SQL and OneTimes expanded via MR, filter the MR-synthetic OneTimes from the billing data view SQL.

I confirmed that this is the only script that consumes data from OneTime.

(Note that the best way would be to check for the SYNTHETIC flag, but syntheticCreationTime has a value iff the flag exists, and parsing the flags field is a relative pain in the neck compared to checking for null -- this is temporary.)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143108200
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
dxy
11584c31d8 -------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140986569
2016-12-06 11:52:46 -05:00
mountford
7cf29366bc Add scrap tool to remove transfer data
We have wound up with a few domains with invalid transfer data; the transfer status is SERVER_CANCELLED, but the other data is missing. This tool should set the transfer data for the specified domain back to null in the database.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140883792
2016-12-06 11:52:46 -05:00
mcilwain
5f32d1bbeb Correctly set HTTP error status codes when list objects fails
This makes the associated nomulus tool commands correctly return error
exit codes when the server-side component fails. This improves
scriptability.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140543216
2016-12-06 11:52:46 -05:00
lukes
8a8276e838 ​​​Remove html ambiguity from soy templates.
Soy is gaining support for parsing/validating html structure and as such a number of patterns will start getting rejected by the parser.  This change fixes newly errant soy templates by:

* transforming '<' characters that are not part of html tags to '&lt;'
* inserting whitespace following tag names so that they are unambiguous
* changing templates not rendering html to kind="text" so html rules don't apply
* fixing control flow such that all tags (and quoted attribute values) are completely defined within a single control flow block.  In some cases this required extracting {let..} vars or whole templates and in others it required duplicating conditions.
* removing stray unmatched quote characters in html tags.
* fixing incorrectly written html comments

LSC: https://docs.google.com/document/d/18MLrX8kUVzYGe1dBaSfh1kcQ_1UB02QHOk4KZtvHkIc/edit#

Tested:
    $ blaze test //third_party/java_src/gtld/java/google/registry/flows:all //third_party/java_src/gtld/java/google/registry/tools/soy:all

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140475284
2016-12-06 11:52:46 -05:00
mcilwain
2b7d580bb3 Run buildifier on codebase to format BUILD files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140362453
2016-11-28 18:15:21 -05:00
mcilwain
5eb9702f05 Improve ListDomainsCommand to list domains on multiple TLDs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140053423
2016-11-28 11:36:17 -05:00
cgoldfeder
5bac24186f Add the ability to dump the history of a single resource
This would have saved me 2+ hours yesterday of mucking around
in datastore and manually copying out the xml bytes so that
I could base64decode them.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139472542
2016-11-28 11:25:06 -05:00
cgoldfeder
419a04bc26 Command to resave datastore entities by websafe key
While I am doing this, promote LoadAndResaveEntityCommand from
javascrap to the main tool since it's repeatedly been useful, and
rename it and update its documentation to better reflect the
difference between that command and the one I am adding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139460841
2016-11-28 11:23:29 -05:00
mcilwain
7ed02f4612 Reload resources before saving in ReloadAllEppResourcesAction
This prevents a potential blind write scenario in which something else has concurrently modified the EppResource in between load and save, and those changes then get overwritten.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138911873
2016-11-15 10:59:47 -05:00
dxy
0263667dae Add Google Cloud Storage helper for non-GAE environment
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138563147
2016-11-10 12:30:00 -05:00
ctingue
7a77819977 Add retry logic to CreateLrpTokensCommand
A transient 404 on entity save interrupts a long-running run of this command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138400654
2016-11-10 11:26:03 -05:00
cgoldfeder
8256120b3a Simplify the fee extensions.
I added shared base classes to all of the Fee extension types that
make it possible to fully ignore the version in the flows. (You
ask for a FeeCreateCommandExtension, for example, and you get one
without having to worry about which). This is an improvement over
the old code that asked you to provide a list of possible fee
extensions and then ask for the first one in preference order.

As part of this I was able to make the Fee implementation a bit
simpler as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137992390
2016-11-10 11:09:23 -05:00
jianglai
59d998954c Use correct <a> tag syntax in javadoc @see tag
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137946021
2016-11-02 15:19:34 -04:00
mcilwain
1627bd4975 Revert Guava 20 features until we get the build working properly
*** Original change description ***

Remove deprecated methods with Guava 20 release

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137945126
2016-11-02 15:19:34 -04:00
jianglai
82b0bff9b5 Remove deprecated methods with Guava 20 release
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137849843
2016-11-02 15:19:34 -04:00
ctingue
b29d11f0de Fix CreateLrpTokensCommand to not split lines on double-quoted comma
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137722530
2016-11-02 15:19:34 -04:00
ctingue
c89a902b72 Add metadata parameters to CreateLrpTokensCommand
As part of this change, built out a KeyValueMapParameter from the existing TransitionListParameter in order to enable other Map types in commands (two of which are used in CreateLrpTokensCommand).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137476564
2016-11-02 15:19:34 -04:00
ctingue
1b7cee61a5 Enable command name autocomplete for nomulus tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137299749
2016-11-02 15:19:34 -04:00
dxy
a4d78afd70 Rename CloudDnsModule to CloudDnsWriterModule for consistency
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137070028
2016-11-02 15:19:34 -04:00
jart
2e81de9954 Make essential Bazel packages publicly visible
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
2016-10-24 11:57:00 -04:00
mcilwain
0f8e398c00 Fix ResaveAllEppResourcesAction to use the tools service
It's provided in ToolsRequestComponent, so it absolutely should be running on
the tools service. This was just a flat-out bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136630743
2016-10-24 10:37:49 -04:00
nickfelt
b1beefab4c Fix nomulus tool AppEngineConnection behavior
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136471276
2016-10-18 10:56:26 -04:00
nickfelt
afa4d66601 Remove final on @ParametersDelegate field
Compile-time constant inlining may interfere with JCommander's processing if a field is made final - @ParametersDelegate fields are particularly misleading.  Remove the one instance of that and add warning comments elsewhere.  See

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136469351
2016-10-18 10:37:33 -04:00
ctingue
3c36b6b928 Clean up billing MR work
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136196221
2016-10-14 17:47:46 -04:00
mountford
4b4a0cb1c1 Fix nomulus help anachronism.
list_hosts no longer differentiates between hosts by TLD.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136180967
2016-10-14 17:38:53 -04:00
mcilwain
6a738557fb Use Dagger to @Inject DnsQueue everywhere that is feasible
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136062053
2016-10-14 16:58:07 -04:00
ctingue
4e0b6d36c4 Change lrpTldStates to Interval
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135957698
2016-10-14 16:58:07 -04:00