Commit graph

119 commits

Author SHA1 Message Date
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
ctingue
1c927a48e9 Filter synthetic OneTimes from billing SQL
I intend to configure ExpandRecurringBillingEventsAction in production in the near future, but until I verify that there's a 1:1 match between OneTimes expanded via SQL and OneTimes expanded via MR, filter the MR-synthetic OneTimes from the billing data view SQL.

I confirmed that this is the only script that consumes data from OneTime.

(Note that the best way would be to check for the SYNTHETIC flag, but syntheticCreationTime has a value iff the flag exists, and parsing the flags field is a relative pain in the neck compared to checking for null -- this is temporary.)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143108200
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
dxy
11584c31d8 -------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140986569
2016-12-06 11:52:46 -05:00
mountford
7cf29366bc Add scrap tool to remove transfer data
We have wound up with a few domains with invalid transfer data; the transfer status is SERVER_CANCELLED, but the other data is missing. This tool should set the transfer data for the specified domain back to null in the database.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140883792
2016-12-06 11:52:46 -05:00
mcilwain
5f32d1bbeb Correctly set HTTP error status codes when list objects fails
This makes the associated nomulus tool commands correctly return error
exit codes when the server-side component fails. This improves
scriptability.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140543216
2016-12-06 11:52:46 -05:00
lukes
8a8276e838 ​​​Remove html ambiguity from soy templates.
Soy is gaining support for parsing/validating html structure and as such a number of patterns will start getting rejected by the parser.  This change fixes newly errant soy templates by:

* transforming '<' characters that are not part of html tags to '&lt;'
* inserting whitespace following tag names so that they are unambiguous
* changing templates not rendering html to kind="text" so html rules don't apply
* fixing control flow such that all tags (and quoted attribute values) are completely defined within a single control flow block.  In some cases this required extracting {let..} vars or whole templates and in others it required duplicating conditions.
* removing stray unmatched quote characters in html tags.
* fixing incorrectly written html comments

LSC: https://docs.google.com/document/d/18MLrX8kUVzYGe1dBaSfh1kcQ_1UB02QHOk4KZtvHkIc/edit#

Tested:
    $ blaze test //third_party/java_src/gtld/java/google/registry/flows:all //third_party/java_src/gtld/java/google/registry/tools/soy:all

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140475284
2016-12-06 11:52:46 -05:00
mcilwain
2b7d580bb3 Run buildifier on codebase to format BUILD files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140362453
2016-11-28 18:15:21 -05:00
mcilwain
5eb9702f05 Improve ListDomainsCommand to list domains on multiple TLDs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140053423
2016-11-28 11:36:17 -05:00
cgoldfeder
5bac24186f Add the ability to dump the history of a single resource
This would have saved me 2+ hours yesterday of mucking around
in datastore and manually copying out the xml bytes so that
I could base64decode them.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139472542
2016-11-28 11:25:06 -05:00
cgoldfeder
419a04bc26 Command to resave datastore entities by websafe key
While I am doing this, promote LoadAndResaveEntityCommand from
javascrap to the main tool since it's repeatedly been useful, and
rename it and update its documentation to better reflect the
difference between that command and the one I am adding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139460841
2016-11-28 11:23:29 -05:00
mcilwain
7ed02f4612 Reload resources before saving in ReloadAllEppResourcesAction
This prevents a potential blind write scenario in which something else has concurrently modified the EppResource in between load and save, and those changes then get overwritten.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138911873
2016-11-15 10:59:47 -05:00
dxy
0263667dae Add Google Cloud Storage helper for non-GAE environment
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138563147
2016-11-10 12:30:00 -05:00
ctingue
7a77819977 Add retry logic to CreateLrpTokensCommand
A transient 404 on entity save interrupts a long-running run of this command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138400654
2016-11-10 11:26:03 -05:00
cgoldfeder
8256120b3a Simplify the fee extensions.
I added shared base classes to all of the Fee extension types that
make it possible to fully ignore the version in the flows. (You
ask for a FeeCreateCommandExtension, for example, and you get one
without having to worry about which). This is an improvement over
the old code that asked you to provide a list of possible fee
extensions and then ask for the first one in preference order.

As part of this I was able to make the Fee implementation a bit
simpler as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137992390
2016-11-10 11:09:23 -05:00
jianglai
59d998954c Use correct <a> tag syntax in javadoc @see tag
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137946021
2016-11-02 15:19:34 -04:00
mcilwain
1627bd4975 Revert Guava 20 features until we get the build working properly
*** Original change description ***

Remove deprecated methods with Guava 20 release

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137945126
2016-11-02 15:19:34 -04:00
jianglai
82b0bff9b5 Remove deprecated methods with Guava 20 release
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137849843
2016-11-02 15:19:34 -04:00
ctingue
b29d11f0de Fix CreateLrpTokensCommand to not split lines on double-quoted comma
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137722530
2016-11-02 15:19:34 -04:00
ctingue
c89a902b72 Add metadata parameters to CreateLrpTokensCommand
As part of this change, built out a KeyValueMapParameter from the existing TransitionListParameter in order to enable other Map types in commands (two of which are used in CreateLrpTokensCommand).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137476564
2016-11-02 15:19:34 -04:00
ctingue
1b7cee61a5 Enable command name autocomplete for nomulus tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137299749
2016-11-02 15:19:34 -04:00
dxy
a4d78afd70 Rename CloudDnsModule to CloudDnsWriterModule for consistency
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137070028
2016-11-02 15:19:34 -04:00
jart
2e81de9954 Make essential Bazel packages publicly visible
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
2016-10-24 11:57:00 -04:00
mcilwain
0f8e398c00 Fix ResaveAllEppResourcesAction to use the tools service
It's provided in ToolsRequestComponent, so it absolutely should be running on
the tools service. This was just a flat-out bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136630743
2016-10-24 10:37:49 -04:00
nickfelt
b1beefab4c Fix nomulus tool AppEngineConnection behavior
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136471276
2016-10-18 10:56:26 -04:00
nickfelt
afa4d66601 Remove final on @ParametersDelegate field
Compile-time constant inlining may interfere with JCommander's processing if a field is made final - @ParametersDelegate fields are particularly misleading.  Remove the one instance of that and add warning comments elsewhere.  See

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136469351
2016-10-18 10:37:33 -04:00
ctingue
3c36b6b928 Clean up billing MR work
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136196221
2016-10-14 17:47:46 -04:00
mountford
4b4a0cb1c1 Fix nomulus help anachronism.
list_hosts no longer differentiates between hosts by TLD.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136180967
2016-10-14 17:38:53 -04:00
mcilwain
6a738557fb Use Dagger to @Inject DnsQueue everywhere that is feasible
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136062053
2016-10-14 16:58:07 -04:00
ctingue
4e0b6d36c4 Change lrpTldStates to Interval
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135957698
2016-10-14 16:58:07 -04:00
ctingue
dd9137fbd1 Rename LrpToken to LrpTokenEntity
This is to better distinguish between an LRP "token" (the string passed along in EPP) and the datastore entity that contains the token and all metadata.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135943480
2016-10-14 16:58:07 -04:00
shikhman
f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
mcilwain
e864fedab1 Remove trailing whitespace in files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135386752
2016-10-07 15:29:47 -04:00
mcilwain
b65b855067 Always use the constructor to make Immutable Collection Builders
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135359669
2016-10-07 14:37:02 -04:00
ctingue
79387f5d1e Centralize token generation and formats
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135313726
2016-10-07 14:35:35 -04:00
mcilwain
071de78a6d Remove last stray mention of gTech
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135101061
2016-10-05 11:58:23 -04:00
mcilwain
3b02d77ceb Rename 'registry_tool' to 'nomulus'
This changes everything with external visibility beyond the codebase
(i.e. the name of the compiled binary and the documentation that refers
to it). It does not change a lot of things internal to the codebase,
i.e. the "RegistryTool" class didn't change its name. We can rename that
in a subsequent CL if we want to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135022087
2016-10-04 09:59:54 -04:00
nickfelt
0d122acec2 Update some internal build configuration options
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135014718
2016-10-04 09:58:26 -04:00
nickfelt
730cca257d Remove GtechCommand and other vestiges of gtech_tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134836647
2016-10-03 16:36:23 -04:00
nickfelt
ee13ee35b0 Merge gtech_tool functionality into registry_tool
This change consolidates gtech_tool into registry_tool.  Since App Engine has
no actual ACLs on the remote API (any access is essentially root access), we're
removing this to avoid giving the impression to users that gtech_tool is truly
locked down from a security perspective compared to registry_tool.

In addition to merging GtechTool.COMMAND_MAP into RegistryTool.COMMAND_MAP, this
change also removes the {create,update}_sandbox_tld commands (which only made
sense for gtech_tool) and removes references to gtech_tool in the documentation.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134828710
2016-10-03 16:27:24 -04:00
nickfelt
51f5c8bcfe Improve tests for RegistryTool and GtechTool
This improves the tests by:

 1) Adding tests for alphabetical ordering of the command maps, to keep them
    organized, and fixing existing mis-orderings.  Note that this is a no-op
    test for RegistryTool since it uses ImmutableSortedMap (to resort the
    commands after inserting GtechTool.COMMAND_MAP), but it'll be relevant
    in the upcoming CL when they get merged.  I changed GtechTool.COMMAND_MAP
    to use regular ImmutableMap.

 2) Checking that RegistryTool.COMMAND_MAP (the full map, after the existing
    GtechTool.COMMAND_MAP has been merged in) contains the exact same set of
    commands as all the concrete classes implementing Command that we can find.

    This is a stronger test than what we had before, which just checked that
    every Command class appeared in RegistryTool (i.e. that RegistryTool's
    commands are a superset of all Commands found).  You'd think that it'd be
    impossible for RegistryTool to contain commands that aren't in the set of
    Commands we found, but it is if we're not searching for commands properly,
    which we weren't (we were only checking within the .tools package and not
    within any subpackages (e.g. tools.javascrap).  This has now been fixed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134451859
2016-10-03 15:51:08 -04:00
mcilwain
7f0cb4eae5 Move prober deletion action to backend module
Also creates a new package named 'batch' to house it.

TESTED=I deployed it to alpha, sent a POST request to the task URL, and it
successfully ran the [].

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134332999
2016-10-03 15:41:38 -04:00
mcilwain
ffcbdbf37b Set method to POST on actions called from cron fanout
The default is to support GET, which doesn't work with cron fanout which only
uses POST.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134284855
2016-09-26 13:50:03 -04:00
mcilwain
afa6648cf7 Fix Java imports across entire codebase to be compliant
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134119822
2016-09-26 13:44:11 -04:00
nickfelt
c3723bfa2f Refactor GetEppResourceCommand hierarchy
This refactors the GetEppResourceCommand hierarchy a bit so that instead of
using the type param on the class to do implicit loading (which doesn't
work that well any more for domain applications anyway), we just do the
loading in each child class and rely on the parent class only for printing
and setting common flags.

I did this to make it possible for loadByForeignKey() to have strong typing
(in a future CL), but I think this changes stands on its own merits for
making the logic here more straightforward and actually somewhat shorter.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134115072
2016-09-26 13:41:14 -04:00
nickfelt
9dde013066 Clean up LoadAndResaveCommand
This refactors LoadAndResaveCommand a bit so that it's more straightfoward
and just switches on an enum to determine the resource type.  This has the
pro of actually making the command line flag friendlier anyway.

Also changed how it handles a non-existent (or non-active) resource so that
it throws an IAE rather than silently doing stageEntityChange(null, null).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134113671
2016-09-26 13:39:46 -04:00
mcilwain
21a98b899c Replace loadByUniqueId() with methods that don't overload unique id
It is replaced by loadByForeignKey(), which does the same thing that
loadByUniqueId() did for contacts, hosts, and domains, and also
loadDomainApplication(), which loads domain application by ROID. This eliminates
the ugly mode-switching of attemping to load by other foreign key or ROID.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133980156
2016-09-26 13:20:22 -04:00
mcilwain
4813ed392b Rename "clientIdentifier" to "clientId" almost everywhere
It's best to be consistent and use the same thing everywhere.  "clientId" was
already used in more places and is shorter and no more ambiguous, so it's the
logical one to win out.

Note that this CL is almost solely a big Eclipse-assisted refactoring. There are
two places that I did not change clientIdentifier -- the actual entity field on
Registrar (though I did change all getters and setters), and the name of a
column on the exported registrar spreadsheet. Both would require data
migrations.

Also fixes a few minor nits discovered in touched files, including an incorrect
test in OfyFilterTest.java and some superfluous uses of String.format() when
calling checkArgument().

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133956465
2016-09-22 14:30:05 -04:00
mcilwain
aa7c05cb8b Improve pretty-print diffing of Datastore entities
This removes the countless lines of the form "[null, []]" in registry_tool diffs
that are an artifact of the way we handle nulls in Objectify.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133409440
2016-09-19 12:06:54 -04:00
ctingue
75203918a9 Add command for creating LRP tokens
Command allows for both one-off creation and bulk import of assignees via file (the latter will be used for the initial import from Play Store).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133048360
2016-09-14 16:02:08 -04:00