Commit graph

65 commits

Author SHA1 Message Date
jianglai
169fd62da7 Add dependencies on classes that are not in the default modules in JDK 9
This is a follow-up to []

Also added jaxws-api Maven dependency and upgraded activation artifacts to 1.2.0, in parity with //third_party/java/activation.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=183714304
2018-02-01 22:00:49 -05:00
jianglai
97e962ba0a Add //third_party/java/jaxb dependency to targets using javax.xml.bind
To make FOSS build compile, third_party vendoring rules for jaxb are added to package all jaxb related targets imported from maven into a uber jar, mirroring the same practice done in //third_party/java/jaxb

Cloned from CL 182666460 by 'g4 patch'.
Original change by cushon@cushon:rosie182283995-0071_Rosie:47348:citc on 2018/01/20 13:36:15.

More information:
https://docs.google.com/document/d/1htErgDIoHMEuMBfGwrtS_O4WwhTw8QOGLva-7aYYvYs/edit?usp=sharing
Tested:
    TAP --sample for global presubmit queue
    []    passed FOSS test

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182855173
2018-01-23 16:18:35 -05:00
mcilwain
b5fb62c984 Change all foreach loops in Soy templates to use the for loop syntax
This also updates to a newer version of Closure Rules and fixes a protobuf dep
compile issue.

Full description of the change:

Soy supports 2 kinds of loops:
* foreach- for iterating over items in a collection, e.g.
  {foreach $item in $list}...{/foreach}
* for - for indexed iteration, e.g. {for $i in range(0, 10)}...{/for}

The reason Soy has two different loops is an accident of history; Soy didn’t use
to have a proper grammar for expressions and so the alternate ‘for...range’
syntax was added to make it possible to write indexed loops.  As the grammar has
improved having the two syntaxes is no longer necessary and so we are
eliminating one of them.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182843207
2018-01-23 16:16:53 -05:00
mcilwain
264b4be966 Upgrade Google Truth library from 0.36 to 0.39
This fixes the build error in ResourceCheckFlowTestCase.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181758287
2018-01-19 14:35:41 -05:00
mcilwain
ffcfa283f6 Roll back changelist 180942763
*** Reason for rollback ***

Breaks the FOSS build.

We'll reincorporate this change once Closure Rules is properly updated to accommodate it.

*** Original change description ***

Change all foreach loops in Soy templates to use the for loop syntax

Soy supports 2 kinds of loops:
foreach- for iterating over items in a collection  e.g. {foreach $item in $list}...{/foreach}
for - for indexed iteration  e.g. {for $i in range(0, 10)}...{/for}

The reason Soy has 2 different loops is an accident of history, Soy didn’t use to have a proper grammar for expressions and so the alternate ‘for...range’ syntax was added to make it possible to write indexed loops.  As the gramma...

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=180961695
2018-01-19 14:17:58 -05:00
lukes
7aa070b0a5 Change all foreach loops in Soy templates to use the for loop syntax
Soy supports 2 kinds of loops:
foreach- for iterating over items in a collection  e.g. {foreach $item in $list}...{/foreach}
for - for indexed iteration  e.g. {for $i in range(0, 10)}...{/for}

The reason Soy has 2 different loops is an accident of history, Soy didn’t use to have a proper grammar for expressions and so the alternate ‘for...range’ syntax was added to make it possible to write indexed loops.  As the grammar has improved having the two syntaxes is no longer necessary and so we are eliminating one of them.

As of 4a7373333f or mvn release "2018-01-03" the two forms are actually aliases for one another, so the only difference is the keyword (‘for’ vs ‘foreach’), and while the foreach loop is more popular the ‘for’ terminology is more standard so we are switching everything to that.

LSC: []
Tested:
    TAP sample presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=180942763
2018-01-19 14:14:31 -05:00
jianglai
07622725bf Move metrics dependencies to artifacts under Maven groupId com.google.monitoring-client
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=180580386
2018-01-04 17:12:35 -05:00
larryruili
36ad38e5df Add beam package to open source build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178833972
2017-12-13 12:43:45 -05:00
larryruili
735112def6 Generate detail reports from Bigquery via Beam
This establishes a fully functional pipeline which generates detail reports for each registrar_tld pair from Bigquery. The main features:

1. Deserialization from AVRO GenericRecord (from Bigquery) into BillingEvent, a POJO we control. This is especially valuable to enable intrinsic type-safety at the start of the  pipeline.
2. Addition of .sql files containing the queries used to generate detail reports. These will later be templated to enable general usage.
3. Multi-file-writing within a single TextIO transform, which writes BillingEvents to different files based on their registrar_tld key combo.

This also upgrades the Beam core SDK referenced in repositories.bzl to 2.2.0 and returns the definitions to alphabetical order, to facilitate use of the check_bazel_deps.py script.

The final steps are:
- Converting this to a Nomulus command
- Templating the .sql queries
- @Injecting the @Config values for a given project

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178124838
2017-12-13 12:43:45 -05:00
jianglai
2ae496bfce Export hamcrest packages from junit and mockito to fix build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176728505
2017-11-23 01:09:07 -05:00
jianglai
7e42ee48a4 Open source GCP proxy
Dagger updated to 2.13, along with all its dependencies.

Also allows us to have multiple config files for different environment (prod, sandbox, alpha, local, etc) and specify which one to use on the command line with a --env flag. Therefore the same binary can be used in all environments.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176551289
2017-11-21 19:19:03 -05:00
mcilwain
cf15b5937c Upgrade to JUnit 4.12
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176147815
2017-11-21 18:42:28 -05:00
jianglai
0796a0ff1c Check the host is under registry suffix instead of public suffix
Guava now has support to distinguish a registry suffix from a public suffix. Since we are only interested in registrable domains, registry suffix is the proper thing to check.

See:

692446a303/guava/src/com/google/common/net/InternetDomainName.java

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176126916
2017-11-21 18:41:06 -05:00
jianglai
054bf4bb3c Add output_licenses to repositories.bzl
https://github.com/bazelbuild/bazel/issues/2286 is resolved.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=175729373
2017-11-21 18:27:22 -05:00
larryruili
eff2266e35 Add apache beam to registry and open source
This is the initial commit of the new billing system, rewritten as an Apache
Beam pipeline. This contains a basic end-to-end pipeline as proof of concept,
and boilerplate for GenerateInvoicesAction, which will eventually be our
automated invoice generation endpoint.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=174184171
2017-11-07 17:36:07 -05:00
larryruili
abf877f5ae Update appengine-testing open source build to 1.9.58
The original impetus for this came from Kokoro build ID 57aa1042-65aa-46eb-9364-23e2bd17b2fd, where cron/TldFanoutActionTest.testSuccess_returnHttpResponse failed due to some strange nondeterminism in the fake task queue. This test isn't flaky in our build, so we assume it's been fixed in later implementations of appengine-testing.

This also adds a domain-registry-maven mirror for truth8 and clarifies the documentation for adding new libraries.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173587282
2017-11-07 17:27:12 -05:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
bbilbo
6d80482875 Remove all jar_urls pointing to maven.ibiblio.org and add missing mirrors
com_google_template_soy wasn't sorted, so I fixed that as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171347611
2017-10-10 12:09:42 -04:00
jianglai
373e174deb Upgrade to Truth 0.36
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171079195
2017-10-05 11:48:54 -04:00
mcilwain
a50ef39c04 Fix mismatch in types of Predicates being used
We're going to need to switch away from Guava's Functions and Predicates for
everything and replace them with the java.util versions. Unfortunately there
does not appear to be an automated tool to do this all at once. Refaster got
close but doesn't seem to care about these particular types of mismatch (I
suspect we're using a different version of the JDK than the outside world;
ours is OK with Guava classes).

This also bumps up Guava to 0.23, which is needed for some new functionality
used in combination with Java 8 features.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170531539
2017-10-04 16:16:45 -04:00
larryruili
578673141c Migrate to new Sheets v4 api
This moves us from the oudated google/data XML api to the OnePlatform REST/JSON api, finally silencing the deprecation warnings we've been seeing.

The synchronization algorithm diffs the spreadsheet's current values with its internally sourced values, adding the row to a batch update request if there's a discrepancy. Additional internal data are added as an append operation to the end of the sheet, and any extraneous spreadsheet data is cleared from the spreadsheet.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169273590
2017-09-20 10:27:17 -04:00
jijiang
211d89695a Migrate from old *Verb* classes to new *SubjectBuilder* classes
This migration does not change any behavior.

More information: []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168248439
2017-09-12 15:51:58 -04:00
lukes
4de8d3eae1 Migrate {css} and {xid} tags to new builtinfunctions css() and xid()
Output should be identical in either syntax, and migration will bring css and xid into consistency with other soy functions, plus it'll allow us to simplify the soy parser.

LSC: https://docs.google.com/document/d/1evNu02pVXGm1QIcN0dTmNi-GnhbCKOWdrZwBJmcNaU0/edit#

Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164887843
2017-08-29 15:56:43 -04:00
Ben McIlwain
5966d8077b Migrate TestVerb.withFailureMessage to use withMessage instead
Also updates Truth version to 0.34 where the replacement method was added.

More information: []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161970305
2017-08-01 16:30:24 -04:00
guyben
75bb1aacdc Update cloudkms_java with the non-beta version
Changed [] to use v1 instead of v1beta1, and replaced v1beta1 with v1 in all the java files.

If there is special build rules for open-source etc. that also need to be updated, or non "TAP-able" tests that need to be run, please check and see if they are OK.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157895888
2017-06-05 18:17:09 -04:00
mmuller
fc5546ad58 Add latest oauth/{java6,jetty} and fix repo procedure
Upgrade to the latest versions of omit_com_google_oauth_client_java6 and
omit_com_google_oauth_client_jetty and fix the broken stuff in our local
mirroring procedures.  Specifically:

- Document that users need to add a GCS url.
- Document how to run the check_bazel_deps script.
- Document that we don't use the in-piper repo.
- Remove the broken ibiblio link for the "truth" package.
- Delete sync_bazel_deps (which doesn't currently work, isn't python-style
  compliant and is no longer relevant since we don't use the in-piper repo)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155898485
2017-05-17 12:12:56 -04:00
jart
d419f6ac11 Export annotations from Guava
These annotations are part of guava's API, and they cannot safely be
optimized off the compile-time classpath.

This works around google/error-prone#615.

See []
H/T @cushon

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155433554
2017-05-17 11:40:02 -04:00
mcilwain
582469e052 Revert upgrade to Bazel 0.4.4
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150633068
2017-03-21 15:40:54 -04:00
jart
f37296c3a1 Set output_licenses on java_plugin
The fix was released in Bazel 0.4.4 a month ago.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150630179
2017-03-21 15:39:28 -04:00
jart
0c7443815a Add generates_api to Auto Factory too
We're going to play it safe, since the third_party definition of Auto Factory seems to have it too.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149534930
2017-03-13 10:32:24 -04:00
jart
13249db5cf Set generates_api=1 for Dagger component processor
Context: [] and []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148885749
2017-03-07 13:27:21 -05:00
mcilwain
8d84397e80 Fix Nomulus build errors
This fixes Java7 generic type inference errors in premium list code
and upgrades the Truth library to v0.32, because we're now using
assert(e).hasMessageThat() which is not in prior versions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148501657
2017-02-27 11:30:45 -05:00
shikhman
be30ecdf66 Add Cloud KMS based secret storage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147791972
2017-02-17 12:22:12 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mcilwain
b5cf58bf2c Add initial implementation of YAML config file
This implements the basic framework that allows global YAML
configuration, per-environment custom configuration, and unit-
test-specific configuration.

TESTED=I deployed to alpha, ran some EPP commands through the
nomulus tool, and verified no errors.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145422680
2017-01-25 12:25:05 -05:00
mcilwain
0c6cd4d7b7 Fix missing @CopyAnnotations by upgrading to AutoValue 1.4-rc2
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=144332981
2017-01-12 14:16:29 -05:00
mcilwain
3a5a7e030d Fix Nomulus build by adding explicit error-prone dependency
The bazel build was failing with the following error message without it:

error: cannot access CanIgnoreReturnValue
  class file for com.google.errorprone.annotations.CanIgnoreReturnValue not found

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143961684
2017-01-09 12:01:09 -05:00
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
mcilwain
e7d3725f51 Revert change to Guava 20 until we get it working properly
*** Reason for rollback ***

This broke the Nomulus build.

*** Original change description ***

Update opensource Guava to version 20

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137945645
2016-11-02 15:19:34 -04:00
ctingue
73e88b2391 Automated g4 rollback of changelist 137841045.
*** Reason for rollback ***

Not necessary for the time being (will do a more comprehensive update at a later date).

*** Original change description ***

Update errorprone to 2.0.14

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137844619
2016-11-02 15:19:34 -04:00
ctingue
883f194e0f Update errorprone to 2.0.14
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137841045
2016-11-02 15:19:34 -04:00
ctingue
a244202267 Update opensource Guava to version 20
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=137832517
2016-11-02 15:19:34 -04:00
nickfelt
38a3d038c0 Change some Bazel deps back to earlier supported versions
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136862675
2016-10-24 11:57:00 -04:00
ctingue
6a4088a8b6 Update Domain Registry maven dependencies
These are used to test our open-source build under the Kokoro Jenkins
environment ([]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136182927
2016-10-14 17:40:24 -04:00
jianglai
71d7a382f3 Change all references to Domain Registry to Nomulus
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136068582
2016-10-14 16:58:07 -04:00
shikhman
f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
ctingue
2cdd252919 Update Truth to v0.30
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135372219
2016-10-07 15:29:38 -04:00
jart
ad66f805cf Fix Closure transpile.js build error
See also:
- google/closure-compiler#2051
- bazelbuild/rules_closure@9e92334

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135138725
2016-10-05 11:58:23 -04:00
shikhman
aca4d569bd Add documentation about keeping the maven mirror up to date
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134836487
2016-10-03 16:34:55 -04:00
shikhman
9ea1c5f0d1 Update fastutil maven dependency
The version of fastutil (required by appengine-[]) that we included as an external maven dependency wasn't in the supported version range for appengine-[].

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134813255
2016-10-03 16:15:31 -04:00