Commit graph

20 commits

Author SHA1 Message Date
mmuller
2ec8246097 Add a dependency on the local tools.jar for doc tool generation.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225565675
2018-12-20 07:46:33 -05:00
shicong
296acf80bb Add build logic to upload dependency JARs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225539482
2018-12-14 11:28:10 -05:00
jianglai
377736e987 Add an option to perform login on remote machines
If the user runs "nomulus -e [ENV] login --remote", an URL will be provided, the user then can visit the URL on any machine (not necessary where the command is run) and copy&paste back the authorization code to complete authorization.

This makes it easy to login on machines where local browsers are not easily accessible.

Also upgraded nebula lint version to 10.3.5.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225198700
2018-12-12 13:22:34 -05:00
jianglai
c7fc964d9c Add a Gradle task to build the nomulus tool
It'd be nice if we can separate out the tool to its own package and reduce the transitive dependencies that it pulls in. However since the entire core project is a dependency of the tool, it doesn't make any difference as we'd be pulling in core and all its transitive dependencies as well.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224821931
2018-12-12 13:22:34 -05:00
jianglai
a85544b3f6 Use gson to make JSON string in proxy log formatter
This is simpler than using fasterxml.jackson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224583713
2018-12-12 13:22:34 -05:00
jianglai
a612e9bf66 Use local credential to deploy beam pipelines
We are moving away from using Application Default Credentials generated by "gcloud auth application-default login" in our code base and consolidate on using self-managed credentials provided from AuthModule.

One of the remaining dependencies on the ADCs is from beam pipeline deployment commands, which by default use the ADCs to talk to GCS and upload the jar files and templates. In this CL, we explicitly provide the locally created credential to the Options used in deployments.

Also moved all credential qualifiers to CredentialModule, and removed @AppEngineAdminApiCredential, which is no longer used.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224199812
2018-12-05 16:07:54 -05:00
jianglai
6352b8a01a Use self-managed credential in remote api installer
RemoteApiOption has a package-private method that takes a Stream representing the content of a JSON and use a GoogleCredential created from it as its credential. This CL uses reflection to change the access modifier of that method in order to supply a credential stream that is self-managed. This is obviously not ideal and prone to breakage in case the getGoogleCredentialStream method is changed. Unfortunately upstream is not willing to make it public citing the reason that GoogleCredential.fromStream() (which getGoogleCredentialStream uses) is a @Beta annotated function (see https://groups.google.com[]forum/#!searchin/domain-registry-eng/remoteapioptions%7Csort:date/domain-registry-eng/Flsah6skszQ/CySZv2XEBwAJ). However this function is introduced 5 years ago as a public function (b857184bfa). I think at this point it is safe to assume that it is part of the widely used APIs and will not change without sufficient notice.

Note here that RemoteApiOptions creates its own copy of GoogleCredential to be used to call App Engine APIs locally, whereas communications to Nomulus endpoints use the Credential provided in AuthModule. Even though both credentials are created from the same client id, client secret and refresh token (the three elements needed to construct a GoogleCredential this way, see https://github.com/googleapis/google-api-java-client/blob/master/google-api-client/src/main/java/com/google/api/client/googleapis/auth/oauth2/GoogleCredential.java#L842), their refreshes cycles are independent of each other. I verified that refreshing one of the credential does not invalidate the access token of the other credential, as long as it is not expired yet.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224156131
2018-12-05 16:02:28 -05:00
shicong
d20b83c820 Add option --non_live_versions to set_num_instances command
This commit introduced a new flag to enable SetNumInstancesCommand to
be able to set the number of instances for all non-live versions for
a given service or for all deployed services.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222826003
2018-12-03 18:58:05 -05:00
jianglai
f46f817f9e Create proxy Docker image with Gradle
This CL adds two ways to build an docker image with gradle:

1) Adds a :proxy:deployJar task that builds an uber jar that contains all runtime dependencies. The jar can then be add to a docker image by calling docker with the added Dockerfile. The base image for this image can be both distroless java or openjdk:alpine.

2) Uses the Gradle distribution plugin to build a distribution tar file that contains all dependencies (as separate jar files) and a run script that sets up the classpath before calling the main class. Then the docker application plugin can build a docker image (with the dockerBuildImage task) using the application tar file. This only works with openjdk:alpline base image as the distroless java image does not contain a shell and therefore the script created by the distribution plugin cannot be launched.

We may later decide to use one of the method and remove the other.

Also adds an outcast test pattern that caused the tests to be flaky.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222145192
2018-11-19 18:19:27 -05:00
jianglai
118367dc6b Move proxy to its separate project
It can be run directly by "gradle :proxy:run"

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222102849
2018-11-19 18:17:47 -05:00
jianglai
997b49f7ca Move util to its separate project
It will make it easier later to have the proxy project depend on it, rather
than on the entire core project

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222081994
2018-11-19 18:16:03 -05:00
jianglai
f7e95e1ff1 Refactor gradle build script
1. Updated nebula lint version to 10.3.1
2. Do not apply plugins in the root projects that are not needed.
3. Only do linting when a build is successful, so that build failure message are not flushed by linter warnings.
4. Added explicitly stated java source and target version.
5. Moved source sets set up to the root project so that it applies to all subprojects. Currently there is only "core", but we will add at least "proxy" and "util" later, both of which will use mostly the same source sets (but with additional inclusion rules to only build classes in a specific sub folder in the source tree). By putting the set up closure inside the subprojects block in the root script, it can be reused.
6. Rename maybe_runtime configuration to maybeRuntime, which is consistent with other camelCase configuration names like testCompile.
7. Added a runtime dependency of the JAXB API which is removed from Java SE as of version 11.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222081092
2018-11-19 18:14:09 -05:00
jianglai
02a23c6ca6 Remove unnecessary source sets
These source sets are not configured correctly (no classpaths for example) and
they subsequently cannot compile. The special tests are put into separate tasks that
run from the same source set (test).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222080273
2018-11-19 18:12:12 -05:00
weiminyu
dee559baee Define multiple test suites under Gradle for better test performance
Tests are divided into three test suites without intra-suite conflicts.
This allows us to unset forkEvery=1 (4x improvement on my desktop) and
increase execution parallelism (additional 2x improvement on my desktop).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221656972
2018-11-16 16:55:55 -05:00
weiminyu
783c010ab4 Add a IDE helper plugin for annotation processors
Without the net.ltgt.apt plugin, Dagger does not retain generated source code,
and AutoValue mixes generated source code with classes. The new plugin fixes this
problem, making it easier to work with IDEs.

Based on limit experience, Intellij Idea can properly import this project. We
may consider adding IDE-specific plugins in the future.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=220802052
2018-11-12 14:51:40 -05:00
weiminyu
6a870e5820 Check for unused dependencies
Enabled unused-dependency check using nebula-lint.

Dependencies that are not used by compile or testing are
labeled with 'maybe_runtime". We leave these dependencies
in the script for easy reference. Before launching Gradle-based
release process we must determine which of these should
be removed and which should be relabeled as runtime.

Label assignment:
- All dependencies recommended for removal from 'compile'
  are changed to maybe_runtime
- All dependencies recommended for move from 'compile' to
  testCompile are split into two lines, one with testCompile,
  the other maybe_runtime

Incidentally, Gradle 4.10.2 needs a groovy upgrade before
it can work with Oracle JDK 11.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219803797
2018-11-02 14:49:18 -04:00
weiminyu
00d0284d5b Add nebula-lint plugin to Gradle scripts
Updated the project config so that the plugin can work.

Reverted to the deprecated compile/testCompile labels
so that the plugin can work. This plugin provides valuable
checks that we do not want to give up.

Added undeclared-dependency check. Still need to add
unused-dependency and one-version check.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219406600
2018-10-31 19:15:03 -04:00
jianglai
c375b0d5f4 Add javax.activation as a dependency of jaxb
In Java 9+, this module is no longer loaded by default. JAXB needs it to compile.

Also changed the root project name to "nomulus".

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=219000053
2018-10-29 15:41:06 -04:00
weiminyu
f6d9b46622 Reorganize Gradle dependencies
Move configs and dependencies from root script to subprojects.
There is not too much commonality between subprojects. Besides,
the gradle lint plugin (not added to this project yet) does not
handle inherited dependencies properly.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218698818
2018-10-25 14:50:26 -04:00
weiminyu
b81525cea4 Set up gradle for Nomulus on github
Also updates apache-sshd dependencies.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217603216
2018-10-22 18:51:39 -04:00