Commit graph

3796 commits

Author SHA1 Message Date
gbrodman
38d8d6e91e Merge branch 'master' into gb/validateEmails 2019-06-14 10:48:57 -04:00
Lai Jiang
f8e40c8902 Add a link to kokoro (#101)
* Add a link to kokoro

This allows us to click on the build badge to visit kokoro and find
out build status. The link will redirect to a status page that is
only accessible to Googlers.
2019-06-14 10:48:35 -04:00
gbrodman
c766ec6322 Merge branch 'master' into gb/validateEmails 2019-06-14 10:44:23 -04:00
gbrodman
6650b0625b Merge pull request #92 from google/gb/upgradeJCommander
Upgrade the JCommander dependency to 1.60
2019-06-13 18:19:07 -04:00
gbrodman
2aab7ce519 Merge branch 'master' into gb/upgradeJCommander 2019-06-13 18:18:55 -04:00
Lai Jiang
e17986dfa4 Remove Bazel build (#94)
* Remove bazel build

Also updated the build badge to reflect internal build status.
2019-06-13 18:15:33 -04:00
gbrodman
45b3aa8e06 Merge branch 'master' into gb/upgradeJCommander 2019-06-13 18:05:28 -04:00
Gus Brodman
aab6ab3c1d Add more tests in RegistrarTest 2019-06-13 18:04:57 -04:00
Lai Jiang
ad20178f18 Fix builds after refactor (#99)
Fixed both GAE and proxy builds after #90 refactored the code structure.

Also removed now unnecessary chmod and chown from GCB scripts.
2019-06-13 18:01:30 -04:00
Lai Jiang
9eb54f94f8 Merge pull request #98 from google/gbrodman-patch-1
Add shicong@ and gbrodman@ to CONTRIBUTORS
2019-06-13 16:56:20 -04:00
gbrodman
1b3fa51788 Add shicong@ and gbrodman@ to CONTRIBUTORS 2019-06-13 16:07:31 -04:00
Gus Brodman
111b3f15b2 Validate provided email addresses when creating a registrar 2019-06-13 13:13:06 -04:00
Gus Brodman
65dfebf99d Upgrade jcommander dep 2019-06-13 13:04:48 -04:00
gbrodman
454dcb9b0e Merge pull request #90 from google/gb/refactorGradle
Refactor to be more in line with a standard Gradle project structure
2019-06-13 13:01:45 -04:00
Gus Brodman
9a59cd587e More moving of hello.xml files -- they're all the same 2019-06-13 11:52:04 -04:00
Gus Brodman
fdf85b507c Move hello.xml and don't use project.ext.foo 2019-06-13 10:14:50 -04:00
Gus Brodman
38cfc9f693 Refactor to be more in line with a standard Gradle project structure 2019-06-13 09:41:11 -04:00
Shicong Huang
98f87bcc03 Merge pull request #96 from google/sc/fix-uber-jar
Use ShadowJar plugin to generate uber jar
2019-06-12 23:07:28 -04:00
Shicong Huang
aefe92b695 Use ShadowJar plugin to generate uber jar
Our existing uber jar task overwrote duplicate files when merging JARs,
and this would break the application using Java SPI service(Beam library
is one of them) so we changed to use ShadowJar plugin as it provides a
convenient way to merge META-INF/services/* files when generating the
uber jar.

TESTED=Deployed spec11 pipeline successfully with the new nomulus.jar;
also randomly checked a few nomulus commands to make sure they are still
working.
2019-06-12 19:48:57 -04:00
guyben
621b373b1c Add action to sync all Registrar RDAP base URLs from the ICANN endpoint
We aren't turning on the cron job yet - we want to make sure it works correctly first.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252726789
2019-06-12 13:15:12 -04:00
mcilwain
691220748d Add a nomulus tool command to bulk ACK poll messages
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252674014
2019-06-12 13:13:33 -04:00
weiminyu
0ed0455100 Replace deprecated CharMatcher method
CharMatcher.isLetterOrDigit() is deprecated for failing to support
supplementary characters. We explicitly declare a matcher for ascii
letters and digits.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252671830
2019-06-12 13:11:44 -04:00
guyben
7cfbfdd0e4 Clarify that the supplied RDAP Base URLs might get overridden
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252661230
2019-06-12 13:09:58 -04:00
jianglai
b664102048 Add GCB workflows to promote the nomulus tool command after deployment
With https://github.com/spinnaker/spinnaker/issues/4048 Spinnaker now natively supports GCB. We are able to start a GCB job from Spinnaker, and also there is better support to consume GCB pub/sub messages. Some changes are made to remove the workaround no longer needed.

Two new workflows are added, one to rsync a GCS folder to live/ after the deployment is done (so that the nomulus.jar file can then be fetched to x20 by a []cron job), and the other to tag the proxy image as live once it is deployed.

Lastly, the docs/ folders are needed when running tests. Remove it from .gcloudignore so that when a test run is kicked off by running "gcloud builds submit" the folder is sent to GCB. Ideally .gcloudignore should be identical to .gitignore but since they both are version controlled it is hard it make one a symlink of another.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252625901
2019-06-12 13:08:11 -04:00
gbrodman
0b51c50a0d Use a 'number' input type for number fields in registrar creation
No tests for this since we don't have any to begin with

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252508756
2019-06-12 13:06:29 -04:00
guyben
203206171e Link the Registrar's RDAP server from RDAP domain replies
To do this we add a field of "rdapServers" in the Registrar object. Currently, we can only set this field manually, but a subsequent CL will add a cron-job to read these values from the ICANN servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252438618
2019-06-12 13:04:40 -04:00
weiminyu
1200fd282f Cloned from CL 251456914 by 'g4 patch'.
Original change by cpovirk@cpovirk:rosie251284456-0055_Rosie:31511:citc on 2019/06/04 09:48:27.

Update to Truth 0.45, and address deprecations.

Renames may include:
- containsAllOf => containsAtLeast
- containsAllIn => containsAtLeastElementsIn
- isSameAs => isSameInstanceAs
- isOrdered => isInOrder
- isStrictlyOrdered => isInStrictOrder

The other major change is to change custom subjects to extend raw Subject instead of supplying type parameters. The type parameters are being removed from Subject. This CL will temporarily produce rawtypes warnings, which will go away when I remove the type parameters (as soon as this batch of CLs is submitted).

Some CLs in this batch also migrate calls away from actualAsString(). Its literal replacement is `"<" + actual + ">"` (unless an object overrides actualCustomStringRepresentation()), but usually I've made a larger change, such as switching from an old-style "Not true that..." failure message to one generated with the Fact API. In that case, the new code usually contains a direct reference to this.actual (a field that I occasionally had to create). Another larger change I sometimes made is to switch from a manual check-and-fail approach to instead use check(...). And sometimes I just remove a withMessage() call that's no longer necessary now that the code uses check(...), or I introduce a check(...) call. (An assertion made with check(...) automatically includes the actual value from the original subject, so there's no need to set it again with withMessage().)

Finally, there's one CL in this batch in which I migrate a Correspondence subclass to instead use Correspondence.from.

END_PUBLIC

If this is too many changes at once, let me know, and I can split it up and/or explain further. In the past, I've erred on the side of sending separate CLs for each change, which has required some owners to manually reapply each one, so now I'm trying this way.

Thanks again for your patience. There is an outside chance that this will be the last CL I send you before Truth 1.0 -- but certainly no promises :)

More information:
  Renames:
    []
    []
    []
  Removing type parameters: []
  Migration from old fail*(...) methods to new ones and to check(...): []
  Changes that replace assert*(...) with introduce check(...): []
  Correspondence subclass to Correspondence.from: []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251921007
2019-06-07 11:46:44 -04:00
jianglai
54e9aa6e9d Fix builder error
We need to first populate the apt sources before installing anything. It was working on my machine because the layer was cached. After I purged all images I was able to replicate the same error message.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251918035
2019-06-07 11:44:57 -04:00
guyben
c3c3520e04 Conform to RDAP Technical Implementation Guide
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251864499
2019-06-07 11:43:09 -04:00
mcilwain
5e4199fae6 Export PollMessage entities to BigQuery
It would be nice to be able to query them using SQL in order to debug
some potential issues registrars might be facing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251863662
2019-06-07 11:41:22 -04:00
mcilwain
c0e8695869 Allow reserved domains to be created during quiet periods
We'll use this for LRP. This is safe because we must specifically reserve a
domain by including it in a reserved list, create an associated allocation
token, and distribute that token, before a create would succeed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250901144
2019-06-06 10:53:27 -04:00
gbrodman
353fb0a35e De-duplicate flow documentation error reasons on a per-code basis
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250894847
2019-06-06 10:51:43 -04:00
guyben
86151a7b40 Enable more DNS update metrics on production
They are useful for viewing and diagnosing problems, and are not that
expensive.

We aren't enabling all the metrics though - the whole "per batch / per domain"
performance indicators are still disabled

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250876554
2019-06-06 10:49:55 -04:00
mcilwain
a4aa614bfe Add tests that some domain operations work in quiet periods
This specifically includes info, transfer, and update. Note that normal
domain creates do not work in quiet periods and are not expected to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250755940
2019-06-06 10:48:05 -04:00
mcilwain
b9fed51d9f Add test that anchor tenants can be created in quiet period
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250752454
2019-06-06 10:46:14 -04:00
mcilwain
3e34f61b8c Smear out DNS updates over time when updating DNS for all domains
These updates can generally ran slowly in the background (i.e. they're low
priority), and shouldn't clog up the system to the point where they slow down
live incoming customer-requested updates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250732171
2019-06-06 10:44:27 -04:00
gbrodman
ba23e3efaa Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
jianglai
3d26359e2d Migrate org.mockito.Matchers#any* to org.mockito.ArgumentMatchers
The former is deprecated and replaced by the latter in Mockito 2. However, there is a
functional difference: ArgumentMatchers will reject `null` and check the type
if the matcher specified a type (e.g. `any(Class)` or `anyInt()`). `any()` will
remain to accept anything.

For more information see []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250690285
2019-05-30 12:52:21 -04:00
jianglai
538c659609 Upload the tool jar to GCS
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250496565
2019-05-30 12:52:21 -04:00
gbrodman
5c127b8020 Modify DeleteAllocationTokensCommand to have the same input structure as UpdateATC
It's dangerous to have a blank prefix delete all tokens and this allows for some code unification.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250493453
2019-05-30 12:52:21 -04:00
guyben
4d4926bac7 Fix bazel breakage caused by []
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)

Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
2019-05-30 12:52:21 -04:00
guyben
a457b57621 Conform to RDAP Response Profile 15feb19
This is only about the Response Profile, not the Technical Implementation guide.

The Response Profile can be found at https://www.icann.org/en/system/files/files/rdap-response-profile-15feb19-en.pdf

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250277559
2019-05-30 12:52:21 -04:00
cpovirk
19a72ec89b Migrate Correspondence subclasses to instead call Correspondence.from()
This makes the code shorter, and the subclassing approach is deprecated.

Open-source note: Correspondence.from was added to Truth 0.43. (So was Correspondence.transforming, which I'm using occasionally instead.) Correspondence.formattingDiffsUsing was added in Truth 0.44.

END_PUBLIC

More information:
  []Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249913322
2019-05-30 12:52:21 -04:00
guyben
65b5a9d063 Increase delay between attempts of rde staging
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.

Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.

8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
2019-05-30 12:52:21 -04:00
jianglai
8d3309647a Add the ability to turn off dependency locking
This makes it easy to experiment with new dependency versions or new plugins
without having to update the lock files, by adding "-PdisableDependencyLocking=true" to any gradle command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249463575
2019-05-30 12:52:21 -04:00
jianglai
6b4bdf8d9a Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-30 12:52:21 -04:00
jianglai
eb4dc95f66 Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-21 16:48:28 -04:00
jianglai
7e2af1eb61 Copy META-INF to GAE deployment jar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249238153
2019-05-21 11:47:49 -04:00
gbrodman
4b581d4268 Compile all Soy JS files
I was testing this to see what the error was and I can't find one, so I assume it must have just been fixed at some point

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248551972
2019-05-21 11:44:26 -04:00
mmuller
e43fffe4b2 Increase NS record default TTLs to 3 hours
Per further discussion on b/130533696, we've decided to increase these so
tenants can have greater resilience to an outage in our DNS servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248546853
2019-05-21 11:42:39 -04:00