Commit graph

2590 commits

Author SHA1 Message Date
gbrodman
1e8b464c67
Merge branch 'master' into gb/upgradeJCommander 2019-06-13 18:18:55 -04:00
Lai Jiang
8c5fc6410b
Remove Bazel build (#94)
* Remove bazel build

Also updated the build badge to reflect internal build status.
2019-06-13 18:15:33 -04:00
gbrodman
a7fcfca354
Merge branch 'master' into gb/upgradeJCommander 2019-06-13 18:05:28 -04:00
Lai Jiang
aca18a1d4a
Fix builds after refactor (#99)
Fixed both GAE and proxy builds after #90 refactored the code structure.

Also removed now unnecessary chmod and chown from GCB scripts.
2019-06-13 18:01:30 -04:00
Lai Jiang
b74b3dbcb3
Merge pull request #98 from google/gbrodman-patch-1
Add shicong@ and gbrodman@ to CONTRIBUTORS
2019-06-13 16:56:20 -04:00
gbrodman
392cd3a31a
Add shicong@ and gbrodman@ to CONTRIBUTORS 2019-06-13 16:07:31 -04:00
Gus Brodman
f06975df75 Upgrade jcommander dep 2019-06-13 13:04:48 -04:00
gbrodman
8f2a8835d7
Merge pull request #90 from google/gb/refactorGradle
Refactor to be more in line with a standard Gradle project structure
2019-06-13 13:01:45 -04:00
Gus Brodman
0a34c5ba36 More moving of hello.xml files -- they're all the same 2019-06-13 11:52:04 -04:00
Gus Brodman
b883f86933 Move hello.xml and don't use project.ext.foo 2019-06-13 10:14:50 -04:00
Gus Brodman
a7a983bfed Refactor to be more in line with a standard Gradle project structure 2019-06-13 09:41:11 -04:00
Shicong Huang
8fa45e8c76
Merge pull request #96 from google/sc/fix-uber-jar
Use ShadowJar plugin to generate uber jar
2019-06-12 23:07:28 -04:00
Shicong Huang
46a58b7141 Use ShadowJar plugin to generate uber jar
Our existing uber jar task overwrote duplicate files when merging JARs,
and this would break the application using Java SPI service(Beam library
is one of them) so we changed to use ShadowJar plugin as it provides a
convenient way to merge META-INF/services/* files when generating the
uber jar.

TESTED=Deployed spec11 pipeline successfully with the new nomulus.jar;
also randomly checked a few nomulus commands to make sure they are still
working.
2019-06-12 19:48:57 -04:00
guyben
cb7ca611a9 Add action to sync all Registrar RDAP base URLs from the ICANN endpoint
We aren't turning on the cron job yet - we want to make sure it works correctly first.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252726789
2019-06-12 13:15:12 -04:00
mcilwain
95111809bd Add a nomulus tool command to bulk ACK poll messages
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252674014
2019-06-12 13:13:33 -04:00
weiminyu
d7da7f6da4 Replace deprecated CharMatcher method
CharMatcher.isLetterOrDigit() is deprecated for failing to support
supplementary characters. We explicitly declare a matcher for ascii
letters and digits.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252671830
2019-06-12 13:11:44 -04:00
guyben
00c7c0f1af Clarify that the supplied RDAP Base URLs might get overridden
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252661230
2019-06-12 13:09:58 -04:00
jianglai
b8ffa732b2 Add GCB workflows to promote the nomulus tool command after deployment
With https://github.com/spinnaker/spinnaker/issues/4048 Spinnaker now natively supports GCB. We are able to start a GCB job from Spinnaker, and also there is better support to consume GCB pub/sub messages. Some changes are made to remove the workaround no longer needed.

Two new workflows are added, one to rsync a GCS folder to live/ after the deployment is done (so that the nomulus.jar file can then be fetched to x20 by a []cron job), and the other to tag the proxy image as live once it is deployed.

Lastly, the docs/ folders are needed when running tests. Remove it from .gcloudignore so that when a test run is kicked off by running "gcloud builds submit" the folder is sent to GCB. Ideally .gcloudignore should be identical to .gitignore but since they both are version controlled it is hard it make one a symlink of another.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252625901
2019-06-12 13:08:11 -04:00
gbrodman
deb826794d Use a 'number' input type for number fields in registrar creation
No tests for this since we don't have any to begin with

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252508756
2019-06-12 13:06:29 -04:00
guyben
4110cae814 Link the Registrar's RDAP server from RDAP domain replies
To do this we add a field of "rdapServers" in the Registrar object. Currently, we can only set this field manually, but a subsequent CL will add a cron-job to read these values from the ICANN servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=252438618
2019-06-12 13:04:40 -04:00
weiminyu
7c64992c73 Cloned from CL 251456914 by 'g4 patch'.
Original change by cpovirk@cpovirk:rosie251284456-0055_Rosie:31511:citc on 2019/06/04 09:48:27.

Update to Truth 0.45, and address deprecations.

Renames may include:
- containsAllOf => containsAtLeast
- containsAllIn => containsAtLeastElementsIn
- isSameAs => isSameInstanceAs
- isOrdered => isInOrder
- isStrictlyOrdered => isInStrictOrder

The other major change is to change custom subjects to extend raw Subject instead of supplying type parameters. The type parameters are being removed from Subject. This CL will temporarily produce rawtypes warnings, which will go away when I remove the type parameters (as soon as this batch of CLs is submitted).

Some CLs in this batch also migrate calls away from actualAsString(). Its literal replacement is `"<" + actual + ">"` (unless an object overrides actualCustomStringRepresentation()), but usually I've made a larger change, such as switching from an old-style "Not true that..." failure message to one generated with the Fact API. In that case, the new code usually contains a direct reference to this.actual (a field that I occasionally had to create). Another larger change I sometimes made is to switch from a manual check-and-fail approach to instead use check(...). And sometimes I just remove a withMessage() call that's no longer necessary now that the code uses check(...), or I introduce a check(...) call. (An assertion made with check(...) automatically includes the actual value from the original subject, so there's no need to set it again with withMessage().)

Finally, there's one CL in this batch in which I migrate a Correspondence subclass to instead use Correspondence.from.

END_PUBLIC

If this is too many changes at once, let me know, and I can split it up and/or explain further. In the past, I've erred on the side of sending separate CLs for each change, which has required some owners to manually reapply each one, so now I'm trying this way.

Thanks again for your patience. There is an outside chance that this will be the last CL I send you before Truth 1.0 -- but certainly no promises :)

More information:
  Renames:
    []
    []
    []
  Removing type parameters: []
  Migration from old fail*(...) methods to new ones and to check(...): []
  Changes that replace assert*(...) with introduce check(...): []
  Correspondence subclass to Correspondence.from: []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251921007
2019-06-07 11:46:44 -04:00
jianglai
e8c789a401 Fix builder error
We need to first populate the apt sources before installing anything. It was working on my machine because the layer was cached. After I purged all images I was able to replicate the same error message.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251918035
2019-06-07 11:44:57 -04:00
guyben
ca1d525e28 Conform to RDAP Technical Implementation Guide
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251864499
2019-06-07 11:43:09 -04:00
mcilwain
55dc735ba0 Export PollMessage entities to BigQuery
It would be nice to be able to query them using SQL in order to debug
some potential issues registrars might be facing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=251863662
2019-06-07 11:41:22 -04:00
mcilwain
ff6d327183 Allow reserved domains to be created during quiet periods
We'll use this for LRP. This is safe because we must specifically reserve a
domain by including it in a reserved list, create an associated allocation
token, and distribute that token, before a create would succeed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250901144
2019-06-06 10:53:27 -04:00
gbrodman
79bcb227be De-duplicate flow documentation error reasons on a per-code basis
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250894847
2019-06-06 10:51:43 -04:00
guyben
e489539df4 Enable more DNS update metrics on production
They are useful for viewing and diagnosing problems, and are not that
expensive.

We aren't enabling all the metrics though - the whole "per batch / per domain"
performance indicators are still disabled

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250876554
2019-06-06 10:49:55 -04:00
mcilwain
dd66664e42 Add tests that some domain operations work in quiet periods
This specifically includes info, transfer, and update. Note that normal
domain creates do not work in quiet periods and are not expected to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250755940
2019-06-06 10:48:05 -04:00
mcilwain
7bdbd89e4d Add test that anchor tenants can be created in quiet period
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250752454
2019-06-06 10:46:14 -04:00
mcilwain
987a648296 Smear out DNS updates over time when updating DNS for all domains
These updates can generally ran slowly in the background (i.e. they're low
priority), and shouldn't clog up the system to the point where they slow down
live incoming customer-requested updates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250732171
2019-06-06 10:44:27 -04:00
gbrodman
d2f955a488 Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
jianglai
ef9478b68c Migrate org.mockito.Matchers#any* to org.mockito.ArgumentMatchers
The former is deprecated and replaced by the latter in Mockito 2. However, there is a
functional difference: ArgumentMatchers will reject `null` and check the type
if the matcher specified a type (e.g. `any(Class)` or `anyInt()`). `any()` will
remain to accept anything.

For more information see []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250690285
2019-05-30 12:52:21 -04:00
jianglai
28393e569c Upload the tool jar to GCS
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250496565
2019-05-30 12:52:21 -04:00
gbrodman
6a272bc8c6 Modify DeleteAllocationTokensCommand to have the same input structure as UpdateATC
It's dangerous to have a blank prefix delete all tokens and this allows for some code unification.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250493453
2019-05-30 12:52:21 -04:00
guyben
44ccd45439 Fix bazel breakage caused by []
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)

Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
2019-05-30 12:52:21 -04:00
guyben
c79e0ea670 Conform to RDAP Response Profile 15feb19
This is only about the Response Profile, not the Technical Implementation guide.

The Response Profile can be found at https://www.icann.org/en/system/files/files/rdap-response-profile-15feb19-en.pdf

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250277559
2019-05-30 12:52:21 -04:00
cpovirk
b34a828b71 Migrate Correspondence subclasses to instead call Correspondence.from()
This makes the code shorter, and the subclassing approach is deprecated.

Open-source note: Correspondence.from was added to Truth 0.43. (So was Correspondence.transforming, which I'm using occasionally instead.) Correspondence.formattingDiffsUsing was added in Truth 0.44.

END_PUBLIC

More information:
  []Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249913322
2019-05-30 12:52:21 -04:00
guyben
7cc5c16ab9 Increase delay between attempts of rde staging
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.

Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.

8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
2019-05-30 12:52:21 -04:00
jianglai
5f7e725c29 Add the ability to turn off dependency locking
This makes it easy to experiment with new dependency versions or new plugins
without having to update the lock files, by adding "-PdisableDependencyLocking=true" to any gradle command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249463575
2019-05-30 12:52:21 -04:00
jianglai
243fce84ce Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-30 12:52:21 -04:00
jianglai
706b47c499 Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-21 16:48:28 -04:00
jianglai
f2012b73de Copy META-INF to GAE deployment jar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249238153
2019-05-21 11:47:49 -04:00
gbrodman
7fd1ad1dc6 Compile all Soy JS files
I was testing this to see what the error was and I can't find one, so I assume it must have just been fixed at some point

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248551972
2019-05-21 11:44:26 -04:00
mmuller
f18f389b03 Increase NS record default TTLs to 3 hours
Per further discussion on b/130533696, we've decided to increase these so
tenants can have greater resilience to an outage in our DNS servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248546853
2019-05-21 11:42:39 -04:00
cpovirk
73f7e185f2 Migrate users from the old, deprecated Subject.fail* methods to the new Subject.fail* methods or, in some cases, to Subject.check.
Most of the changes in this CL were made manually.

I've tried to preserve all information (and of course behavior!), but the format and grammar of the messages does change. For example before-and-after messages, see the LSC doc.

In some of the CLs in this round (e.g., jscomp), I don't know a lot about the domain being tested, and the assertions are complex, so please let me know if my new phrasing is wrong or confusing.

Thanks again for your patience with all the Truth changes lately.

END_PUBLIC

More information:
  []

Tested:
    TAP for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248543745
2019-05-21 11:40:53 -04:00
gbrodman
6ce74ff61b Delete the AbstractDomainBaseSubject
It only had one subclass and it unnecessarily complicates things. It originally existed because DomainBase was abstract and could have multiple implementations but that is no longer the case.

This will also help with []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248529389
2019-05-21 11:39:08 -04:00
gbrodman
8f9f384406 Add a small prompt to the UpdateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248523681
2019-05-21 11:37:22 -04:00
jianglai
b12a462f5e Add Google Analytics to registrar console
To support the open source community, which may want to use different analytics services, we implement a soy template for analytics services that:

1) Does not require users to implement Google Analytics
2) Allows users to add their own analytics code to `Analytics.soy`
3) Gives users the flexibility to pass as much or as little static configuration to their custom analytics code as needed.
4) Ensures that users can merge upstream Nomulus code in the future without having to delete their custom analytics implementations
5) Does not allow code to be injected from configuration, which Soy as a framework actively discourages.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248340081
2019-05-21 11:35:34 -04:00
gbrodman
7f69ebc5d9 Add a bit more verification text when unrenewing domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248150595
2019-05-21 11:33:47 -04:00
guyben
a0040c5eda Save the RDAP request time globally instead of passing it around
Also removed the rdapWhoisServer value, as it's just null and will likely stay that way (it isn't mentioned in the RDAP response profile)

If it'll ever become required, we can add it back.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247643981
2019-05-13 12:44:55 -04:00