Commit graph

2714 commits

Author SHA1 Message Date
guyben
e489539df4 Enable more DNS update metrics on production
They are useful for viewing and diagnosing problems, and are not that
expensive.

We aren't enabling all the metrics though - the whole "per batch / per domain"
performance indicators are still disabled

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250876554
2019-06-06 10:49:55 -04:00
mcilwain
dd66664e42 Add tests that some domain operations work in quiet periods
This specifically includes info, transfer, and update. Note that normal
domain creates do not work in quiet periods and are not expected to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250755940
2019-06-06 10:48:05 -04:00
mcilwain
7bdbd89e4d Add test that anchor tenants can be created in quiet period
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250752454
2019-06-06 10:46:14 -04:00
mcilwain
987a648296 Smear out DNS updates over time when updating DNS for all domains
These updates can generally ran slowly in the background (i.e. they're low
priority), and shouldn't clog up the system to the point where they slow down
live incoming customer-requested updates.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250732171
2019-06-06 10:44:27 -04:00
gbrodman
d2f955a488 Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
jianglai
ef9478b68c Migrate org.mockito.Matchers#any* to org.mockito.ArgumentMatchers
The former is deprecated and replaced by the latter in Mockito 2. However, there is a
functional difference: ArgumentMatchers will reject `null` and check the type
if the matcher specified a type (e.g. `any(Class)` or `anyInt()`). `any()` will
remain to accept anything.

For more information see []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250690285
2019-05-30 12:52:21 -04:00
jianglai
28393e569c Upload the tool jar to GCS
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250496565
2019-05-30 12:52:21 -04:00
gbrodman
6a272bc8c6 Modify DeleteAllocationTokensCommand to have the same input structure as UpdateATC
It's dangerous to have a blank prefix delete all tokens and this allows for some code unification.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250493453
2019-05-30 12:52:21 -04:00
guyben
44ccd45439 Fix bazel breakage caused by []
Although the GSON version in repositories.bzl was 2.8.5, the same name
(com_google_code_gson) was apparently used internally for version 2.7 for some
dependency of com.google.closure (soy?)

Changed the name, and it solved the problem. The new name is also "more
correct" as the package is com.google.gson, not com.google.code.gson.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250313017
2019-05-30 12:52:21 -04:00
guyben
c79e0ea670 Conform to RDAP Response Profile 15feb19
This is only about the Response Profile, not the Technical Implementation guide.

The Response Profile can be found at https://www.icann.org/en/system/files/files/rdap-response-profile-15feb19-en.pdf

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250277559
2019-05-30 12:52:21 -04:00
cpovirk
b34a828b71 Migrate Correspondence subclasses to instead call Correspondence.from()
This makes the code shorter, and the subclassing approach is deprecated.

Open-source note: Correspondence.from was added to Truth 0.43. (So was Correspondence.transforming, which I'm using occasionally instead.) Correspondence.formattingDiffsUsing was added in Truth 0.44.

END_PUBLIC

More information:
  []Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249913322
2019-05-30 12:52:21 -04:00
guyben
7cc5c16ab9 Increase delay between attempts of rde staging
Our database is big enough now that it takes > 4 hours to run the RDE Staging mapreduce.

Retrying after 4 hours just causes a new mapreduce to run, interfering with the old one.

8 hours still means 3 attempts per 24 hour period (00:07, 08:07, 16:07) so we can catch up with the cursor if needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249829914
2019-05-30 12:52:21 -04:00
jianglai
5f7e725c29 Add the ability to turn off dependency locking
This makes it easy to experiment with new dependency versions or new plugins
without having to update the lock files, by adding "-PdisableDependencyLocking=true" to any gradle command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249463575
2019-05-30 12:52:21 -04:00
jianglai
243fce84ce Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-30 12:52:21 -04:00
jianglai
706b47c499 Upgrade to AutoValue 1.6.3
This fixes linter errors when running gradle commands. However the investigation into the linter error revealed potential problematic behaviors with dependency resolution in Gradle. See the attached bug for details.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249256283
2019-05-21 16:48:28 -04:00
jianglai
f2012b73de Copy META-INF to GAE deployment jar
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=249238153
2019-05-21 11:47:49 -04:00
gbrodman
7fd1ad1dc6 Compile all Soy JS files
I was testing this to see what the error was and I can't find one, so I assume it must have just been fixed at some point

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248551972
2019-05-21 11:44:26 -04:00
mmuller
f18f389b03 Increase NS record default TTLs to 3 hours
Per further discussion on b/130533696, we've decided to increase these so
tenants can have greater resilience to an outage in our DNS servers.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248546853
2019-05-21 11:42:39 -04:00
cpovirk
73f7e185f2 Migrate users from the old, deprecated Subject.fail* methods to the new Subject.fail* methods or, in some cases, to Subject.check.
Most of the changes in this CL were made manually.

I've tried to preserve all information (and of course behavior!), but the format and grammar of the messages does change. For example before-and-after messages, see the LSC doc.

In some of the CLs in this round (e.g., jscomp), I don't know a lot about the domain being tested, and the assertions are complex, so please let me know if my new phrasing is wrong or confusing.

Thanks again for your patience with all the Truth changes lately.

END_PUBLIC

More information:
  []

Tested:
    TAP for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248543745
2019-05-21 11:40:53 -04:00
gbrodman
6ce74ff61b Delete the AbstractDomainBaseSubject
It only had one subclass and it unnecessarily complicates things. It originally existed because DomainBase was abstract and could have multiple implementations but that is no longer the case.

This will also help with []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248529389
2019-05-21 11:39:08 -04:00
gbrodman
8f9f384406 Add a small prompt to the UpdateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248523681
2019-05-21 11:37:22 -04:00
jianglai
b12a462f5e Add Google Analytics to registrar console
To support the open source community, which may want to use different analytics services, we implement a soy template for analytics services that:

1) Does not require users to implement Google Analytics
2) Allows users to add their own analytics code to `Analytics.soy`
3) Gives users the flexibility to pass as much or as little static configuration to their custom analytics code as needed.
4) Ensures that users can merge upstream Nomulus code in the future without having to delete their custom analytics implementations
5) Does not allow code to be injected from configuration, which Soy as a framework actively discourages.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248340081
2019-05-21 11:35:34 -04:00
gbrodman
7f69ebc5d9 Add a bit more verification text when unrenewing domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=248150595
2019-05-21 11:33:47 -04:00
guyben
a0040c5eda Save the RDAP request time globally instead of passing it around
Also removed the rdapWhoisServer value, as it's just null and will likely stay that way (it isn't mentioned in the RDAP response profile)

If it'll ever become required, we can add it back.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247643981
2019-05-13 12:44:55 -04:00
shicong
b10f880b7f Add source code and JAR file to internal repository
This CL made the following changes to achieve the purpose:

1. Modified HttpProxy.java to generate a file containing Maven
   coordinates for all requested JAR files. This is to replace
   Gradle task "generateMavenCoordinateForDependency" which served
   for the same purpose before. The reason is during the course
   of the build, Gradle may request some POM file of a artifact
   for some reason but it doesn't actually need the JAR file.
   However, task "generateMavenCoordinateForDependency" cannot
   identify this use case so it added more JARs to the dependency
   which are not necessary and are not consistent with what we
   get from HttpProxy.java. Going forward, we will use HttpProxy.java
   as the single source to provide dependency.

2. Updated update_deps_metadata.py to retrieve *-sources.jar for each
   dependency for obvious reason.

3. Improved update_dependency.sh by removing 1 unnecessary build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247607871
2019-05-13 12:41:31 -04:00
gbrodman
aa83472393 Remove the scrap AllocationToken command and OnLoad method
I have resaved all the tokens and verified in BigQuery that there are no
remaining null tokenTypes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247425132
2019-05-13 12:39:47 -04:00
cpovirk
5bb93e9e4a Instead of calling Subject.actual(), store the actual value in a field, and read that.
actual() is being removed.

More information:
  []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247415370
2019-05-13 12:38:09 -04:00
guyben
38e2175699 Convert RDAP tests from simple.json to Gson
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247237275
2019-05-13 12:36:31 -04:00
guyben
3b8a8892bb Inject RdapAuthorization
We currently create it from injected arguments, and pass it to every function.

Instead, we just create a provider for it and inject it where needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247072517
2019-05-13 12:34:53 -04:00
jianglai
ce7456ea66 Do not track IntelliJ files in SCM
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=247012595
2019-05-13 12:33:15 -04:00
guyben
bdc41edd34 Reimplement the RDAP Json creation using Jsonables
Currently we try to reimplemnet the same behavior of the existing code as much
as possible.

We only fix issues that go against the RFC7483, but we don't yet update the
code to follow the latest (15feb19) RDAP Response Profile. That will require a
much bigger change especially for the test files, so it'll wait for a followup
CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246948018
2019-05-13 12:31:32 -04:00
shicong
e382299212 Resolve carry-over state issue in TestServerRule
The TestServerRule object is shared between retries in the same test,
so the testServer object constructed in TestServerRule's constructor
will be shared as well. This should be the reason why the test retry
carries over some state. (The log in the test proves that the
testServer object is shared because it listens to the same port in
all retries, which should not happen if its constructor is invoked
every time. You can find multiple "TestServerRule is listening on:
[]in this test
[]

So, this CL delayed the construction to rule.before() method which is
invoked before every retry.(You can see each retry has a test server
listening to different port and the error is "It differed by <16> pixels."
for all attempts instead of not clickable button []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246904446
2019-05-13 12:29:53 -04:00
weiminyu
d1b7824cc2 Refactor Gradle project setup
Make root build.gradle easier to read:
- Moved appengine war-assembly into separate script
- Moved java static analysis configs to separate script,
  for easier sharing with buildSrc folder.

Update Gradle config in the buildSrc folder:
- Applied static analysis, and cleaned up a few style and
  dependency lint errors.
- Added dependency locks
- Set up Google java format enforcement (Spotless)
  since this is new code.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246858172
2019-05-06 17:03:18 -04:00
jianglai
d4698ae386 Fix Bazel build error
Bazel's GenTestRules does not support the "timeout" parameter, introduced in []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246830344
2019-05-06 17:00:03 -04:00
jianglai
1dcbfae314 Remove failing tests from Nomulus GCB pipeline
Also upgrade to Gradle 5.4.1.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246829399
2019-05-06 16:58:15 -04:00
gbrodman
3483f348dc Add a command to update allocation tokens
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824258
2019-05-06 16:56:33 -04:00
gbrodman
df7e9a1225 Validate on-load that an AllocationToken can be used
Check the timing (that is, whether or not we're in a promotion), the allowed registrar client IDs, and the allowed TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824080
2019-05-06 16:54:55 -04:00
cpovirk
1480181fe1 Migrate from containsAllOf to containsAtLeast
The two behave identically, and containsAllOf is being removed.

More information:
  []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246791491
2019-05-06 16:53:03 -04:00
jianglai
e1f8817746 Tune timeout for test targets ([]
This CL looks good? Just LGTM and Approve it!
This CL doesn't look good? File a bug using []

What else can you do?
* Suggest a fix on the CL ([]
* Revert this CL, by replying "REVERT: <provide reason>".
* Revert this CL and not get a CL that cleans up this file in the future by replying "BLACKLIST: <provide reason>". This action is not reversible!

This CL was automatically generated by ATTT ([] using CL Shepherd ([] Please contact [] for questions.

//third_party/java_src/gtld/javatests/google/registry/model:registry/label/GenrulePremiumListTest: We're adjusting the test target timeout because it has a 1.248050 percent time out rate based on last week's (2019-04-27 - 2019-05-04) tap postsubmit running statistics. Its original timeout_sec is 60, and we're adjusting it to moderate, which is 300 seconds.
Some detailed statistics for the target:
  Num of invocation: 641
  Num of passed invocations: 633
  Num of timeout invocations: 8
  ****** Stats for passing runs ******
    Min run time in secs: 14
    Max run time in secs: 57
    Mean run time in secs: 28.955766
    Sd run time in secs: 9.104767

#codehealth

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246791200
2019-05-06 16:50:39 -04:00
cpovirk
021405eb17 Migrate Truth Subjects from no-arg check() to use description
The overload that accepts a description generally produces better failure messages:
- The first line of the message it produces is something like: "value of: myProto.getResponse()" (where "getResponse()" is taken from the provided description)
- The last line of the message it produces is something like: "myProto was: response: query was throttled" (the full value of myProto)
- And the existing text goes in between.

Additional motivation: We are deleting the no-arg overload externally (and probably internally thereafter).
Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246582329
2019-05-06 16:49:01 -04:00
gbrodman
e9d220e6f3 Add promotional fields in GenerateAllocationTokensCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246579125
2019-05-06 16:47:25 -04:00
gbrodman
c9ee5c3fb3 Add a scrap command to re-save all allocation token entities
This will populate the token type to be SINGLE_USE for existing tokens that
don't have a type.

Modeled after the ResaveEntitiesCommand, but modified so that we load all AllocationTokens (rather than having to provide a list of entity IDs on the command line)

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246572756
2019-05-06 16:45:46 -04:00
cpovirk
251ec1ed32 Migrate from containsAllIn to containsAtLeastElementsIn
The two behave identically, and containsAllIn is being removed.

More information:
  []
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246565417
2019-05-06 16:44:06 -04:00
cpovirk
7cb6d23884 Migrate from is(Not)SameAs to is(Not)SameInstanceAs.
They behave identically, and the old names are being removed.

Open-source note: The new methods are available in Truth as of version 0.44.

END_PUBLIC

More information:
  []

Tested:
    TAP --sample ran all affected tests and none failed
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246550001
2019-05-06 16:42:30 -04:00
weiminyu
e6e6303b9e Update to Truth 0.44 in Bazel build
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246548162
2019-05-06 16:40:54 -04:00
weiminyu
3145770f7c Upgrade truth to v0.44
Also upgraded Guava to 27.1-jre and truth java8 extension to v0.44.

Also removed lockfiles no longer in use in Gradle 5.4.

Follow up change will update bazel dependencies.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246515393
2019-05-06 16:39:17 -04:00
jianglai
cdb9a820e9 Update to bazel 0.25.0
Also updated the docker rules to be compatible with the new version of bazel,
and updated the proxy base image to the latest Java 8 version.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246509816
2019-05-06 16:36:04 -04:00
guyben
5962c9c561 Fix Kokoro failure
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246411007
2019-05-06 16:34:27 -04:00
gbrodman
625e613b07 Upgrade the version of google-closure-library that we use in NPM
This is the version that's available in internal Artifactory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246392223
2019-05-06 16:31:14 -04:00
guyben
e1926dbc57 Add Jsonable and AbstractJsonableObject for easier RDAP object building
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246345611
2019-05-06 16:27:59 -04:00