Commit graph

69 commits

Author SHA1 Message Date
mcilwain
ed0670b614 Remove unnecessary type specifications
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179585680
2017-12-27 11:23:03 -05:00
mcilwain
fbe11ff33c Use method references instead of lambdas when possible
The assertThrows/expectThrows refactoring script does not use method
references, apparently.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179089048
2017-12-27 10:55:03 -05:00
mcilwain
7dc224627f Automatically refactor more exception testing to use new JUnit rules
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178911894
2017-12-27 10:42:36 -05:00
mcilwain
03c782f38e Replace ExceptionRule with ExpectedException
This is in preparation for running the automatic refactoring script that
will replace all ExpectedExceptions with use of JUnit 4.13's assertThrows/
expectThrows.

Note that I have recorded the callsites of assertions about EppExceptions
being marshallable and will edit those specific assertions back in after
running the automatic refactoring script (which do not understand these).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178812403
2017-12-13 12:43:45 -05:00
mcilwain
b825a2b5a8 Get rid of custom ExceptionRule methods
The only remaining methods on ExceptionRule after this are methods that
also exist on ExpectedException, which will allow us to, in the next CL,
swap out the one for the other and then run the automated refactoring to
turn it all into assertThrows/expectThrows.

Note that there were some assertions about root causes that couldn't
easily be turned into ExpectedException invocations, so I simply
converted them directly to usages of assertThrows/expectThrows.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178623431
2017-12-13 12:43:45 -05:00
jianglai
1c1f95992a Move backported JUnit file to third_party (part 2)
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
2017-12-02 11:37:46 -05:00
mcilwain
e2db3f914e Clean up some code quality issues
This removes some qualifiers that aren't necessary (e.g. public/abstract on interfaces, private on enum constructors, final on private methods, static on nested interfaces/enums), uses Java 8 lambdas and features where that's an improvement

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177182945
2017-12-01 22:14:06 -05:00
mcilwain
30bfcf9c55 Remove more unused Truth8.assertThat() static imports in tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=173562809
2017-11-07 17:22:56 -05:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
guyben
840d53c819 Allow EventSample.record to accept numSamples=0
There's really no reason not to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171037754
2017-10-24 16:50:30 -04:00
mcilwain
5edb7935ed Run automatic Java 8 conversion over codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171174380
2017-10-10 12:09:41 -04:00
larryruili
578673141c Migrate to new Sheets v4 api
This moves us from the oudated google/data XML api to the OnePlatform REST/JSON api, finally silencing the deprecation warnings we've been seeing.

The synchronization algorithm diffs the spreadsheet's current values with its internally sourced values, adding the row to a batch update request if there's a discrepancy. Additional internal data are added as an append operation to the end of the sheet, and any extraneous spreadsheet data is cleared from the spreadsheet.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169273590
2017-09-20 10:27:17 -04:00
nickfelt
67116c5fa1 Remove long-dead LogsExportCursor and PersistedRangeLong
LogsExportCursor was only used by ExportLogsTaskServlet, which we removed a long time ago.  It's just dead code.  The PersistedRangeLong type was only written for use by LogsExportCursor, and since it hasn't picked up new users in 3+ years I don't think we need to keep it around.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169264994
2017-09-20 10:27:17 -04:00
jianglai
c3f2002e6c Set up github CI kokoro job
TESTED=Set up the job in kokoro-qa:

[]

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166875896
2017-09-20 10:27:17 -04:00
guyben
978149e677 Create an injectable LockHandler
We create an injectable LockHandler that just calls the static
Lock.executeWithLocks function.

I'm not sure what's the correct place to put the LockHandler. I think
model/server is only appropriate for the actual datastore lock. This is a "per request" lock, so maybe request/lock?

-----------------------------

This is the initial step in adding the "lock implicitly released on request death" feature, but it's also useful on its own - easier to test Actions when we can use a fake lock.

To keep this CL simple, we keep using the old Lock as is in most places. We just choose a single example to convert to LockHandler to showcase it. Converting all other uses will be in a subsequent CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167357564
2017-09-12 15:51:50 -04:00
guyben
aee4f7acc2 Remove queueing from Lock
It was buggy (didn't work) and was never actually used.

Why never actually used: for it to be used executeWithLock has to be called
with different requesters on the same lockId. That never happend in the code.

How it was buggy: Logically, the queue is deleted on release of the lock (meaning it was
meaningless the only time it mattered - when the lock isn't taken). In
addition, a different bug meant that having items in the queue prevented the
lock from being released forcing all other tasks to have to wait for lock
timeout even if the task that acquired the lock is long done.

Alternative: fix the queue. This would mean we don't want to delete the lock on release (since we want to keep the queue). Instead, we resave the same lock with expiration date being START_OF_TIME. In addition - we need to fix the .equals used to determine if the lock the same as the acquired lock - instead use some isSame function that ignores the queue.

Note: the queue is dangerous! An item (calling class / action) in the first place of a queue means no other calling class can get that lock. Everything is waiting for the first calling class to be re-run - but that might take a long time (depending on that action's rerun policy) and even might never happen (if for some reason that action decided it was no longer needed without acquiring the lock) - causing all other actions to stall forever!

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163705463
2017-08-01 17:06:20 -04:00
larryruili
d2cd576796 Add standardSQL views to Bigquery Datastore snapshots
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163124895
2017-08-01 17:03:28 -04:00
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
mcilwain
2521409e39 Perform some minor test cleanup
This standardizes use of annotations/inheritance/formatting across
tests, to make the code more legible and consistent.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161810734
2017-08-01 16:20:49 -04:00
jianglai
0013312f5c Export billing account map to registrar sheet
The billing account map will be serialized in the following format:

{currency1=id1, currency2=id2, ...}

In order for the output to be deterministic, the billing account map is stored as a sorted map.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161075814
2017-07-10 11:37:23 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
jianglai
7a3852b5a5 Add visibleInWhoisAsAbuse field in RegistrarContact
Also added corresponding getters and setters for the new field. Note that
nothing has changed on the RDAP front for now, as the CL&D only concerns WHOIS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155116134
2017-05-17 11:27:24 -04:00
guyben
93c2a1e4f0 Make JSch injection lazy
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154958750
2017-05-17 11:22:53 -04:00
mountford
e00b0440b2 Roll back accidental inclusion of non-public Mockito helper methods
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154232020
2017-04-26 11:19:59 -04:00
mountford
cb145e0721 Remove some unnecessary uses of MockitoJUnitRunner, which is discouraged
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154202151
2017-04-26 11:15:07 -04:00
mountford
d35be27b65 Remove some unnecessary uses of MockitoJUnitRunner, which is discouraged
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153589104
2017-04-26 10:42:43 -04:00
mcilwain
1751ea0b98 Replace deprecated Truth hasMessage() method with hasMessageThat()
hasMessageThat() was added in Truth 0.32 and we are already on that
version in the Nomulus release.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153095111
2017-04-26 10:35:23 -04:00
mcilwain
b03bd3b525 Remove UNRESERVED as a reservation type
This is a follow-up to Lai's refactoring of the get reservation types
code to return a set rather than a single type. Since we're always
returning a set now, the more natural way to represent a label that is
not reserved is to return an empty set rather than a set containing
UNRESERVED.

Also fixes some minor style issues I ran across regarding static
importing and test method naming that I ran across (no logic
implications).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151132116
2017-03-27 13:31:33 -04:00
shikhman
be30ecdf66 Add Cloud KMS based secret storage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147791972
2017-02-17 12:22:12 -05:00
mcilwain
e8c5720826 Save bloom filters for premium list entries
This is the first step in the migration to remove the need to load all of
the premium list entries every time the cache expires (which causes slow-
downs). Once this is deployed, we can re-save all premium lists, creating
the bloom filters, and then the next step will be to read from them to
more efficiently determine if a label might be premium.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147525017
2017-02-17 12:15:02 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
mcilwain
61230b035c Finish YAMLification of last necessary config values
There are still some options in RegistryConfig that can't be configured
in YAML, but it's not clear why anyone would need to change them from
their default values.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146482435
2017-02-03 12:37:38 -05:00
mcilwain
636da9f7f0 Convert yet more configuration options to YAML
With a particular focus on custom logic and caching.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146258446
2017-02-02 17:05:28 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
jianglai
4fed3a9ae6 Daggerize ExportSnapshotServlet and CheckSnapshotServlet
Eradicate the last remnants of un-injectable servlets!

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145598002
2017-01-30 15:03:53 -05:00
mcilwain
f647ea1190 Move registry policy settings and some others into YAML config
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145571850
2017-01-30 15:03:53 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00
mcilwain
d3397e991e Remove the old RegistryConfig paradigm entirely
We are now ready to begin configuration using YAML, mediated by ConfigModule.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143818507
2017-01-09 12:01:09 -05:00
mcilwain
c5c74961bb Make all but one field on RegistryConfig static
The next step will be to get rid of RegistryConfig descendants and RegistryConfigLoader entirely.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143812815
2017-01-09 12:01:09 -05:00
mcilwain
25a8bbe890 Deprecate more fields in RegistryConfig
This primarily addresses issues with TMCH testing mode and email sending utils.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143710550
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
mcilwain
f0d2f96c26 Clean up some more TLD-specific code
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142300719
2016-12-19 11:09:20 -05:00
mcilwain
352ef13160 In ExportConstantsTest, write path to golden file instead of contents
In my previous refactoring I failed to realize that getResource() was not
returning the contents of the file.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142065550
2016-12-19 11:09:19 -05:00
mcilwain
1ed1ba20ba Refactor ExportConstantsTest to reduce duplicated code
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142047229
2016-12-19 11:09:19 -05:00
Tom Johnson
b0ebeed5a5 Add @ReportedOn annotation for BigQuery exports
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142038227
2016-12-19 11:09:11 -05:00
mcilwain
2620097599 Retry attempted syncs to Google Groups that fail
I also moved to a non-concurrent modification syncing model. It was adding more
complexity than was justified just to have two requests going simultaneously
instead of one. The API doesn't reliably allow much more than that anyway.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141210192
2016-12-07 15:19:43 -05:00
mcilwain
2b7d580bb3 Run buildifier on codebase to format BUILD files
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140362453
2016-11-28 18:15:21 -05:00
mcilwain
274b070b54 Make driveFolderId on Registry entity optional
This defaults to null, and leaving it to null now simply disables reserved terms
exporting, rather than throwing an error every time the action runs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138763161
2016-11-15 10:38:20 -05:00
mcilwain
cef07f6bc5 Use cursor to track updating of registrar sheet
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138221931
2016-11-10 11:21:29 -05:00
ctingue
b7a98451e6 Add retry logic for SheetSynchronizer
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=138211996
2016-11-10 11:18:25 -05:00