Commit graph

12 commits

Author SHA1 Message Date
mcilwain
ded40851d3 Use locking on async mapreduces
This ensures that only one will run at a time, which should help fix the
clogged up mapreduces we've seen on sandbox.

In order to do this, the UnlockerOutput is introduced. This unlocks the
given Lock after all reducer shards have finished.

Also increases the lease duration of the DNS refresh action from 20 to
240 minutes. 20 minutes isn't long enough; when there's a lot of domains
and decent system load the mapreduce could take longer than that in the
ordinary case.

TESTED=Deployed to alpha and verified that more than one copy of the
mapreduce wouldn't run simultaneously, and also that the lock is
released when the mapreduce is finished.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205887554
2018-08-10 13:44:25 -04:00
mcilwain
43ed2cd7b3 Clean up annotation imports in BUILD files
This affects JSR305, JSR330, and Guava annotations.

The exact command run to generate this CL was:

build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
2018-06-27 15:28:53 -04:00
mcilwain
7d3cb3d426 Make our Clock util Serializable
It doesn't entirely make semantic sense, since the actual state of the
SystemClock isn't being preserved, but it makes injection into serializable
classes (e.g. mapreduces) much simpler, so it's worth doing.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201755949
2018-06-27 15:28:52 -04:00
mcilwain
8245d2f1c4 Make LockHandlerImpl.clock transient
SystemClock isn't Serializable (for obvious reasons), whereas LockHandlerImpl is
used as a field on some Serializable mapreduce classes. So mark it transient and
then re-generate it on first use following de-serialization when it happens to
be null.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201707209
2018-06-27 15:28:52 -04:00
mcilwain
f971583dc0 Handle timeouts better in lock handler
Rather than just logging a generic TimeoutException, this will say what action
timed out and how long it had been executing for.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201049025
2018-06-18 18:15:11 -04:00
jianglai
70b13596e4 Migrate to Flogger (green)
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.

No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.

Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.

For more information, see []
Base CL: 197826149

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
2018-05-30 12:18:54 -04:00
jianglai
fc60890136 Migrate to internal FormattingLogger in preparation of migration to Flogger
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197744904
2018-05-30 12:18:54 -04:00
mcilwain
e2db3f914e Clean up some code quality issues
This removes some qualifiers that aren't necessary (e.g. public/abstract on interfaces, private on enum constructors, final on private methods, static on nested interfaces/enums), uses Java 8 lambdas and features where that's an improvement

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177182945
2017-12-01 22:14:06 -05:00
mcilwain
c0f8da0c6e Switch from Guava Optionals to Java 8 Optionals
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
2017-10-24 16:53:47 -04:00
mcilwain
a50ef39c04 Fix mismatch in types of Predicates being used
We're going to need to switch away from Guava's Functions and Predicates for
everything and replace them with the java.util versions. Unfortunately there
does not appear to be an automated tool to do this all at once. Refaster got
close but doesn't seem to care about these particular types of mismatch (I
suspect we're using a different version of the JDK than the outside world;
ours is OK with Guava classes).

This also bumps up Guava to 0.23, which is needed for some new functionality
used in combination with Java 8 features.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170531539
2017-10-04 16:16:45 -04:00
guyben
892424b148 Check if lock owner is finished on lock acquisition
Sometimes requests "die" suddenly, without going through catch/finally blocks.
If this happens, any lock they own will remain locked until it times out (which
can take hours in some cases).

This cl implicitly unlocks any lock if the owner of the lock isn't running
anymore.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168880938
2017-09-20 10:27:17 -04:00
guyben
18a13a09a4 Add request/lock to opensource build
This is to fix Kokoro, given the directory added in [] Also, added forgotten keyring/api directory.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167598465
2017-09-12 15:51:50 -04:00