Commit graph

134 commits

Author SHA1 Message Date
mcilwain
5edb7935ed Run automatic Java 8 conversion over codebase
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171174380
2017-10-10 12:09:41 -04:00
nickfelt
3ad21e3834 Fix RDE import pending transfer handling
Mostly based on the original PR, but with some tweaking by nfelt@, in particular to add some support for autorenew grace period subsumption.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169922894
2017-10-04 16:16:45 -04:00
mcilwain
8bd9b35dfd Link domain to subordinate hosts on RDE import
Based on the original pull request below with some modifications for code drift
over time (including adding handling/testing for the case where superordinate
domains are in pending delete, and thus can't have hosts linked to them).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169698157
2017-10-04 16:16:45 -04:00
guyben
c3861f6e95 Swap all uses of Lock to LockHandler
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167661348
2017-09-12 15:51:50 -04:00
mcilwain
103b3d7608 Correct usages of DateTime.now() everywhere to DateTime.now(UTC)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=166393128
2017-08-29 17:09:53 -04:00
bbilbo
e291da7d8d Fix comment and exception message
Changed transfer request to transfer server approved.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165736915
2017-08-29 16:45:56 -04:00
mmuller
b0c254dc65 Remove RdeUploadActionTest from FOSS build
Second attempt at removing this test by simply removing from the generated
test rules.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165575517
2017-08-29 16:29:47 -04:00
mmuller
f9a42457a8 Automated g4 rollback of changelist 165340539.
*** Reason for rollback ***

Breaks kokoro build, still investigating the fix.

*** Original change description ***

Remove RdeUploadActionTest from FOSS tests

Remove from the FOSS test suite until we can figure out why it's timing out.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165359201
2017-08-29 16:18:09 -04:00
mmuller
2a1cd6490b Remove RdeUploadActionTest from FOSS tests
Remove from the FOSS test suite until we can figure out why it's timing out.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165340539
2017-08-29 16:16:44 -04:00
mmuller
7748ef072d Set "Development" environment in RDE upload test
Set SystemProperty.environment to "Development" so we don't try to actually
use GCS.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165019677
2017-08-29 16:03:56 -04:00
mmuller
5653b2f479 Fix method call that requires it's arg checked
Fix a verification call to a method that requires it's arg checked that is
breaking in the FOSS build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164974785
2017-08-29 15:59:41 -04:00
guyben
aee4f7acc2 Remove queueing from Lock
It was buggy (didn't work) and was never actually used.

Why never actually used: for it to be used executeWithLock has to be called
with different requesters on the same lockId. That never happend in the code.

How it was buggy: Logically, the queue is deleted on release of the lock (meaning it was
meaningless the only time it mattered - when the lock isn't taken). In
addition, a different bug meant that having items in the queue prevented the
lock from being released forcing all other tasks to have to wait for lock
timeout even if the task that acquired the lock is long done.

Alternative: fix the queue. This would mean we don't want to delete the lock on release (since we want to keep the queue). Instead, we resave the same lock with expiration date being START_OF_TIME. In addition - we need to fix the .equals used to determine if the lock the same as the acquired lock - instead use some isSame function that ignores the queue.

Note: the queue is dangerous! An item (calling class / action) in the first place of a queue means no other calling class can get that lock. Everything is waiting for the first calling class to be re-run - but that might take a long time (depending on that action's rerun policy) and even might never happen (if for some reason that action decided it was no longer needed without acquiring the lock) - causing all other actions to stall forever!

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163705463
2017-08-01 17:06:20 -04:00
mcilwain
d536cef20f Make Registrar load methods return Optionals instead of Nullables
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).

This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
2017-08-01 16:58:59 -04:00
bbilbo
c88a776741 Re-enable some PendingDepositChecker Tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162254143
2017-08-01 16:37:13 -04:00
mcilwain
2521409e39 Perform some minor test cleanup
This standardizes use of annotations/inheritance/formatting across
tests, to make the code more legible and consistent.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161810734
2017-08-01 16:20:49 -04:00
nickfelt
60022ae315 Remove outdated comment in RdeUploadActionTest
I think this comment was meant to be the justification for not using "localhost" (aka a hostname) in the URLs, because jsch would mangle it.  However, we already cut over to using "localhost" in [] to avoid a dependency on IPv4, and it's been fine.  So this comment no longer makes sense.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=158063880
2017-06-06 10:57:56 -04:00
mountford
74a0defef3 Add the ability to generate RDE deposits in lenient mode
We will not want to run this under normal circumstances, but in cases (such as PDT testing in sandbox) where it's desirable to generate an escrow deposit even if it isn't technically valid XML, this gives us that option. Manual-mode RDE generation is also changed so that, if no watermark date is specified, it defaults to the previous midnight, to better support running of RDE in sandbox to catch data problems.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156108295
2017-05-17 12:22:49 -04:00
mcilwain
ef1487cb57 Make name and address fields required on Registrar
The absence of these fields causes RDE failures, so they are in effect
required on any functioning registry system. We are currently
experiencing problems in sandbox caused by null values on these fields.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155474895
2017-05-17 11:44:46 -04:00
guyben
93c2a1e4f0 Make JSch injection lazy
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154958750
2017-05-17 11:22:53 -04:00
nickfelt
40fa9ff022 Fix some direct uses of ForeignKey(C/H/D)Index
Mostly these are calls to ForeignKeyIndex.create() (a static method) via subclasses, which is pretty misleading in this case since the type of the return value has nothing to do with the subclass you're qualifying the static method call with (the returned type depends only on the type of the EppResource parameter).

Note however though that while the style guide indeed prohibits qualifying static member references with things other than the class name, the subclassing case is apparently not considered subject to that prohibition in general:
https://groups.google.com[]d/msg/java-style/8ViX-Rh2_sc/48n2lz5nAAAJ

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154331605
2017-05-03 10:40:21 -04:00
mountford
89b3e9c55b Remove some unnecessary uses of MockitoJUnitRunner, which is discouraged
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=153719402
2017-04-26 10:47:13 -04:00
mountford
dea386d08a Change GenerateEscrowDepositCommand to trigger back end deposit generation
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
2017-04-13 10:57:24 -04:00
mcilwain
8653d2f204 Refactor out creation of server TRIDs so they can be tested
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152837185
2017-04-13 10:55:47 -04:00
mountford
4f94464eaf Allow RdeStagingAction to be invoked manually
RdeStagingAction always processed all RDE and BRDA deposits currently outstanding, updating the cursors appropriately and kicking off the upload job. Sometimes we don't want all that. We just want to create a specific deposit by hand, without modifying the cursors or uploading. This CL adds parameters to support that.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152415959
2017-04-10 13:47:51 -04:00
nickfelt
09f619cce2 Remove obsolete TransferData.extendedRegistrationYears
Now that transfers are always restricted to 1 year, it's unnecessary to store
extendedRegistrationYears on TransferData - it will always be equal to 1.  This
simplifies logic in a few other places, e.g. RdeDomainImportAction.

I verified in BigQuery that no DomainBases exist with extendedRegistrationYears
values that aren't either null or equal to 1.  At some point we should remove
the persisted fields from datastore via e.g. resaving all those domains, but
it's low priority and can wait until we have some more pressing migration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150373897
2017-03-21 15:35:00 -04:00
shikhman
5779cc988b Refactor RdeKeyringModule into FakeKeyringModule
A fake Keyring implementation is needed to test Keyring related tools now, so
I've refactored the fake RdeKeyring to support all Keyring API methods and
moved it to the common testing package.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149977968
2017-03-21 14:53:25 -04:00
cgoldfeder
9174855a47 Remove the ofy().load() inside of HostResource.cloneProjectedAtTime
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.

Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
2017-03-13 11:22:55 -04:00
nickfelt
93b5cfb71b Make RdeUploadActionTest compatible with IPv6-only mode
This fixes an issue where RdeUploadActionTest relies on connecting to a local dummy SFTP server via the IPv4 loopback (127.0.0.1), which causes the test to fail on machines that only support IPv6.  This updates the test to use just "localhost" instead.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148698178
2017-03-07 13:19:56 -05:00
Wolfgang Meyers
9f90597691 Handle pending transfers on domain imports from RDE deposits
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148496597
2017-02-27 11:30:40 -05:00
cgoldfeder
ab6e7b177a Fix a comment nit (can't @link to a generic type)
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147731422
2017-02-17 12:20:45 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
Wolfgang Meyers
da1f83a7b4 Add host linker mapreduce for RDE imports
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147025088
2017-02-09 14:24:27 -05:00
Wolfgang Meyers
6cdb3d81d3 Generate authInfo for imported domains
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146956307
2017-02-08 17:42:57 -05:00
Wolfgang Meyers
60a3d2a53b Synchronize domains with DNS on import
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146813124
2017-02-08 17:42:43 -05:00
Wolfgang Meyers
a904f2c6ee Add mapper to import domains from RDE deposits
With some additional cleanup by Ben McIlwain.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146695665
2017-02-06 16:46:46 -05:00
cgoldfeder
50c5f856a2 Document StatusValue better and add per-resource restrictions
This generalizes the "LINKED can't be anywhere" idea into more
targeted restrictions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146158204
2017-02-02 16:46:02 -05:00
emcmanus
a0d6a81276 Use the original class name, not the App Engine-repackaged one
(Usually the latter is because an IDE chose the repackaged name without the developer noticing.)
Tested:
    TAP --sample for global presubmit queue
    []

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146121772
2017-02-02 16:38:24 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
cgoldfeder
0b1781b110 Make LINKED into a virtual status value
* Remove LINKED when loading an EppResource
* Enforce that you can't add it to a resource
* Ignore LINKED on xjc import of contacts and hosts

After running ResaveAllEppResourcesAction we will no
longer have persisted LINKED statuses in datastore.

In the process of writing this I discovered that RDAP
treats LINKED like any other status value and returns
the persisted value rather than the derived one. Since
this is an existing bug and is orthogonal to the changes
in this CL, I am addressing it in a separate CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145585227
2017-01-30 15:03:53 -05:00
mcilwain
eaec03e670 Move ConfigModule and LocalTestConfig into RegistryConfig
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
2017-01-09 12:01:09 -05:00
mcilwain
c5c74961bb Make all but one field on RegistryConfig static
The next step will be to get rid of RegistryConfig descendants and RegistryConfigLoader entirely.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143812815
2017-01-09 12:01:09 -05:00
jart
734130aa73 Restructure Maven dependencies in build
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072

We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018

A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55

The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.

Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:

- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
2017-01-09 11:59:04 -05:00
mcilwain
28f6c770c8 Add MOE equivalence for sync on 2016-12-19
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142449539
2017-01-09 11:59:04 -05:00
ctingue
8fdf67d332 Increase rde package test size
The open source build uses a single shard for the rde package's tests. In order to account for the longer runtime, increase the timeout for this package.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=142261623
2016-12-19 11:09:19 -05:00
mcilwain
e46fdf9a87 Fix missing RDE imports files in Nomulus
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141942879
2016-12-13 18:01:39 -05:00
ctingue
50c862b7c1 Disable test sharding for rde package
The tests in this package are failing for reasons likely related to resource consumption. Sharding an expensive test within a single test runner is probably causing more harm than good, so run this test in a single shard.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141766295
2016-12-13 16:02:21 -05:00
mcilwain
a56a0677ba Refactor RDE import stuff into its own rde.imports package
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141566236
2016-12-13 15:57:46 -05:00
mcilwain
78123dfe7b Fix Nomulus build broken by invalid import of JSch
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141318398
2016-12-07 15:19:43 -05:00
mountford
50aad72383 Add retry logic to RDE upload
We experience intermittent exceptions when connecting to Iron Mountain. Just retrying is probably the easiest solution to the problem.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141221179
2016-12-07 15:19:43 -05:00
Wolfgang Meyers
f2faf49d58 Add XjcToDomainResourceConverter for RDE import
With some code cleanup/refactoring/formatting by Ben McIlwain.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141199855
2016-12-07 15:19:29 -05:00