Since HistoryEntry always represents a read-only log of mutation to a core resource, that mutation should always happen in a transaction, and the HistoryEntry should be saved in that transaction. As such, it's always more accurate to use ofy().getTransactionTime() for the modificationTime of the HistoryEntry rather than just DateTime.now(UTC).
In addition, having these be the exact same timestamp makes it possible to align HistoryEntries with commit log manifests using modificationTime = transactionTime, which is useful for recovery and analysis purposes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170136957
Mostly based on the original PR, but with some tweaking by nfelt@, in particular to add some support for autorenew grace period subsumption.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169922894
Based on the original pull request below with some modifications for code drift
over time (including adding handling/testing for the case where superordinate
domains are in pending delete, and thus can't have hosts linked to them).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=169698157
The lock is used to prevent dual writing of the same output file, and to
prevent us from generating the same data twice (if RdeStaging was run twice).
However, at times when we're creating both RDE and BRDA reports - we use the
same lock for both even though they write to different files and create
different reports.
This causes one of the two to almost always fail (when both are created, which
is once a week I think) delaying that report by several hours.
Note that the "prefix" part of the filenames has the mode in it, so different
modes will not have the same files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=168882007
Allow superusers to change the transfer period to zero years and allow
superusers to change the automatic transfer length.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=167598314
It was buggy (didn't work) and was never actually used.
Why never actually used: for it to be used executeWithLock has to be called
with different requesters on the same lockId. That never happend in the code.
How it was buggy: Logically, the queue is deleted on release of the lock (meaning it was
meaningless the only time it mattered - when the lock isn't taken). In
addition, a different bug meant that having items in the queue prevented the
lock from being released forcing all other tasks to have to wait for lock
timeout even if the task that acquired the lock is long done.
Alternative: fix the queue. This would mean we don't want to delete the lock on release (since we want to keep the queue). Instead, we resave the same lock with expiration date being START_OF_TIME. In addition - we need to fix the .equals used to determine if the lock the same as the acquired lock - instead use some isSame function that ignores the queue.
Note: the queue is dangerous! An item (calling class / action) in the first place of a queue means no other calling class can get that lock. Everything is waiting for the first calling class to be re-run - but that might take a long time (depending on that action's rerun policy) and even might never happen (if for some reason that action decided it was no longer needed without acquiring the lock) - causing all other actions to stall forever!
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163705463
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).
This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
[] opened up the settings on RdeStaging, in order to make it usable by the nomulus tool. But in retrospect, we think that all we needed to do was support the POST method, not loosen the auth settings, since nomulus invokes RdeStaging via a task queue. Removing the looser auth settings will bring this action into line with other backend actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157262595
This replaces the memcache caching, which we think is overall a bad idea.
We load all registrars at once instead of caching each as needed, so that
the loadAllCached() methods can be cached as well, and therefore will
always produce results consistent with loadByClientIdCached()'s view of the
registrar's values. All of our prod registrars together total 300k of data
right now, so this is hardly worth optimizing further, and in any case this
will likely reduce latency even further since most requests will be
served out of memory.
While I was in the Registrar file I standardized the error messages for incorrect
password and clientId length to be the same format, and cleaned up a few
random things I noticed in the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156151828
We will not want to run this under normal circumstances, but in cases (such as PDT testing in sandbox) where it's desirable to generate an escrow deposit even if it isn't technically valid XML, this gives us that option. Manual-mode RDE generation is also changed so that, if no watermark date is specified, it defaults to the previous midnight, to better support running of RDE in sandbox to catch data problems.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156108295
This change means that RdeStagingAction now accepts POST requests and supports the API auth method, which is needed for the new GenerateEscrowDepositCommand (as of [] to be able to invoke the action in manual mode. Note that this shouldn't affect the cron setting that invokes it currently, which are GET requests with INTERNAL auth.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154764870
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
RdeStagingAction always processed all RDE and BRDA deposits currently outstanding, updating the cursors appropriately and kicking off the upload job. Sometimes we don't want all that. We just want to create a specific deposit by hand, without modifying the cursors or uploading. This CL adds parameters to support that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152415959
Now that transfers are always restricted to 1 year, it's unnecessary to store
extendedRegistrationYears on TransferData - it will always be equal to 1. This
simplifies logic in a few other places, e.g. RdeDomainImportAction.
I verified in BigQuery that no DomainBases exist with extendedRegistrationYears
values that aren't either null or equal to 1. At some point we should remove
the persisted fields from datastore via e.g. resaving all those domains, but
it's low priority and can wait until we have some more pressing migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150373897
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.
Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
* Remove LINKED when loading an EppResource
* Enforce that you can't add it to a resource
* Ignore LINKED on xjc import of contacts and hosts
After running ResaveAllEppResourcesAction we will no
longer have persisted LINKED statuses in datastore.
In the process of writing this I discovered that RDAP
treats LINKED like any other status value and returns
the persisted value rather than the derived one. Since
this is an existing bug and is orthogonal to the changes
in this CL, I am addressing it in a separate CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145585227
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
The next step will be to get rid of RegistryConfig descendants and RegistryConfigLoader entirely.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143812815
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072
We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018
A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55
The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.
Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:
- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
We experience intermittent exceptions when connecting to Iron Mountain. Just retrying is probably the easiest solution to the problem.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141221179
With some code cleanup/refactoring/formatting by Ben McIlwain.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141199855
Because RDE reads from the commit logs to get its data, the bad transfer data resulting from a bug in the system is currently preventing proper RDE operation even for domains which have been fixed in Datastore. This CL adds a check to see if the transfer data is messed up (i.e. it doesn't have valid gaining and losing registrars), and if so, omits it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141074673
RDE logs somewhat inscrutable errors when XML validation of the output fails due to bad data in Datastore. This CL adds a little more information to help us track down this sort of problem in the future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140872462
It's likely that either 1) we should be doing this for more than a
SocketTimeoutException - we probably want to do this in the case of a number
of different transient failures 2) we don't want to do this at all because it
happens in a background task that will get re-run anyway.
In any case, this seems like the right fix and it addresses a problem we see
occassionally.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139935190
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
