[] opened up the settings on RdeStaging, in order to make it usable by the nomulus tool. But in retrospect, we think that all we needed to do was support the POST method, not loosen the auth settings, since nomulus invokes RdeStaging via a task queue. Removing the looser auth settings will bring this action into line with other backend actions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157262595
This replaces the memcache caching, which we think is overall a bad idea.
We load all registrars at once instead of caching each as needed, so that
the loadAllCached() methods can be cached as well, and therefore will
always produce results consistent with loadByClientIdCached()'s view of the
registrar's values. All of our prod registrars together total 300k of data
right now, so this is hardly worth optimizing further, and in any case this
will likely reduce latency even further since most requests will be
served out of memory.
While I was in the Registrar file I standardized the error messages for incorrect
password and clientId length to be the same format, and cleaned up a few
random things I noticed in the code.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156151828
We will not want to run this under normal circumstances, but in cases (such as PDT testing in sandbox) where it's desirable to generate an escrow deposit even if it isn't technically valid XML, this gives us that option. Manual-mode RDE generation is also changed so that, if no watermark date is specified, it defaults to the previous midnight, to better support running of RDE in sandbox to catch data problems.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=156108295
This change means that RdeStagingAction now accepts POST requests and supports the API auth method, which is needed for the new GenerateEscrowDepositCommand (as of [] to be able to invoke the action in manual mode. Note that this shouldn't affect the cron setting that invokes it currently, which are GET requests with INTERNAL auth.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154764870
Previously, GenerateEscrowDepositCommand generated the deposit itself. Channeling it through the existing deposit generation code make things more maintainable.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152847950
RdeStagingAction always processed all RDE and BRDA deposits currently outstanding, updating the cursors appropriately and kicking off the upload job. Sometimes we don't want all that. We just want to create a specific deposit by hand, without modifying the cursors or uploading. This CL adds parameters to support that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152415959
Now that transfers are always restricted to 1 year, it's unnecessary to store
extendedRegistrationYears on TransferData - it will always be equal to 1. This
simplifies logic in a few other places, e.g. RdeDomainImportAction.
I verified in BigQuery that no DomainBases exist with extendedRegistrationYears
values that aren't either null or equal to 1. At some point we should remove
the persisted fields from datastore via e.g. resaving all those domains, but
it's low priority and can wait until we have some more pressing migration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=150373897
In fact, completely eviscerate cloneProjectedAtTime (to be removed in
a followup CL) in favor of doing the projection of transfers and the
loading of values from the superordinate domain at call sites. This
is one of the issues that blocked the memcache audit work, since the
load inside of cloneProjectedAtTime could not be controlled by the
caller.
Note: fixed a minor bug where a subordinate host created after its superordinate domain was last transferred should have lastTransferTime==null but was previously reporting the domain's lastTransferTime.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149769125
* Remove LINKED when loading an EppResource
* Enforce that you can't add it to a resource
* Ignore LINKED on xjc import of contacts and hosts
After running ResaveAllEppResourcesAction we will no
longer have persisted LINKED statuses in datastore.
In the process of writing this I discovered that RDAP
treats LINKED like any other status value and returns
the persisted value rather than the derived one. Since
this is an existing bug and is orthogonal to the changes
in this CL, I am addressing it in a separate CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=145585227
This is the final preparatory step necessary in order to load and load
configuration from YAML in a static context and then provide it either via
Dagger (using ConfigModule) or through RegistryConfig's existing static
functions.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143819983
The next step will be to get rid of RegistryConfig descendants and RegistryConfigLoader entirely.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143812815
We're now using java_import_external instead of maven_jar. This allows
us to specify the relationships between jars, thereby allowing us to
eliminate scores of vendor BUILD files that did nothing but re-export
@foo//jar targets, thus addressing the concerns of djhworld on Hacker
News: https://news.ycombinator.com/item?id=12738072
We now have redundant failover mirrors, which is a feature I added to
Bazel 0.4.2 in ed7ced0018
A new standard naming convention is now being used for all Maven repos.
Those names are calculated from the group_artifact name using the
following algorithm that eliminates redundancy:
https://gist.github.com/jart/41bfd977b913c2301627162f1c038e55
The JSR330 dep has been removed from java targets if they also depend
on Dagger, since Dagger always exports JSR330.
Annotation processor dependencies should now be leaner and meaner, by
more appropriately managing what needs to be on the classpath at
runtime. This should trim down the production jar by >1MB. As it stands
currently in the open source world:
- backend_jar_deploy.jar: 50MB
- frontend_jar_deploy.jar: 30MB
- tools_jar_deploy.jar: 45MB
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=143487929
We experience intermittent exceptions when connecting to Iron Mountain. Just retrying is probably the easiest solution to the problem.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141221179
With some code cleanup/refactoring/formatting by Ben McIlwain.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141199855
Because RDE reads from the commit logs to get its data, the bad transfer data resulting from a bug in the system is currently preventing proper RDE operation even for domains which have been fixed in Datastore. This CL adds a check to see if the transfer data is messed up (i.e. it doesn't have valid gaining and losing registrars), and if so, omits it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=141074673
RDE logs somewhat inscrutable errors when XML validation of the output fails due to bad data in Datastore. This CL adds a little more information to help us track down this sort of problem in the future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140872462
It's likely that either 1) we should be doing this for more than a
SocketTimeoutException - we probably want to do this in the case of a number
of different transient failures 2) we don't want to do this at all because it
happens in a background task that will get re-run anyway.
In any case, this seems like the right fix and it addresses a problem we see
occassionally.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139935190
This allows separate Bazel projects to reference Nomulus as an external
repository. They can then copy the []
directory structure into their own project and customize the Action
and Module lists for the GAE modules in their own deployment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=136863886
This changes everything with external visibility beyond the codebase
(i.e. the name of the compiled binary and the documentation that refers
to it). It does not change a lot of things internal to the codebase,
i.e. the "RegistryTool" class didn't change its name. We can rename that
in a subsequent CL if we want to.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135022087
Almost all uses were in test classes, which I replaced with clock.nowUTC().
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134993149
TESTED=I deployed it on alpha, renamed some hosts, and verified that
the [] ran as expected.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=134991941
It's best to be consistent and use the same thing everywhere. "clientId" was
already used in more places and is shorter and no more ambiguous, so it's the
logical one to win out.
Note that this CL is almost solely a big Eclipse-assisted refactoring. There are
two places that I did not change clientIdentifier -- the actual entity field on
Registrar (though I did change all getters and setters), and the name of a
column on the exported registrar spreadsheet. Both would require data
migrations.
Also fixes a few minor nits discovered in touched files, including an incorrect
test in OfyFilterTest.java and some superfluous uses of String.format() when
calling checkArgument().
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=133956465
This change replaces all Ref objects in the code with Key objects. These are
stored in datastore as the same object (raw datastore keys), so this is not
a model change.
Our best practices doc says to use Keys not Refs because:
* The .get() method obscures what's actually going on
- Much harder to visually audit the code for datastore loads
- Hard to distinguish Ref<T> get()'s from Optional get()'s and Supplier get()'s
* Implicit ofy().load() offers much less control
- Antipattern for ultimate goal of making Ofy injectable
- Can't control cache use or batch loading without making ofy() explicit anyway
* Serialization behavior is surprising and could be quite dangerous/incorrect
- Can lead to serialization errors. If it actually worked "as intended",
it would lead to a Ref<> on a serialized object being replaced upon
deserialization with a stale copy of the old value, which could potentially
break all kinds of transactional expectations
* Having both Ref<T> and Key<T> introduces extra boilerplate everywhere
- E.g. helper methods all need to have Ref and Key overloads, or you need to
call .key() to get the Key<T> for every Ref<T> you want to pass in
- Creating a Ref<T> is more cumbersome, since it doesn't have all the create()
overloads that Key<T> has, only create(Key<T>) and create(Entity) - no way to
create directly from kind+ID/name, raw Key, websafe key string, etc.
(Note that Refs are treated specially by Objectify's @Load method and Keys are not;
we don't use that feature, but it is the one advantage Refs have over Keys.)
The direct impetus for this change is that I am trying to audit our use of memcache,
and the implicit .get() calls to datastore were making that very hard.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=131965491
This will help [] be submitted without breaking the linter.
License headers are now added automatically where they were previously
added by hand. We're also now adding the license header to Soy and SQL
files.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=129017424
