In RFC 5730, clTrid is specified as optional. We ran into an error earlier this
year in which a registrar was not passing a client transaction id and we didn't
handle it correctly. So, this CL adds some tests of common EPP operations verify
that they work correctly when the clTrid is not specified.
This also slightly improves some flow logic to make it more obvious at first
glance that clTrid is indeed optional.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202000845
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
This is one last hanging piece of work left over from last year's Java 8
migration. There's no functionality changes in this CL, just refactoring.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201947600
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
Also explicitly state that contacts missing GAE-UserId can't access the
registrar console
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402953
This is consistent with how we treat RESTORE billing events as well- in
general, fees are considered to be amortized over the course of a year (by the
invoicing team).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199684843
I ran into this while writing some other code and having the exception message
would have made it easier to debug.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199292321
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.
No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.
Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
You don't want to use the cache when loading them for the purposes of updating
them, but you definitely do still want to use the cache when checking the
price of individual domains.
In [] the cache clearing of premium lists on update was removed. This
is a good thing in aggregate because the cache is per-instance and thus
misleading, but it also caused us to not be able to update the same premium
list twice within an hour because the second update would hit a "PremiumList
was concurrently edited" exception, owing to first loading the stale version
from the cache for the purposes of updating it. Now we bypass the cache for
that purpose.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197768142
The issue is that the premium list cache is configured to persist for 60
minutes. So after updating the list, checks/creates for up to the next 60
minutes could still be referring to the old list. That's fine and dandy, unless
you delete the old premium list immediately (*bad*), which makes all domains
appear to now be non-premium for as long as the cache lasts. The reason deleting
the premium list entries makes names appear as non-premium is that a load-by-key
existence check with the domain label itself is used to determine if a name is
premium.
I also removed a misleading cache update statement, which doesn't do what it
appears to be doing (it appears to fix this issue) because cache is
instance-level, and so even if the premium list were updated from the frontend
instance only one of 100 instances would have its cache updated. But it's
updated from the tools service anyway, so it's guaranteed to not be a shared
cache with any instance serving EPP traffic.
On a sidenote, I introduced this bug on 2014-10-27 in [] The domain
label list refactor was my Noogler project.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=197033604
This allows list_domains to continue working for large TLDs.
TESTED=Deploys to alpha and it works to list the most recently created domains even
on a TLD with a huge number of domains on it (much more than .app has currently).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=196717389
Currently we determine fee type from the fee extension description by checking if the format string of the FeeType contains the description we received. The formatting string for EAP is "Early Access Period, fee expires: %s", so the fee description generated by a domain check command, like "Early Access Period, fee expires: 2022-03-01T00:00:00.000Z", is not recognized as EAP.
This CL adds the ability to add arbitrary extra description strings to the FeeType for a description to match against. It also changes the match to "the given description contains any of the strings from the list of format string plus extra description strings".
For EAP, we added an extra description string "Early Access Period", so any fee description that contains "Early Access Period" will be matched to EAP FeeType, including the specific description (that contains the expiry time) that we send in a domain check.
Also improved error message on multiple fee type matching.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194149162
Per EPP RFC 5730, the <clTRID> element is optional. However, we weren't handling
it not being specified in asynchronous contact/host deletions because we were
adding it directly as a parameter value on a task, which does not allow null and
thus threw a NullPointerException.
This fixes handling for nulls (the parameter isn't set at all) and adds a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194123259
This should prevent having issues with hot key paths on entities that
experience a heavy WHOIS volume (e.g. contacts that registrars reuse on
many domains).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191506124
Also increases the number of commit log buckets on alpha to 397 and correspondingly
reduces the frequency of commit log diff exporting to once every 3 minutes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191440586
Implement a checkbox in the "Resources" tab to allow registrars to toggle
their "premium price ack required" flag.
Tested:
Verfied the console functionality by hand. I've started work on an
automated test, but we can't actually test those from blaze and the
kokoro tests are way too time-consuming to be practical for development, so
we're going to have to either find a way to run those locally outside of
the normal process or make do without a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190212177
This simplifies calculating the overall invoice by giving RESTORE fees a
period equal to the period of the associated RENEW (1 year). Older
BillingEvents will not be backfilled, and will have periodYears = null.
Invoicing and business both agree this is a valid representation, since RESTORE fees are intrinsically tied to the 1-year RENEW it's associated with.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188041777
Changed SUNRISE to START_SUNRISE and added a registry/registrar pair for testing EAP. The EAP period is set to 2018-03-01 to 2022-03-01 with a price of $100.
A temporary flag is added to only create EAP registry/registrar pair so that we can update existing registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187897405
It was nullable all along, but wasn't tagged as such, and thus it was
possible to misuse the method from its call sites.
Also adds an assertion about no NORDN tasks being enqueued in a failing
domain create test for a required signed mark.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187649865
This enables sharded DNS publishing on a per-TLD basis. Instead of a TLD-wide lock, the sharded scheme locks each update on the shard number, allowing parallel writes to DNS.
We allow N (the number of shards) to be 0 or 1 for no sharding, and N > 1 for an N-way sharding scheme. Unless explicitly set, all TLDs default to a numShards of 0, so we don't have to reload all registry objects explicitly.
WARNING: This will change the lock name upon deployment for the PublishDnsAction from "<TLD> Dns Updates" to "<TLD> Dns Updates shard 0". This may cause concurrency issues if the underlying DNSWriter is not parallel-write tolerant (currently all production usages are ZonemanWriter, which is parallel-tolerant, so no issues are expected).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187525655
Also changed the name of "verifyRegistryStateAllowsLaunchFlows" to "verifyRegistryStateAllowsApplicationFlows", because there are now launch flows that don't use applications (start-date sunrise).
Finally, added a test to showcase the "super-user" power that EPPs with Anchor Tenants have. There's no change in behavior in that regard in this CL - we just add a test to make it explicit.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187517199
After investigating common domain create/update command usage
patterns by registrars, we noticed that it is frequent for a
given registrar to reuse both hosts (using a standardized set of
nameservers) as well as contacts (e.g. for privacy/proxy
services). With these usage patterns, potential per-registrar
throughput during high volume scenarios (i.e. first moments of
General Availability) suffers from hitting hot keys in Datastore.
The solution, implemented in this CL, is to add short-term
in-memory caching for contacts and hosts, analogous to how we are
already caching Registry and Registrar entities. These new
cached paths are only used inside domain flows to determine
existence and deleted/pending delete status of contacts and
hosts. This is a potential loss of transactional consistency, but
in practice it's hard to imagine this having negative effects, as
contacts or hosts that are in use cannot be deleted, and caching
would primarily affect widely used contacts and hosts.
Note that this caching can be turned on or off through a
configuration option, and by default would be off. We'd only want
it on when we really needed it, i.e. during a big launch.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=187093378
Currently we validate the fee extension by summing up all fees present in the extension and comparing it against the total fee to be charged. While this works in most cases, we'd like the ability to individually validate each fee. This is especially useful during EAP when two fees are charged, a regular "create" fee that would also be amount we charge during renewal, and a one time "EAP" fee.
Because we can only distinguish fees by their descriptions, we try to match the description to the format string of the fee type enums. We also only require individual fee matches when we are charging more than one type of fees, which makes the change compatible with most existing use cases where only one fees is charged and the description field is ignored in the extension.
We expect the workflow to be that a registrar sends a domain check, and we reply with exactly what fees we are expecting, and then it will use the descriptions in the response to send us a domain create with the correct fees.
Note that we aggregate fees within the same FeeType together. Normally there will only be one fee per type, but in case of custom logic there could be more than one fee for the same type. There is no way to distinguish them as they both use the same description. So it is simpler to just aggregate them.
This CL also includes some reformatting that conforms to google-java-format output.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=186530316
The START_DATE_SUNRISE phase allows registration of domains only with a signed mark. In all other respects - it is identical to the GENERAL_AVAILABILITY phase.
Note that Anchor Tenants bypass all checks, and are hence able to register domains without a signed mark.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185534793
It's been long enough since the format change adding in years that all
registrars should no longer have any IDs in the old format lying around
that they're still attempting to ACK. All poll messages have already been
coming back to registrars with the new format for months now.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184714735
When enabled for a registrar, all EPP operations on premium domains that have
costs (e.g. creates, renews, transfers) will fail unless the EPP fee extension
is used to explicitly ack the amount of fee as part of the EPP transaction.
This ack is required regardless of whether premium fee acking is required at
the registry level. No data migration is necessary since false is the desired
default for this new attribute.
This CL also contains some slight refactoring of static utility methods used to
perform fee verification; there was short-circuiting at call-sites in two
places when what was really needed was two methods, one implementing additional
functionality on top of the other, and calling the inner method in the places
where short-circuiting had previously been necessary.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184229363
To make FOSS build compile, third_party vendoring rules for jaxb are added to package all jaxb related targets imported from maven into a uber jar, mirroring the same practice done in //third_party/java/jaxb
Cloned from CL 182666460 by 'g4 patch'.
Original change by cushon@cushon:rosie182283995-0071_Rosie:47348:citc on 2018/01/20 13:36:15.
More information:
https://docs.google.com/document/d/1htErgDIoHMEuMBfGwrtS_O4WwhTw8QOGLva-7aYYvYs/edit?usp=sharing
Tested:
TAP --sample for global presubmit queue
[] passed FOSS test
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182855173
The next step is to add them for domain checks as well (which is simpler
because it doesn't involve validation).
This requires the addition of a TrimWhitespaceAdapter for XML JAXB objects,
which will prove useful for other @XmlValue attributes in the future.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181526726
The extension itself isn't used for anything yet; allocation tokens can be
passed for domain creates and checks but are ignored if present. This will
be changed in a subsequent CL that adds AllocationToken entities and related
logic. Usage of this extension in any other EPP flow will throw an
UnsupportedExtensionException.
The relevant spec is https://tools.ietf.org/html/draft-gould-allocation-token-04
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181343188
Logic actually using this entity to follow in subsequent CLs introducing
the command to generate/save these entities as well as the flow logic for
considering them during domain EPP operations.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181342600
Stream.concat only accepts 2 parameters. Streams.concat on the other hand
accepts any number of parameters.
Moving to Streams.concat for all uses (2 or more) makes sense for uniformity
and convenience reasons.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179716648
This also incorporates general improvements and additions to the existing EPP
lifecycle tests around domain deletion. As a refresher: There is a 5 day
add grace period (AGP) following domain creation. Domains that are deleted
during that period have their create costs (but not EAP costs) refunded. This
deletion takes place immediately. Refunds are implemented by issuing a
Cancellation for the associated OneTime billing event.
Domains that are deleted after AGP ends first go through a 30 day redemption
grace period followed by a 5 day pending deletion period. No create fees are
refunded in this case.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=179597874
