mirror of
https://github.com/google/nomulus.git
synced 2025-04-30 03:57:51 +02:00
Add Cursor documentation
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=133994188
This commit is contained in:
parent
b783acfcc6
commit
414781e79b
2 changed files with 61 additions and 0 deletions
|
@ -13,6 +13,43 @@ particularly important pieces of the system are implemented.
|
||||||
|
|
||||||
## Cursors
|
## Cursors
|
||||||
|
|
||||||
|
Cursors are `DateTime` pointers used to ensure rolling transactional isolation
|
||||||
|
of various reporting and other maintenance operations. Utilizing a `Cursor`
|
||||||
|
within an operation ensures that instances in time are processed exactly once
|
||||||
|
for a given task, and that tasks can catch up from any failure states at any
|
||||||
|
time.
|
||||||
|
|
||||||
|
Cursors are rolled forward at the end of successful tasks, are not rolled
|
||||||
|
forward in the case of failure, and can be manually set backwards using the
|
||||||
|
update_cursors command in registry_tool to reprocess a past action.
|
||||||
|
|
||||||
|
The following cursor types are defined:
|
||||||
|
|
||||||
|
* **`BRDA`** - BRDA (thin) escrow deposits
|
||||||
|
* **`RDE_REPORT`** - XML RDE report uploads
|
||||||
|
* **`RDE_STAGING`** - RDE (thick) escrow deposit staging
|
||||||
|
* **`RDE_UPLOAD`** - RDE (thick) escrow deposit upload
|
||||||
|
* **`RDE_UPLOAD_SFTP`** - Cursor that tracks the last time we talked to the
|
||||||
|
escrow provider's SFTP server for a given TLD.
|
||||||
|
* **`RECURRING_BILLING`** - Expansion of `Recurring` (renew) billing events
|
||||||
|
into `OneTime` events.
|
||||||
|
|
||||||
|
All `Cursor` entities in Datastore contain a `DateTime` that represents the next
|
||||||
|
timestamp at which an operation should resume processing and a `CursorType` that
|
||||||
|
identifies which operation the cursor is associated with. In many cases, there
|
||||||
|
are multiple cursors per operation; for instance, the cursors related to RDE
|
||||||
|
reporting, staging, and upload are per-TLD cursors. To accomplish this, each
|
||||||
|
`Cursor` also has a scope, a `Key<ImmutableObject>` to which the particular
|
||||||
|
cursor applies (this can be e.g. a `Registry` or any other `ImmutableObject` in
|
||||||
|
datastore, depending on the operation). If the `Cursor` applies to the entire
|
||||||
|
registry environment, it is considered a global cursor and has a scope of
|
||||||
|
`EntityGroupRoot.getCrossTldKey()`.
|
||||||
|
|
||||||
|
Cursors are singleton entities by type and scope. The id for a `Cursor` is a
|
||||||
|
deterministic string that consists of the websafe string of the Key of the scope
|
||||||
|
object concatenated with the name of the name of the cursor type, separated by
|
||||||
|
an underscore.
|
||||||
|
|
||||||
## Mapreduces
|
## Mapreduces
|
||||||
|
|
||||||
## Actions and servlets
|
## Actions and servlets
|
||||||
|
|
|
@ -266,3 +266,27 @@ exampletld_some-other-list
|
||||||
```
|
```
|
||||||
|
|
||||||
## StackDriver monitoring
|
## StackDriver monitoring
|
||||||
|
|
||||||
|
## Updating cursors
|
||||||
|
|
||||||
|
In most cases, cursors will not advance if a task that utilizes a cursor fails
|
||||||
|
(so that the task can be retried for that given timestamp). However, there are
|
||||||
|
some cases where a cursor is updated at the end of a job that produces bad
|
||||||
|
output (for example, RDE export), and in order to re-run a job, the cursor will
|
||||||
|
need to be rolled back.
|
||||||
|
|
||||||
|
In rare cases it might be useful to roll a cursor forward if there is some bad
|
||||||
|
data at a given time that prevents a task from completing successfully, and an
|
||||||
|
acceptable solution is to simply skip the bad data.
|
||||||
|
|
||||||
|
Cursors can be updated as follows:
|
||||||
|
|
||||||
|
```shell
|
||||||
|
$ registry_tool -e {ENVIRONMENT} update_cursors exampletld --type RDE_STAGING \
|
||||||
|
--timestamp 2016-09-01T00:00:00Z
|
||||||
|
Update Cursor@ahFzfmRvbWFpbi1yZWdpc3RyeXIzCxIPRW50aXR5R3JvdXBSb290Igljcm9zcy10bGQMCxIIUmVnaXN0cnkiB3lvdXR1YmUM_RDE_STAGING
|
||||||
|
cursorTime -> [2016-09-23T00:00:00.000Z, 2016-09-01T00:00:00.000Z]
|
||||||
|
|
||||||
|
Perform this command? (y/N): Y
|
||||||
|
Updated 1 entities.
|
||||||
|
```
|
||||||
|
|
Loading…
Add table
Reference in a new issue