mirror of
https://github.com/google/nomulus.git
synced 2025-04-29 19:47:51 +02:00
Add Cursor documentation
------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=133994188
This commit is contained in:
parent
b783acfcc6
commit
414781e79b
2 changed files with 61 additions and 0 deletions
|
@ -13,6 +13,43 @@ particularly important pieces of the system are implemented.
|
|||
|
||||
## Cursors
|
||||
|
||||
Cursors are `DateTime` pointers used to ensure rolling transactional isolation
|
||||
of various reporting and other maintenance operations. Utilizing a `Cursor`
|
||||
within an operation ensures that instances in time are processed exactly once
|
||||
for a given task, and that tasks can catch up from any failure states at any
|
||||
time.
|
||||
|
||||
Cursors are rolled forward at the end of successful tasks, are not rolled
|
||||
forward in the case of failure, and can be manually set backwards using the
|
||||
update_cursors command in registry_tool to reprocess a past action.
|
||||
|
||||
The following cursor types are defined:
|
||||
|
||||
* **`BRDA`** - BRDA (thin) escrow deposits
|
||||
* **`RDE_REPORT`** - XML RDE report uploads
|
||||
* **`RDE_STAGING`** - RDE (thick) escrow deposit staging
|
||||
* **`RDE_UPLOAD`** - RDE (thick) escrow deposit upload
|
||||
* **`RDE_UPLOAD_SFTP`** - Cursor that tracks the last time we talked to the
|
||||
escrow provider's SFTP server for a given TLD.
|
||||
* **`RECURRING_BILLING`** - Expansion of `Recurring` (renew) billing events
|
||||
into `OneTime` events.
|
||||
|
||||
All `Cursor` entities in Datastore contain a `DateTime` that represents the next
|
||||
timestamp at which an operation should resume processing and a `CursorType` that
|
||||
identifies which operation the cursor is associated with. In many cases, there
|
||||
are multiple cursors per operation; for instance, the cursors related to RDE
|
||||
reporting, staging, and upload are per-TLD cursors. To accomplish this, each
|
||||
`Cursor` also has a scope, a `Key<ImmutableObject>` to which the particular
|
||||
cursor applies (this can be e.g. a `Registry` or any other `ImmutableObject` in
|
||||
datastore, depending on the operation). If the `Cursor` applies to the entire
|
||||
registry environment, it is considered a global cursor and has a scope of
|
||||
`EntityGroupRoot.getCrossTldKey()`.
|
||||
|
||||
Cursors are singleton entities by type and scope. The id for a `Cursor` is a
|
||||
deterministic string that consists of the websafe string of the Key of the scope
|
||||
object concatenated with the name of the name of the cursor type, separated by
|
||||
an underscore.
|
||||
|
||||
## Mapreduces
|
||||
|
||||
## Actions and servlets
|
||||
|
|
|
@ -266,3 +266,27 @@ exampletld_some-other-list
|
|||
```
|
||||
|
||||
## StackDriver monitoring
|
||||
|
||||
## Updating cursors
|
||||
|
||||
In most cases, cursors will not advance if a task that utilizes a cursor fails
|
||||
(so that the task can be retried for that given timestamp). However, there are
|
||||
some cases where a cursor is updated at the end of a job that produces bad
|
||||
output (for example, RDE export), and in order to re-run a job, the cursor will
|
||||
need to be rolled back.
|
||||
|
||||
In rare cases it might be useful to roll a cursor forward if there is some bad
|
||||
data at a given time that prevents a task from completing successfully, and an
|
||||
acceptable solution is to simply skip the bad data.
|
||||
|
||||
Cursors can be updated as follows:
|
||||
|
||||
```shell
|
||||
$ registry_tool -e {ENVIRONMENT} update_cursors exampletld --type RDE_STAGING \
|
||||
--timestamp 2016-09-01T00:00:00Z
|
||||
Update Cursor@ahFzfmRvbWFpbi1yZWdpc3RyeXIzCxIPRW50aXR5R3JvdXBSb290Igljcm9zcy10bGQMCxIIUmVnaXN0cnkiB3lvdXR1YmUM_RDE_STAGING
|
||||
cursorTime -> [2016-09-23T00:00:00.000Z, 2016-09-01T00:00:00.000Z]
|
||||
|
||||
Perform this command? (y/N): Y
|
||||
Updated 1 entities.
|
||||
```
|
||||
|
|
Loading…
Add table
Reference in a new issue