zydronium/google-nomulus
1
0
Fork 0
mirror of https://github.com/google/nomulus.git synced 2025-06-29 15:53:35 +02:00
Code Issues Projects Releases Packages Wiki Activity

Remove unused dns-cron queue

Browse source 
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=132884314
This commit is contained in:
shikhman 2016-09-12 09:49:42 -07:00 • committed by Ben McIlwain
parent b9b2829f7c
commit cee08d48f2
2 changed files with 72 additions and 83 deletions
Download patch file Download diff file Expand all files Collapse all files

55
g3doc/app-engine-architecture.md
View file

@ -109,40 +109,41 @@ explicitly marked as otherwise.
streamed to BigQuery in the `Metrics` class. Tasks are enqueued during EPP
flows in `EppController`. This means that there is a lag of a few seconds to
a few minutes between when metrics are generated and when they are queryable
in BigQuery, but this is preferable to slowing all EPP flows down and blocking
them on BigQuery streaming.
in BigQuery, but this is preferable to slowing all EPP flows down and
blocking them on BigQuery streaming.
* `brda` -- Queue for tasks to upload weekly Bulk Registration Data Access
(BRDA) files to a location where they are available to ICANN. The
`RdeStagingReducer` (part of the RDE MapReduce) creates these tasks at the end
of generating an RDE dump.
* `delete-commits` -- Cron queue for tasks to regularly delete commit logs that
are more than thirty days stale. These tasks execute the
`RdeStagingReducer` (part of the RDE MapReduce) creates these tasks at the
end of generating an RDE dump.
* `delete-commits` -- Cron queue for tasks to regularly delete commit logs
that are more than thirty days stale. These tasks execute the
`DeleteOldCommitLogsAction`.
* `dns-cron` (cron queue) and `dns-pull` (pull queue) -- A push/pull pair of
queues. Cron regularly enqueues tasks in dns-cron each minute, which are then
executed by `ReadDnsQueueAction`, which leases a batch of tasks from the pull
queue, groups them by TLD, and writes them as a single task to `dns-publish`
to be published to the configured DNS writer for the TLD.
* `dns-publish` -- Queue for batches of DNS updates to be pushed to DNS writers.
* `dns-pull` -- A pull queue to enqueue DNS modifications. Cron regularly runs
`ReadDnsQueueAction`, which drains the queue, batches modifications by TLD,
and writes the batches to `dns-publish` to be published to the configured
`DnsWriter` for the TLD.
* `dns-publish` -- Queue for batches of DNS updates to be pushed to DNS
writers.
* `export-bigquery-poll` -- Queue for tasks to query the success/failure of a
given BigQuery export job. Tasks are enqueued by `BigqueryPollJobAction`.
* `export-commits` -- Queue for tasks to export commit log checkpoints. Tasks
are enqueued by `CommitLogCheckpointAction` (which is run every minute by
cron) and executed by `ExportCommitLogDiffAction`.
* `export-reserved-terms` -- Cron queue for tasks to export the list of reserved
terms for each TLD. The tasks are executed by `ExportReservedTermsAction`.
* `export-reserved-terms` -- Cron queue for tasks to export the list of
reserved terms for each TLD. The tasks are executed by
`ExportReservedTermsAction`.
* `export-snapshot` -- Cron and push queue for tasks to load a Datastore
snapshot that was stored in Google Cloud Storage and export it to BigQuery.
Tasks are enqueued by both cron and `CheckSnapshotServlet` and are executed by
both `ExportSnapshotServlet` and `LoadSnapshotAction`.
Tasks are enqueued by both cron and `CheckSnapshotServlet` and are executed
by both `ExportSnapshotServlet` and `LoadSnapshotAction`.
* `export-snapshot-poll` -- Queue for tasks to check that a Datastore snapshot
has been successfully uploaded to Google Cloud Storage (this is an
asynchronous background operation that can take an indeterminate amount of
time). Once the snapshot is successfully uploaded, it is imported into
BigQuery. Tasks are enqueued by `ExportSnapshotServlet` and executed by
`CheckSnapshotServlet`.
* `export-snapshot-update-view` -- Queue for tasks to update the BigQuery views
to point to the most recently uploaded snapshot. Tasks are enqueued by
* `export-snapshot-update-view` -- Queue for tasks to update the BigQuery
views to point to the most recently uploaded snapshot. Tasks are enqueued by
`LoadSnapshotAction` and executed by `UpdateSnapshotViewAction`.
* `flows-async` -- Queue for asynchronous tasks that are enqueued during EPP
command flows. Currently all of these tasks correspond to invocations of any
@ -151,15 +152,15 @@ explicitly marked as otherwise.
* `group-members-sync` -- Cron queue for tasks to sync registrar contacts (not
domain contacts!) to Google Groups. Tasks are executed by
`SyncGroupMembersAction`.
* `load[0-9]` -- Queues used to load-test the system by `LoadTestAction`. These
queues don't need to exist except when actively running load tests (which is
not recommended on production environments). There are ten of these queues to
provide simple sharding, because the Domain Registry system is capable of
handling significantly more Queries Per Second than the highest throttle limit
available on task queues (which is 500 qps).
* `lordn-claims` and `lordn-sunrise` -- Pull queues for handling LORDN exports.
Tasks are enqueued synchronously during EPP commands depending on whether the
domain name in question has a claims notice ID.
* `load[0-9]` -- Queues used to load-test the system by `LoadTestAction`.
These queues don't need to exist except when actively running load tests
(which is not recommended on production environments). There are ten of
these queues to provide simple sharding, because the Domain Registry system
is capable of handling significantly more Queries Per Second than the
highest throttle limit available on task queues (which is 500 qps).
* `lordn-claims` and `lordn-sunrise` -- Pull queues for handling LORDN
exports. Tasks are enqueued synchronously during EPP commands depending on
whether the domain name in question has a claims notice ID.
* `marksdb` -- Queue for tasks to verify that an upload to NORDN was
successfully received and verified. These tasks are enqueued by
`NordnUploadAction` following an upload and are executed by

12
java/google/registry/env/common/default/WEB-INF/queue.xml vendored
View file

@ -7,18 +7,6 @@
<bucket-size>5</bucket-size>
</queue>
<queue>
<name>dns-cron</name>
<!-- There is no point allowing more than 10/s because the pull queue that feeds
this job will refuse to service more than 10 qps. See
https://cloud.google.com/appengine/docs/java/javadoc/com/google/appengine/api/taskqueue/Queue#leaseTasks-long-java.util.concurrent.TimeUnit-long- -->
<rate>10/s</rate>
<bucket-size>100</bucket-size>
<retry-parameters>
<task-retry-limit>1</task-retry-limit>
</retry-parameters>
</queue>
<queue>
<name>dns-pull</name>
<mode>pull</mode>