guyben bba975a991 Allow over 1000 dns-updates to be handled at once ... The task-queue API only allows reading 1000 tasks at a time, hence the original reason for this limit. We get over that limit by reading (and processing) items from the queue in a loop - 1000 at a time. This is important because the 1000 dns-updates are shared among all TLDs, meaning that a TLD with >1000 waiting updates can affect the update latency of other TLDs. In addition, partially fixes the bug where if there are more than 1000 updates to paused / non-existing TLDs, we completely block all updated to all TLDs. By partially fixed, I mean "if we have around 1000 updates to paused TLDs, we will read them every time ReadDnsUpdates is called, ignore then, and only then get to the actual updates we want to process". This works for a number of 1000 updates waiting - but if paused TLDs have tens or hundreds of thousands of updates waiting - this might still choke up other TLDs (not to mention we keep reading / updating 10s or 100s of thousands of tasks in the queue, that's... bad.) A more thorough fix will come in a future CL, as it requires a more thorough change in the code. Note that the queue lease command supports a maximum of 10 QPS. Any more than that - and we get errors / empty results. Hence we limit our QPS to 9 to be on the safe side. ------------- Created by MOE: https://github.com/google/moe MOE_MIGRATED_REVID=185218684		2018-02-20 15:42:09 -05:00
..
backup	Rationalize logging statements across codebase	2018-01-19 14:56:45 -05:00
batch	Migrate final try/catch test assertions to use assert/expectThrows	2018-01-19 14:43:12 -05:00
beam	Move invoice generation to billing bucket and improve emailing	2018-01-19 14:34:14 -05:00
bigquery	Automatically refactor more exception testing to use new JUnit rules	2017-12-27 10:42:36 -05:00
builddefs	Update copyright year on all license headers	2017-02-02 16:27:22 -05:00
config	Remove more unused Truth8.assertThat() static imports in tests	2017-11-07 17:22:56 -05:00
cron	Automatically refactor more exception testing to use new JUnit rules	2017-12-27 10:42:36 -05:00
dns	Allow over 1000 dns-updates to be handled at once	2018-02-20 15:42:09 -05:00
export	Migrate final try/catch test assertions to use assert/expectThrows	2018-01-19 14:43:12 -05:00
flows	Add DateTime as extensibility param for allocation token logic	2018-02-20 15:27:42 -05:00
groups	Use method references when possible	2017-12-27 11:25:02 -05:00
keyring	Move backported JUnit file to third_party (part 2)	2017-12-02 11:37:46 -05:00
mapreduce/inputs	Migrate final try/catch test assertions to use assert/expectThrows	2018-01-19 14:43:12 -05:00
model	Remove leniency on poll message ID format without years in them	2018-02-20 15:12:43 -05:00
module	Refactor ICANN reporting and billing into common package	2018-02-01 21:56:06 -05:00
monitoring/whitebox	Add dependencies on classes that are not in the default modules in JDK 9	2018-02-01 22:00:49 -05:00
pricing	Automatically refactor more exception testing to use new JUnit rules	2017-12-27 10:42:36 -05:00
proxy	Register quota metrics in GCP proxy	2018-02-20 15:39:15 -05:00
rdap	RDAP: Change data policy remark for redacted contacts	2018-02-20 15:33:31 -05:00
rde	Add //third_party/java/jaxb dependency to targets using javax.xml.bind	2018-01-23 16:18:35 -05:00
reporting	Refactor ICANN reporting and billing into common package	2018-02-01 21:56:06 -05:00
request	Remove unnecessary type specifications	2017-12-27 11:23:03 -05:00
security	Remove more unused Truth8.assertThat() static imports in tests	2017-11-07 17:22:56 -05:00
server	Move backported JUnit file to third_party (part 2)	2017-12-02 11:37:46 -05:00
storage/drive	Remove unnecessary type specifications	2017-12-27 11:23:03 -05:00
testing	Allow over 1000 dns-updates to be handled at once	2018-02-20 15:42:09 -05:00
tldconfig/idn	Remove unnecessary type specifications	2017-12-27 11:23:03 -05:00
tmch	Use method references instead of lambdas when possible	2017-12-27 10:55:03 -05:00
tools	Delete obsolete billing code	2018-02-20 15:09:45 -05:00
ui	Replace uses of @code in Javascript documentation with Markdown backticks	2018-02-05 23:51:49 -05:00
util	Replace com.google.common.base.Function with java.util.function.Function	2017-12-27 11:08:55 -05:00
whois	Migrate final try/catch test assertions to use assert/expectThrows	2018-01-19 14:43:12 -05:00
xjc	Automatically refactor more exception testing to use new JUnit rules	2017-12-27 10:42:36 -05:00
xml	Add basic AllocationToken validation/redemption for domain creates	2018-01-19 14:28:26 -05:00