mirror of
https://github.com/google/nomulus.git
synced 2025-07-30 06:26:30 +02:00
8a53edd57b
Relevant error log message: https://pantheon.corp.google.com/logs/viewer?project=domain-registry&minLogLevel=0&expandAll=false×tamp=2021-10-11T15:28:01.047783000Z&customFacets=&limitCustomFacetWidth=true&dateRangeEnd=2021-10-11T20:51:40.591Z&interval=PT1H&resource=gae_app&logName=projects%2Fdomain-registry%2Flogs%2Fappengine.googleapis.com%252Frequest_log&scrollTimestamp=2021-10-11T15:10:23.174336000Z&filters=text:icannReportingUpload&dateRangeUnbound=backwardInTime&advancedFilter=resource.type%3D%22gae_app%22%0AlogName%3D%22projects%2Fdomain-registry%2Flogs%2Fappengine.googleapis.com%252Frequest_log%22%0A%22icannReportingUpload%22%0Aoperation.id%3D%22616453df00ff02a873d26cedb40001737e646f6d61696e2d726567697374727900016261636b656e643a6e6f6d756c75732d76303233000100%22 note the "invalid handle" bit From https://cloud.google.com/datastore/docs/concepts/transactions: "Transactions expire after 270 seconds or if idle for 60 seconds." From b/202309933: "There is a 60 second timeout on Datastore operations after which they will automatically rollback and the handles become invalid." From the logs we can see that the action is lasting significantly longer than 270 seconds -- roughly 480 seconds in the linked log (more or less). My running theory is that ICANN is, for some reason, now being significantly more slow to respond than they used to be. Some uploads in the log linked above are taking upwards of 10 seconds, especially when they have to retry. Because we have >=45 TLDs, it's not surprising that the action is taking >400 seconds to run. The fix here is to perform each per-TLD operation in its own transaction. The only reason why we need the transactions is for the cursors anyway, and we can just grab and store those at the beginning of the transaction. |
||
|---|---|---|
| .. | ||
| gradle/dependency-locks | ||
| src | ||
| build.gradle | ||
| Dockerfile | ||