Enable daily backup of Datastore in sandbox and production

This uses the new backup implementation, and starts after
the old exportSnapshot task has completed. The old task
will be removed later.

Daily backups in alpha has been running successfully.

Manually triggered exports in both environments also completed
successfully.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227908077
This commit is contained in:
weiminyu 2019-01-04 13:49:35 -08:00 committed by Ben McIlwain
parent 9eaeab9cfe
commit 6022353e57
3 changed files with 41 additions and 2 deletions

View file

@ -152,8 +152,11 @@
<!--
Keep google.registry.export.CheckBackupAction.MAXIMUM_BACKUP_RUNNING_TIME less than
this interval.
-->
<schedule>every day 07:00</schedule>
To facilitate troubleshooting, run after the exportSnapshot task below (typically finishes in
90 minutes in production) has completed.
TODO(b/122271637): move to 06:00 (UTC) when the exportSnapshot task is removed. -->
<schedule>every day 09:00</schedule>
<target>backend</target>
</cron>

View file

@ -174,6 +174,24 @@
<target>backend</target>
</cron>
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/backupDatastore&runInEmpty]]></url>
<description>
This job fires off a Datastore managed-export job that generates snapshot files in GCS.
It also enqueues a new task to wait on the completion of that job and then load the resulting
snapshot into bigquery.
</description>
<!--
Keep google.registry.export.CheckBackupAction.MAXIMUM_BACKUP_RUNNING_TIME less than
this interval.
To facilitate troubleshooting, run after the exportSnapshot task below (typically finishes in
90 minutes in production) has completed.
TODO(b/122271637): move to 06:00 (UTC) when the exportSnapshot task is removed. -->
<schedule>every day 09:00</schedule>
<target>backend</target>
</cron>
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/exportSnapshot&runInEmpty]]></url>
<description>

View file

@ -151,6 +151,24 @@
<target>backend</target>
</cron>
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/backupDatastore&runInEmpty]]></url>
<description>
This job fires off a Datastore managed-export job that generates snapshot files in GCS.
It also enqueues a new task to wait on the completion of that job and then load the resulting
snapshot into bigquery.
</description>
<!--
Keep google.registry.export.CheckBackupAction.MAXIMUM_BACKUP_RUNNING_TIME less than
this interval.
To facilitate troubleshooting, run after the exportSnapshot task below (typically finishes in
90 minutes in production) has completed.
TODO(b/122271637): move to 06:00 (UTC) when the exportSnapshot task is removed. -->
<schedule>every day 09:00</schedule>
<target>backend</target>
</cron>
<cron>
<url><![CDATA[/_dr/cron/fanout?queue=export-snapshot&endpoint=/_dr/task/exportSnapshot&runInEmpty]]></url>
<description>