Commit graph

5 commits

Author SHA1 Message Date
weiminyu
4e71421c81 Support datastore restore in Nomulus tool
Two commands are being added:
- ImportDatastoreCommand starts an async import operation.
  User may choose to wait until import completes or quit
  immediately.
- GetOperationStatusCommand checks the status of an operation.
  It may be used to check the status of an operation started by
  ImportDatastoreCommand.

Both commands communicate with Datastore admin api directly, without
going through the Registry server.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228400059
2019-01-10 16:23:35 -05:00
weiminyu
9c706e79fd Backup Datastore using the Admin REST API
Add server end points to backup Datastore using managed-export mechanism.
A cron job is defined in Alpha to run daily exports using this implementation.

Existing backup is left running. The new backups are saved to a new set of
locations:
- GCS bucket: gs://PROJECT-ID-datastore-backups
- Big Query data set: datastore_backups
- Big Query latest back up view name: latest_datastore_backup
Also, the names of Bigquery tables now use the export timestamp
assigned by Datastore. E.g., 2018_12_05T23_56_18_50532_ContactResource,

After the new import mechanism is implemented and the back-restore flow is
tested, we will stop the existing backup runs and deploy the new
implementation to all environments.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224932957
2018-12-12 13:22:34 -05:00
weiminyu
7d380256af Add helper methods to DatastoreAdmin Operation object
These are needed by Datastore export management actions.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224242281
2018-12-06 13:30:56 -05:00
weiminyu
676fa422db Use proper external dependency references recognized by moe
This fixes kokoro build failure.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221830188
2018-11-16 17:03:50 -05:00
weiminyu
af21b0c32b Add a package to open source
Add {java,tests}/google/registry/export/datastore to open source.
This is part of the migration to Datastore Managed Import/Export
for backup.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221800709
2018-11-16 17:02:15 -05:00