zydronium/manage.get.gov
1
0
Fork 0
mirror of https://github.com/cisagov/manage.get.gov.git synced 2025-08-04 17:01:56 +02:00
Code Issues Projects Releases Packages Wiki Activity

Merge pull request #2246 from cisagov/dk/2194-import-export

Browse source 
Issues #2194, #2204: added command scripts clean_tables, export_tables and import_tables
This commit is contained in:
dave-kennedy-ecs 2024-06-04 10:58:02 -04:00 • committed by GitHub
commit 7f55b6dc6d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 693 additions and 19 deletions
Download patch file Download diff file Expand all files Collapse all files

84
docs/operations/import_export.md
View file

@ -1,18 +1,29 @@
# Export / Import Tables
A means is provided to export and import individual tables from
A means is provided to export and import tables from
one environment to another. This allows for replication of
production data in a development environment. Import and export
are provided through the django admin interface, through a modified
library, django-import-export. Each supported model has an Import
and an Export button on the list view.
are provided through a modified library, django-import-export.
Simple scripts are provided as detailed below.
### Export
When exporting models from the source environment, make sure that
no filters are selected. This will ensure that all rows of the model
are exported. Due to database dependencies, the following models
need to be exported:
To export from the source environment, run the following command from src directory:
manage.py export_tables
Connect to the source sandbox and run the command:
cf ssh {source-app}
/tmp/lifecycle/shell
./manage.py export_tables
example exporting from getgov-stable:
cf ssh getgov-stable
/tmp/lifecycle/shell
./manage.py export_tables
This exports a file, exported_tables.zip, to the tmp directory
For reference, the zip file will contain the following tables in csv form:
* User
* Contact
@ -25,6 +36,20 @@ need to be exported:
* Host
* HostIP
After exporting the file from the target environment, scp the exported_tables.zip
file from the target environment to local. Run the below commands from local.
Get passcode by running:
cf ssh-code
scp file from source app to local file:
scp -P 2222 -o User=cf:$(cf curl /v3/apps/$(cf app {source-app} --guid)/processes | jq -r '.resources[] | select(.type=="web") | .guid')/0 ssh.fr.cloud.gov:app/tmp/exported_tables.zip {local_file_path}
when prompted, supply the passcode retrieved in the 'cf ssh-code' command
example copying from stable to local cwd:
scp -P 2222 -o User=cf:$(cf curl /v3/apps/$(cf app getgov-stable --guid)/processes | jq -r '.resources[] | select(.type=="web") | .guid')/0 ssh.fr.cloud.gov:app/tmp/exported_tables.zip .
### Import
When importing into the target environment, if the target environment
@ -34,7 +59,18 @@ that there are no database conflicts on import.
#### Preparing Target Environment
Delete all rows from tables in the following order through django admin:
In order to delete all rows from the appropriate tables, run the following
command:
cf ssh {target-app}
/tmp/lifecycle/shell
./manage.py clean_tables
example cleaning getgov-backup:
cf ssh getgov-backup
/tmp/lifecycle/backup
./manage.py clean_tables
For reference, this deletes all rows from the following tables:
* DomainInformation
* DomainRequest
@ -48,10 +84,34 @@ Delete all rows from tables in the following order through django admin:
#### Importing into Target Environment
Once target environment is prepared, files can be imported in the following
order:
Once target environment is prepared, files can be imported.
* User (After importing User table, you need to delete all rows from Contact table before importing Contacts)
To scp the exported_tables.zip file from local to the sandbox, run the following:
Get passcode by running:
cf ssh-code
scp file from local to target app:
scp -P 2222 -o User=cf:$(cf curl /v3/apps/$(cf app {target-app} --guid)/processes | jq -r '.resources[] | select(.type=="web") | .guid')/0 {local_file_path} ssh.fr.cloud.gov:app/tmp/exported_tables.zip
when prompted, supply the passcode retrieved in the 'cf ssh-code' command
example copy of local file in tmp to getgov-backup:
scp -P 2222 -o User=cf:$(cf curl /v3/apps/$(cf app getgov-backup --guid)/processes | jq -r '.resources[] | select(.type=="web") | .guid')/0 tmp/exported_tables.zip ssh.fr.cloud.gov:app/tmp/exported_tables.zip
Then connect to a shell in the target environment, and run the following import command:
cf ssh {target-app}
/tmp/lifecycle/shell
./manage.py import_tables
example cleaning getgov-backup:
cf ssh getgov-backup
/tmp/lifecycle/backup
./manage.py import_tables
For reference, this imports tables in the following order:
* User
* Contact
* Domain
* Host