diff --git a/docs/architecture/decisions/0013-database-migration.md b/docs/architecture/decisions/0013-database-migration.md new file mode 100644 index 000000000..641ccfa1b --- /dev/null +++ b/docs/architecture/decisions/0013-database-migration.md @@ -0,0 +1,38 @@ +# 13. Use CloudFoundry tasks and Github Actions to manually run migrations + +Date: 2022-10-04 + +## Status + +Accepted + +## Context + +A database-backed web application needs a way to change the “schema” or +structure of the database (adding/dropping tables, adding/dropping/altering +columns of tables). These changes are called “migrations”. They are checked-in +to the Github repository along with the code that uses the changed database. +When that new code is deployed, it will usually run, but may give errors until +the matching migrations have been applied to the database. + +## Decision + +We will not run the checked-in migrations automatically as part of deploys. We +will manually apply database migrations to our Cloud.gov environments using +Cloudfoundry’s `run-task` command to run Django’s `manage.py migrate` command. +For the `unstable` environment, we can run the `cf run-task . . .` command +directly. For the `staging` environment we have a Github Actions workflow +called “Run Migrations” + that will +run that `cf` command using the credentials for `staging` that are only in the +Github Actions configuration. + +## Consequences + +Not automatically applying database migrations before running new code means +that the site might stop working briefly until the database migrations are +applied. This trade-off feels worth the gain in control that we get by not +having migrations run unexpectedly. + +Some of these consequences could be mitigated by developer attention to the +impacts in the unstable and/or staging environment. \ No newline at end of file diff --git a/docs/developer/database-access.md b/docs/developer/database-access.md new file mode 100644 index 000000000..8635543a1 --- /dev/null +++ b/docs/developer/database-access.md @@ -0,0 +1,77 @@ +# Working with Cloud.gov Databases + +You can connect to a Cloud.gov database using the +[cf-service-connect](https://github.com/cloud-gov/cf-service-connect) plugin. +After installing it, use the command + +```shell +cf connect-to-service getgov-unstable getgov-unstable-databse +``` + +to get a `psql` shell on the `unstable` environment's database. + +## Running Migrations + +When new code changes the database schema, we need to apply Django's migrations. +We can run these using CloudFoundry's tasks to run the `manage.py migrate` +command in the correct environment. For the `unstable` environment, developers +can manually run the task with + +```shell +cf run-task getgov-unstable --command 'python manage.py migrate' --name migrate +``` + +For the `staging` environment, developers don't have credentials so we need to +run that command using Github Actions. Go to + and select +the "Run workflow" button, making sure that `staging` is selected. + +## Dropping and re-creating the database + +For `unstable`, it might be necessary to start the database over from scratch. +The easiest way to do that is `DROP DATABASE ...` followed by `CREATE DATABASE +...`. In the `psql` shell, first run the `\l` command to see all of the +databases that are present: + +```shell +cgawsbrokerprodyaobv93n2g3me5i=> \l + List of databases + Name | Owner | Encoding | Collate | Ctype | Access privileges +--------------------------------+------------------+----------+-------------+-------------+--------------------------------------- + cgawsbrokerprodyaobv93n2g3me5i | ugsyn42g56vtykfr | UTF8 | en_US.UTF-8 | en_US.UTF-8 | + postgres | ugsyn42g56vtykfr | UTF8 | en_US.UTF-8 | en_US.UTF-8 | +... +``` + +You will need the name of the database beginning with `cgawsbroker...` for the +next step. To drop that database, you first have to connect to a different +database (you can't drop the database that you are connected to). We connect to +the default `postgres` database instead + +```shell +cgawsbrokerprodyaobv93n2g3me5i=> \c postgres; +psql (14.4, server 12.11) +SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256, compression: off) +You are now connected to database "postgres" as user "ugsyn42g56vtykfr". +``` + +Now drop and create the database with the name above. + +```shell +postgres=> DROP DATABASE cgawsbrokerprodyaobv93n2g3me5i; +DROP DATABASE +postgres=> CREATE DATABASE cgawsbrokerprodyaobv93n2g3me5i; +CREATE DATABASE +``` + +Now the database is empty and Django will need to re-run all of its migrations +in order for the app to start again. + +### Warnings + +This is a very intrusive procedure and it can go wrong in a number of ways. +For example, if the running cloud.gov application goes down, the +`connect-to-service` SSH tunnel will go away and if the app can't come back up +(say, because the database has been dropped but not created) then it isn't +possible to SSH back into the database to fix it and the Cloudfoundry +resources may have to be completely deleted and re-created. diff --git a/ops/scripts/deploy.sh b/ops/scripts/deploy.sh index 7ff641752..a83c4cfa1 100755 --- a/ops/scripts/deploy.sh +++ b/ops/scripts/deploy.sh @@ -7,4 +7,5 @@ cf target -o cisa-getgov-prototyping -s unstable cf push getgov-unstable -f ../ops/manifests/manifest-unstable.yaml -cf run-task getgov-unstable --command 'python manage.py migrate' --name migrate +# migrations need to be run manually. Developers can use this command +#cf run-task getgov-unstable --command 'python manage.py migrate' --name migrate \ No newline at end of file diff --git a/src/run.sh b/src/run.sh index 6caefccfa..1e0e473af 100755 --- a/src/run.sh +++ b/src/run.sh @@ -3,18 +3,6 @@ set -o errexit set -o pipefail -# Only run migrations on the zeroth index when in a cloud.gov environment -if [[ -v CF_INSTANCE_INDEX && $CF_INSTANCE_INDEX == 0 ]] -then - python manage.py migrate --settings=registrar.config.settings --noinput -else - echo "Migrations did not run." - if [[ -v CF_INSTANCE_INDEX ]] - then - echo "CF Instance Index is ${CF_INSTANCE_INDEX}." - fi -fi - # Make sure that django's `collectstatic` has been run locally before pushing up to unstable, # so that the styles and static assets to show up correctly on unstable.