diff --git a/docs/developer/database-access.md b/docs/developer/database-access.md index 859ef2fd6..f77261bbb 100644 --- a/docs/developer/database-access.md +++ b/docs/developer/database-access.md @@ -56,6 +56,13 @@ cf ssh getgov-ENVIRONMENT ./manage.py dumpdata ``` +## Access certain table in the database +1. `cf connect-to-service getgov-ENVIRONMENT getgov-ENVIRONMENT-database` gets you into whichever environments database you'd like +2. `\c [table name here that starts cgaws...etc];` connects to the [cgaws...etc] table +3. `\dt` retrieves information about that table and displays it +4. Make sure the table you are looking for exists. For this example, we are looking for `django_migrations` +5. Run `SELECT * FROM django_migrations;` to see everything that's in it! + ## Dropping and re-creating the database For your sandbox environment, it might be necessary to start the database over from scratch. diff --git a/docs/developer/migration-troubleshooting.md b/docs/developer/migration-troubleshooting.md index b90c02ae3..22a02503d 100644 --- a/docs/developer/migration-troubleshooting.md +++ b/docs/developer/migration-troubleshooting.md @@ -121,3 +121,19 @@ https://cisa-corp.slack.com/archives/C05BGB4L5NF/p1697810600723069 2. `./manage.py migrate model_name_here file_name_WITH_create` (run the last data creation migration AND ONLY THAT ONE) 3. `./manage.py migrate --fake model_name_here most_recent_file_name` (fake migrate the last migration in the migration list) 4. `./manage.py load` (rerun fixtures) + +### Scenario 9: Inconsistent Migration History +If you see `django.db.migrations.exceptions.InconsistentMigrationHistory` error, or when you run `./manage.py showmigrations` it looks like: + +[x] 0056_example_migration +[ ] 0057_other_migration +[x] 0058_some_other_migration + +1. Go to [database-access.md](../database-access.md#access-certain-table-in-the-database) to see the commands on how to access a certain table in the database. +2. In this case, we want to remove the migration "history" from the `django_migrations` table +3. Once you are in the `cgaws...` table, select the `django_migrations` table with the command `SELECT * FROM django_migrations;` +4. Find the id of the "history" you want to delete. This will be the one in the far left column. For this example, let's pretend the id is 101. +5. Run `DELETE FROM django_migrations WHERE id=101;` where 101 is an example id as seen above. +6. Go to your shell and run `./manage.py showmigrations` to make sure your migrations are now back to the right state. Most likely you will see several unapplied migrations. +7. If you still have unapplied migrations, run `./manage.py migrate`. If an error occurs saying one has already been applied, fake that particular migration `./manage.py migrate --fake model_name_here migration_number` and then run the normal `./manage.py migrate` command to then apply those migrations that come after the one that threw the error. + diff --git a/docs/operations/data_migration.md b/docs/operations/data_migration.md index b7e413c05..0846208de 100644 --- a/docs/operations/data_migration.md +++ b/docs/operations/data_migration.md @@ -586,3 +586,59 @@ Example: `cf ssh getgov-za` | | Parameter | Description | |:-:|:-------------------------- |:----------------------------------------------------------------------------| | 1 | **debug** | Increases logging detail. Defaults to False. | + + +## Populate Organization type +This section outlines how to run the `populate_organization_type` script. +The script is used to update the organization_type field on DomainRequest and DomainInformation when it is None. +That data are synthesized from the generic_org_type field and the is_election_board field by concatenating " - Elections" on the end of generic_org_type string if is_elections_board is True. + +### Running on sandboxes + +#### Step 1: Login to CloudFoundry +```cf login -a api.fr.cloud.gov --sso``` + +#### Step 2: Get the domain_election_board file +The latest domain_election_board csv can be found [here](https://drive.google.com/file/d/1aDeCqwHmBnXBl2arvoFCN0INoZmsEGsQ/view). +After downloading this file, place it in `src/migrationdata` + +#### Step 2: Upload the domain_election_board file to your sandbox +Follow [Step 1: Transfer data to sandboxes](#step-1-transfer-data-to-sandboxes) and [Step 2: Transfer uploaded files to the getgov directory](#step-2-transfer-uploaded-files-to-the-getgov-directory) from the [Set Up Migrations on Sandbox](#set-up-migrations-on-sandbox) portion of this doc. + +#### Step 2: SSH into your environment +```cf ssh getgov-{space}``` + +Example: `cf ssh getgov-za` + +#### Step 3: Create a shell instance +```/tmp/lifecycle/shell``` + +#### Step 4: Running the script +```./manage.py populate_organization_type {domain_election_board_filename}``` + +- The domain_election_board_filename file must adhere to this format: + - example.gov\ + example2.gov\ + example3.gov + +Example: +`./manage.py populate_organization_type migrationdata/election-domains.csv` + +### Running locally + +#### Step 1: Get the domain_election_board file +The latest domain_election_board csv can be found [here](https://drive.google.com/file/d/1aDeCqwHmBnXBl2arvoFCN0INoZmsEGsQ/view). +After downloading this file, place it in `src/migrationdata` + + +#### Step 2: Running the script +```docker-compose exec app ./manage.py populate_organization_type {domain_election_board_filename}``` + +Example (assuming that this is being ran from src/): +`docker-compose exec app ./manage.py populate_organization_type migrationdata/election-domains.csv` + + +### Required parameters +| | Parameter | Description | +|:-:|:------------------------------------|:-------------------------------------------------------------------| +| 1 | **domain_election_board_filename** | A file containing every domain that is an election office. diff --git a/src/registrar/admin.py b/src/registrar/admin.py index 5f6c71f45..974eb1995 100644 --- a/src/registrar/admin.py +++ b/src/registrar/admin.py @@ -663,6 +663,7 @@ class ContactAdmin(ListHeaderAdmin): list_display = [ "contact", "email", + "user_exists", ] # this ordering effects the ordering of results # in autocomplete_fields for user @@ -679,6 +680,13 @@ class ContactAdmin(ListHeaderAdmin): change_form_template = "django/admin/email_clipboard_change_form.html" + def user_exists(self, obj): + """Check if the Contact has a related User""" + return "Yes" if obj.user is not None else "No" + + user_exists.short_description = "Is user" # type: ignore + user_exists.admin_order_field = "user" # type: ignore + # We name the custom prop 'contact' because linter # is not allowing a short_description attr on it # This gets around the linter limitation, for now. @@ -1445,12 +1453,36 @@ class DomainRequestAdmin(ListHeaderAdmin): """ Override changelist_view to set the selected value of status filter. """ + # there are two conditions which should set the default selected filter: + # 1 - there are no query parameters in the request and the request is the + # initial request for this view + # 2 - there are no query parameters in the request and the referring url is + # the change view for a domain request + should_apply_default_filter = False # use http_referer in order to distinguish between request as a link from another page # and request as a removal of all filters http_referer = request.META.get("HTTP_REFERER", "") # if there are no query parameters in the request - # and the request is the initial request for this view - if not bool(request.GET) and request.path not in http_referer: + if not bool(request.GET): + # if the request is the initial request for this view + if request.path not in http_referer: + should_apply_default_filter = True + # elif the request is a referral from changelist view or from + # domain request change view + elif request.path in http_referer: + # find the index to determine the referring url after the path + index = http_referer.find(request.path) + # Check if there is a character following the path in http_referer + next_char_index = index + len(request.path) + if index + next_char_index < len(http_referer): + next_char = http_referer[next_char_index] + + # Check if the next character is a digit, if so, this indicates + # a change view for domain request + if next_char.isdigit(): + should_apply_default_filter = True + + if should_apply_default_filter: # modify the GET of the request to set the selected filter modified_get = copy.deepcopy(request.GET) modified_get["status__in"] = "submitted,in review,action needed" @@ -1487,10 +1519,11 @@ class DomainInformationInline(admin.StackedInline): We had issues inheriting from both StackedInline and the source DomainInformationAdmin since these classes conflict, so we'll just pull what we need - from DomainInformationAdmin""" + from DomainInformationAdmin + """ form = DomainInformationInlineForm - + template = "django/admin/includes/domain_info_inline_stacked.html" model = models.DomainInformation fieldsets = copy.deepcopy(DomainInformationAdmin.fieldsets) @@ -1500,10 +1533,8 @@ class DomainInformationInline(admin.StackedInline): del fieldsets[index] break + readonly_fields = DomainInformationAdmin.readonly_fields analyst_readonly_fields = DomainInformationAdmin.analyst_readonly_fields - # For each filter_horizontal, init in admin js extendFilterHorizontalWidgets - # to activate the edit/delete/view buttons - filter_horizontal = ("other_contacts",) autocomplete_fields = [ "creator", @@ -1669,11 +1700,15 @@ class DomainAdmin(ListHeaderAdmin): if extra_context is None: extra_context = {} - # Pass in what the an extended expiration date would be for the expiration date modal if object_id is not None: domain = Domain.objects.get(pk=object_id) - years_to_extend_by = self._get_calculated_years_for_exp_date(domain) + # Used in the custom contact view + if domain is not None and hasattr(domain, "domain_info"): + extra_context["original_object"] = domain.domain_info + + # Pass in what the an extended expiration date would be for the expiration date modal + years_to_extend_by = self._get_calculated_years_for_exp_date(domain) try: curr_exp_date = domain.registry_expiration_date except KeyError: diff --git a/src/registrar/assets/js/get-gov.js b/src/registrar/assets/js/get-gov.js index 69f916b7f..3c1b1099f 100644 --- a/src/registrar/assets/js/get-gov.js +++ b/src/registrar/assets/js/get-gov.js @@ -589,7 +589,7 @@ function hideDeletedForms() { let isDotgovDomain = document.querySelector(".dotgov-domain-form"); // The Nameservers formset features 2 required and 11 optionals if (isNameserversForm) { - cloneIndex = 2; + // cloneIndex = 2; formLabel = "Name server"; // DNSSEC: DS Data } else if (isDsDataForm) { @@ -789,6 +789,43 @@ function hideDeletedForms() { HookupYesNoListener("additional_details-has_anything_else_text",'anything-else', null) })(); +/** + * An IIFE that disables the delete buttons on nameserver forms on page load if < 3 forms + * + */ +(function nameserversFormListener() { + let isNameserversForm = document.querySelector(".nameservers-form"); + if (isNameserversForm) { + let forms = document.querySelectorAll(".repeatable-form"); + if (forms.length < 3) { + // Hide the delete buttons on the 2 nameservers + forms.forEach((form) => { + Array.from(form.querySelectorAll('.delete-record')).forEach((deleteButton) => { + deleteButton.setAttribute("disabled", "true"); + }); + }); + } + } +})(); + +/** + * An IIFE that disables the delete buttons on nameserver forms on page load if < 3 forms + * + */ +(function nameserversFormListener() { + let isNameserversForm = document.querySelector(".nameservers-form"); + if (isNameserversForm) { + let forms = document.querySelectorAll(".repeatable-form"); + if (forms.length < 3) { + // Hide the delete buttons on the 2 nameservers + forms.forEach((form) => { + Array.from(form.querySelectorAll('.delete-record')).forEach((deleteButton) => { + deleteButton.setAttribute("disabled", "true"); + }); + }); + } + } +})(); /** * An IIFE that listens to the yes/no radio buttons on the CISA representatives form and toggles form field visibility accordingly diff --git a/src/registrar/assets/sass/_theme/_admin.scss b/src/registrar/assets/sass/_theme/_admin.scss index 4b69dc8e3..f5717d067 100644 --- a/src/registrar/assets/sass/_theme/_admin.scss +++ b/src/registrar/assets/sass/_theme/_admin.scss @@ -495,6 +495,8 @@ address.dja-address-contact-list { overflow: hidden; white-space: nowrap; text-overflow: ellipsis; + font-size: medium; + padding-top: 3px !important; } } @@ -505,6 +507,7 @@ address.dja-address-contact-list { @media screen and (min-width:768px) { .visible-768 { display: block; + padding-top: 0; } } @@ -593,6 +596,7 @@ address.dja-address-contact-list { right: auto; left: 4px; height: 100%; + top: -1px; } button { font-size: unset !important; diff --git a/src/registrar/forms/domain.py b/src/registrar/forms/domain.py index 7b0ac2956..da1462bdb 100644 --- a/src/registrar/forms/domain.py +++ b/src/registrar/forms/domain.py @@ -83,25 +83,34 @@ class DomainNameserverForm(forms.Form): # after clean_fields. it is used to determine form level errors. # is_valid is typically called from view during a post cleaned_data = super().clean() + self.clean_empty_strings(cleaned_data) + server = cleaned_data.get("server", "") - # remove ANY spaces in the server field - server = server.replace(" ", "") - # lowercase the server - server = server.lower() + server = server.replace(" ", "").lower() cleaned_data["server"] = server - ip = cleaned_data.get("ip", None) - # remove ANY spaces in the ip field + + ip = cleaned_data.get("ip", "") ip = ip.replace(" ", "") cleaned_data["ip"] = ip + domain = cleaned_data.get("domain", "") ip_list = self.extract_ip_list(ip) - # validate if the form has a server or an ip + # Capture the server_value + server_value = self.cleaned_data.get("server") + + # Validate if the form has a server or an ip if (ip and ip_list) or server: self.validate_nameserver_ip_combo(domain, server, ip_list) + # Re-set the server value: + # add_error which is called on validate_nameserver_ip_combo will clean-up (delete) any invalid data. + # We need that data because we need to know the total server entries (even if invalid) in the formset + # clean method where we determine whether a blank first and/or second entry should throw a required error. + self.cleaned_data["server"] = server_value + return cleaned_data def clean_empty_strings(self, cleaned_data): @@ -149,6 +158,19 @@ class BaseNameserverFormset(forms.BaseFormSet): """ Check for duplicate entries in the formset. """ + + # Check if there are at least two valid servers + valid_servers_count = sum( + 1 for form in self.forms if form.cleaned_data.get("server") and form.cleaned_data.get("server").strip() + ) + if valid_servers_count >= 2: + # If there are, remove the "At least two name servers are required" error from each form + # This will allow for successful submissions when the first or second entries are blanked + # but there are enough entries total + for form in self.forms: + if form.errors.get("server") == ["At least two name servers are required."]: + form.errors.pop("server") + if any(self.errors): # Don't bother validating the formset unless each form is valid on its own return @@ -156,10 +178,13 @@ class BaseNameserverFormset(forms.BaseFormSet): data = [] duplicates = [] - for form in self.forms: + for index, form in enumerate(self.forms): if form.cleaned_data: value = form.cleaned_data["server"] - if value in data: + # We need to make sure not to trigger the duplicate error in case the first and second nameservers + # are empty. If there are enough records in the formset, that error is an unecessary blocker. + # If there aren't, the required error will block the submit. + if value in data and not (form.cleaned_data.get("server", "").strip() == "" and index == 1): form.add_error( "server", NameserverError(code=nsErrorCodes.DUPLICATE_HOST, nameserver=value), diff --git a/src/registrar/management/commands/populate_organization_type.py b/src/registrar/management/commands/populate_organization_type.py new file mode 100644 index 000000000..a7dd98b24 --- /dev/null +++ b/src/registrar/management/commands/populate_organization_type.py @@ -0,0 +1,237 @@ +import argparse +import logging +import os +from typing import List +from django.core.management import BaseCommand +from registrar.management.commands.utility.terminal_helper import TerminalColors, TerminalHelper, ScriptDataHelper +from registrar.models import DomainInformation, DomainRequest +from registrar.models.utility.generic_helper import CreateOrUpdateOrganizationTypeHelper + +logger = logging.getLogger(__name__) + + +class Command(BaseCommand): + help = ( + "Loops through each valid DomainInformation and DomainRequest object and updates its organization_type value. " + "A valid DomainInformation/DomainRequest in this sense is one that has the value None for organization_type. " + "In other words, we populate the organization_type field if it is not already populated." + ) + + def __init__(self): + super().__init__() + # Get lists for DomainRequest + self.request_to_update: List[DomainRequest] = [] + self.request_failed_to_update: List[DomainRequest] = [] + self.request_skipped: List[DomainRequest] = [] + + # Get lists for DomainInformation + self.di_to_update: List[DomainInformation] = [] + self.di_failed_to_update: List[DomainInformation] = [] + self.di_skipped: List[DomainInformation] = [] + + # Define a global variable for all domains with election offices + self.domains_with_election_boards_set = set() + + def add_arguments(self, parser): + """Adds command line arguments""" + parser.add_argument( + "domain_election_board_filename", + help=("A file that contains" " all the domains that are election offices."), + ) + + def handle(self, domain_election_board_filename, **kwargs): + """Loops through each valid Domain object and updates its first_created value""" + + # Check if the provided file path is valid + if not os.path.isfile(domain_election_board_filename): + raise argparse.ArgumentTypeError(f"Invalid file path '{domain_election_board_filename}'") + + # Read the election office csv + self.read_election_board_file(domain_election_board_filename) + + domain_requests = DomainRequest.objects.filter(organization_type__isnull=True) + + # Code execution will stop here if the user prompts "N" + TerminalHelper.prompt_for_execution( + system_exit_on_terminate=True, + info_to_inspect=f""" + ==Proposed Changes== + Number of DomainRequest objects to change: {len(domain_requests)} + + Organization_type data will be added for all of these fields. + """, + prompt_title="Do you wish to process DomainRequest?", + ) + logger.info("Updating DomainRequest(s)...") + + self.update_domain_requests(domain_requests) + + # We should actually be targeting all fields with no value for organization type, + # but do have a value for generic_org_type. This is because there is data that we can infer. + domain_infos = DomainInformation.objects.filter(organization_type__isnull=True) + # Code execution will stop here if the user prompts "N" + TerminalHelper.prompt_for_execution( + system_exit_on_terminate=True, + info_to_inspect=f""" + ==Proposed Changes== + Number of DomainInformation objects to change: {len(domain_infos)} + + Organization_type data will be added for all of these fields. + """, + prompt_title="Do you wish to process DomainInformation?", + ) + logger.info("Updating DomainInformation(s)...") + + self.update_domain_informations(domain_infos) + + def read_election_board_file(self, domain_election_board_filename): + """ + Reads the election board file and adds each parsed domain to self.domains_with_election_boards_set. + As previously implied, this file contains information about Domains which have election boards. + + The file must adhere to this format: + ``` + domain1.gov + domain2.gov + domain3.gov + ``` + (and so on) + """ + with open(domain_election_board_filename, "r") as file: + for line in file: + # Remove any leading/trailing whitespace + domain = line.strip() + if domain not in self.domains_with_election_boards_set: + self.domains_with_election_boards_set.add(domain) + + def update_domain_requests(self, domain_requests): + """ + Updates the organization_type for a list of DomainRequest objects using the `sync_organization_type` function. + Results are then logged. + + This function updates the following variables: + - self.request_to_update list is appended to if the field was updated successfully. + - self.request_skipped list is appended to if the field has `None` for `request.generic_org_type`. + - self.request_failed_to_update list is appended to if an exception is caught during update. + """ + for request in domain_requests: + try: + if request.generic_org_type is not None: + domain_name = None + if request.requested_domain is not None and request.requested_domain.name is not None: + domain_name = request.requested_domain.name + + request_is_approved = request.status == DomainRequest.DomainRequestStatus.APPROVED + if request_is_approved and domain_name is not None and not request.is_election_board: + request.is_election_board = domain_name in self.domains_with_election_boards_set + + self.sync_organization_type(DomainRequest, request) + self.request_to_update.append(request) + logger.info(f"Updating {request} => {request.organization_type}") + else: + self.request_skipped.append(request) + logger.warning(f"Skipped updating {request}. No generic_org_type was found.") + except Exception as err: + self.request_failed_to_update.append(request) + logger.error(err) + logger.error(f"{TerminalColors.FAIL}" f"Failed to update {request}" f"{TerminalColors.ENDC}") + + # Do a bulk update on the organization_type field + ScriptDataHelper.bulk_update_fields( + DomainRequest, self.request_to_update, ["organization_type", "is_election_board", "generic_org_type"] + ) + + # Log what happened + log_header = "============= FINISHED UPDATE FOR DOMAINREQUEST ===============" + TerminalHelper.log_script_run_summary( + self.request_to_update, self.request_failed_to_update, self.request_skipped, True, log_header + ) + + update_skipped_count = len(self.request_to_update) + if update_skipped_count > 0: + logger.warning( + f"""{TerminalColors.MAGENTA} + Note: Entries are skipped when generic_org_type is None + {TerminalColors.ENDC} + """ + ) + + def update_domain_informations(self, domain_informations): + """ + Updates the organization_type for a list of DomainInformation objects + and updates is_election_board if the domain is in the provided csv. + Results are then logged. + + This function updates the following variables: + - self.di_to_update list is appended to if the field was updated successfully. + - self.di_skipped list is appended to if the field has `None` for `request.generic_org_type`. + - self.di_failed_to_update list is appended to if an exception is caught during update. + """ + for info in domain_informations: + try: + if info.generic_org_type is not None: + domain_name = info.domain.name + + if not info.is_election_board: + info.is_election_board = domain_name in self.domains_with_election_boards_set + + self.sync_organization_type(DomainInformation, info) + + self.di_to_update.append(info) + logger.info(f"Updating {info} => {info.organization_type}") + else: + self.di_skipped.append(info) + logger.warning(f"Skipped updating {info}. No generic_org_type was found.") + except Exception as err: + self.di_failed_to_update.append(info) + logger.error(err) + logger.error(f"{TerminalColors.FAIL}" f"Failed to update {info}" f"{TerminalColors.ENDC}") + + # Do a bulk update on the organization_type field + ScriptDataHelper.bulk_update_fields( + DomainInformation, self.di_to_update, ["organization_type", "is_election_board", "generic_org_type"] + ) + + # Log what happened + log_header = "============= FINISHED UPDATE FOR DOMAININFORMATION ===============" + TerminalHelper.log_script_run_summary( + self.di_to_update, self.di_failed_to_update, self.di_skipped, True, log_header + ) + + update_skipped_count = len(self.di_skipped) + if update_skipped_count > 0: + logger.warning( + f"""{TerminalColors.MAGENTA} + Note: Entries are skipped when generic_org_type is None + {TerminalColors.ENDC} + """ + ) + + def sync_organization_type(self, sender, instance): + """ + Updates the organization_type (without saving) to match + the is_election_board and generic_organization_type fields. + """ + + # Define mappings between generic org and election org. + # These have to be defined here, as you'd get a cyclical import error + # otherwise. + + # For any given organization type, return the "_ELECTION" enum equivalent. + # For example: STATE_OR_TERRITORY => STATE_OR_TERRITORY_ELECTION + generic_org_map = DomainRequest.OrgChoicesElectionOffice.get_org_generic_to_org_election() + + # For any given "_election" variant, return the base org type. + # For example: STATE_OR_TERRITORY_ELECTION => STATE_OR_TERRITORY + election_org_map = DomainRequest.OrgChoicesElectionOffice.get_org_election_to_org_generic() + + # Manages the "organization_type" variable and keeps in sync with + # "is_election_board" and "generic_organization_type" + org_type_helper = CreateOrUpdateOrganizationTypeHelper( + sender=sender, + instance=instance, + generic_org_to_org_map=generic_org_map, + election_org_to_generic_org_map=election_org_map, + ) + + org_type_helper.create_or_update_organization_type(force_update=True) diff --git a/src/registrar/management/commands/utility/terminal_helper.py b/src/registrar/management/commands/utility/terminal_helper.py index 49ab89b9a..b54209750 100644 --- a/src/registrar/management/commands/utility/terminal_helper.py +++ b/src/registrar/management/commands/utility/terminal_helper.py @@ -49,6 +49,7 @@ class ScriptDataHelper: Usage: bulk_update_fields(Domain, page.object_list, ["first_ready"]) """ + logger.info(f"{TerminalColors.YELLOW} Bulk updating fields... {TerminalColors.ENDC}") # Create a Paginator object. Bulk_update on the full dataset # is too memory intensive for our current app config, so we can chunk this data instead. paginator = Paginator(update_list, batch_size) @@ -59,13 +60,16 @@ class ScriptDataHelper: class TerminalHelper: @staticmethod - def log_script_run_summary(to_update, failed_to_update, skipped, debug: bool): + def log_script_run_summary(to_update, failed_to_update, skipped, debug: bool, log_header=None): """Prints success, failed, and skipped counts, as well as all affected objects.""" update_success_count = len(to_update) update_failed_count = len(failed_to_update) update_skipped_count = len(skipped) + if log_header is None: + log_header = "============= FINISHED ===============" + # Prepare debug messages debug_messages = { "success": (f"{TerminalColors.OKCYAN}Updated: {to_update}{TerminalColors.ENDC}\n"), @@ -85,7 +89,7 @@ class TerminalHelper: if update_failed_count == 0 and update_skipped_count == 0: logger.info( f"""{TerminalColors.OKGREEN} - ============= FINISHED =============== + {log_header} Updated {update_success_count} entries {TerminalColors.ENDC} """ @@ -93,7 +97,7 @@ class TerminalHelper: elif update_failed_count == 0: logger.warning( f"""{TerminalColors.YELLOW} - ============= FINISHED =============== + {log_header} Updated {update_success_count} entries ----- SOME DATA WAS INVALID (NEEDS MANUAL PATCHING) ----- Skipped updating {update_skipped_count} entries @@ -103,7 +107,7 @@ class TerminalHelper: else: logger.error( f"""{TerminalColors.FAIL} - ============= FINISHED =============== + {log_header} Updated {update_success_count} entries ----- UPDATE FAILED ----- Failed to update {update_failed_count} entries, diff --git a/src/registrar/models/domain_information.py b/src/registrar/models/domain_information.py index ec2a4ca22..c61f1e2a2 100644 --- a/src/registrar/models/domain_information.py +++ b/src/registrar/models/domain_information.py @@ -245,14 +245,17 @@ class DomainInformation(TimeStampedModel): except Exception: return "" - def save(self, *args, **kwargs): - """Save override for custom properties""" + def sync_organization_type(self): + """ + Updates the organization_type (without saving) to match + the is_election_board and generic_organization_type fields. + """ # Define mappings between generic org and election org. # These have to be defined here, as you'd get a cyclical import error # otherwise. - # For any given organization type, return the "_election" variant. + # For any given organization type, return the "_ELECTION" enum equivalent. # For example: STATE_OR_TERRITORY => STATE_OR_TERRITORY_ELECTION generic_org_map = DomainRequest.OrgChoicesElectionOffice.get_org_generic_to_org_election() @@ -271,6 +274,12 @@ class DomainInformation(TimeStampedModel): # Actually updates the organization_type field org_type_helper.create_or_update_organization_type() + + return self + + def save(self, *args, **kwargs): + """Save override for custom properties""" + self.sync_organization_type() super().save(*args, **kwargs) @classmethod diff --git a/src/registrar/models/domain_request.py b/src/registrar/models/domain_request.py index 02324ce61..1e8091c44 100644 --- a/src/registrar/models/domain_request.py +++ b/src/registrar/models/domain_request.py @@ -675,14 +675,16 @@ class DomainRequest(TimeStampedModel): help_text="Notes about this request", ) - def save(self, *args, **kwargs): - """Save override for custom properties""" - + def sync_organization_type(self): + """ + Updates the organization_type (without saving) to match + the is_election_board and generic_organization_type fields. + """ # Define mappings between generic org and election org. # These have to be defined here, as you'd get a cyclical import error # otherwise. - # For any given organization type, return the "_election" variant. + # For any given organization type, return the "_ELECTION" enum equivalent. # For example: STATE_OR_TERRITORY => STATE_OR_TERRITORY_ELECTION generic_org_map = self.OrgChoicesElectionOffice.get_org_generic_to_org_election() @@ -701,6 +703,10 @@ class DomainRequest(TimeStampedModel): # Actually updates the organization_type field org_type_helper.create_or_update_organization_type() + + def save(self, *args, **kwargs): + """Save override for custom properties""" + self.sync_organization_type() super().save(*args, **kwargs) def __str__(self): diff --git a/src/registrar/models/utility/generic_helper.py b/src/registrar/models/utility/generic_helper.py index 32f767ede..892298967 100644 --- a/src/registrar/models/utility/generic_helper.py +++ b/src/registrar/models/utility/generic_helper.py @@ -49,7 +49,7 @@ class CreateOrUpdateOrganizationTypeHelper: self.generic_org_to_org_map = generic_org_to_org_map self.election_org_to_generic_org_map = election_org_to_generic_org_map - def create_or_update_organization_type(self): + def create_or_update_organization_type(self, force_update=False): """The organization_type field on DomainRequest and DomainInformation is consituted from the generic_org_type and is_election_board fields. To keep the organization_type field up to date, we need to update it before save based off of those field @@ -59,6 +59,14 @@ class CreateOrUpdateOrganizationTypeHelper: one of the excluded types (FEDERAL, INTERSTATE, SCHOOL_DISTRICT), the organization_type is set to a corresponding election variant. Otherwise, it directly mirrors the generic_org_type value. + + args: + force_update (bool): If an existing instance has no values to change, + try to update the organization_type field (or related fields) anyway. + This is done by invoking the new instance handler. + + Use to force org type to be updated to the correct value even + if no other changes were made (including is_election). """ # A new record is added with organization_type not defined. @@ -67,7 +75,7 @@ class CreateOrUpdateOrganizationTypeHelper: if is_new_instance: self._handle_new_instance() else: - self._handle_existing_instance() + self._handle_existing_instance(force_update) return self.instance @@ -92,7 +100,7 @@ class CreateOrUpdateOrganizationTypeHelper: # Update the field self._update_fields(organization_type_needs_update, generic_org_type_needs_update) - def _handle_existing_instance(self): + def _handle_existing_instance(self, force_update_when_no_are_changes_found=False): # == Init variables == # # Instance is already in the database, fetch its current state current_instance = self.sender.objects.get(id=self.instance.id) @@ -109,17 +117,19 @@ class CreateOrUpdateOrganizationTypeHelper: # This will not happen in normal flow as it is not possible otherwise. raise ValueError("Cannot update organization_type and generic_org_type simultaneously.") elif not organization_type_changed and (not generic_org_type_changed and not is_election_board_changed): - # No values to update - do nothing - return None - # == Program flow will halt here if there is no reason to update == # + # No changes found + if force_update_when_no_are_changes_found: + # If we want to force an update anyway, we can treat this record like + # its a new one in that we check for "None" values rather than changes. + self._handle_new_instance() + else: + # == Update the linked values == # + # Find out which field needs updating + organization_type_needs_update = generic_org_type_changed or is_election_board_changed + generic_org_type_needs_update = organization_type_changed - # == Update the linked values == # - # Find out which field needs updating - organization_type_needs_update = generic_org_type_changed or is_election_board_changed - generic_org_type_needs_update = organization_type_changed - - # Update the field - self._update_fields(organization_type_needs_update, generic_org_type_needs_update) + # Update the field + self._update_fields(organization_type_needs_update, generic_org_type_needs_update) def _update_fields(self, organization_type_needs_update, generic_org_type_needs_update): """ diff --git a/src/registrar/templates/admin/stacked.html b/src/registrar/templates/admin/stacked.html new file mode 100644 index 000000000..5ca9324df --- /dev/null +++ b/src/registrar/templates/admin/stacked.html @@ -0,0 +1,52 @@ +{% load i18n admin_urls %} +{% load i18n static %} + +{% comment %} +This is copied from Djangos implementation of this template, with added "blocks" +It is not inherently customizable on its own, so we can modify this instead. +https://github.com/django/django/blob/main/django/contrib/admin/templates/admin/edit_inline/stacked.html +{% endcomment %} + +
+ +
+ {% if inline_admin_formset.formset.max_num == 1 %} +

{{ inline_admin_formset.opts.verbose_name|capfirst }}

+ {% else %} +

{{ inline_admin_formset.opts.verbose_name_plural|capfirst }}

+ {% endif %} + {{ inline_admin_formset.formset.management_form }} + {{ inline_admin_formset.formset.non_form_errors }} + + {% for inline_admin_form in inline_admin_formset %}
+

{{ inline_admin_formset.opts.verbose_name|capfirst }}: {% if inline_admin_form.original %}{{ inline_admin_form.original }}{% if inline_admin_form.model_admin.show_change_link and inline_admin_form.model_admin.has_registered_model %} {% if inline_admin_formset.has_change_permission %}{% translate "Change" %}{% else %}{% translate "View" %}{% endif %}{% endif %} + {% else %}#{{ forloop.counter }}{% endif %} + {% if inline_admin_form.show_url %}{% translate "View on site" %}{% endif %} + {% if inline_admin_formset.formset.can_delete and inline_admin_formset.has_delete_permission and inline_admin_form.original %}{{ inline_admin_form.deletion_field.field }} {{ inline_admin_form.deletion_field.label_tag }}{% endif %} +

+ {% if inline_admin_form.form.non_field_errors %} + {{ inline_admin_form.form.non_field_errors }} + {% endif %} + + {% for fieldset in inline_admin_form %} + {# .gov override #} + {% block fieldset %} + {% include "admin/includes/fieldset.html" %} + {% endblock fieldset%} + {# End of .gov override #} + {% endfor %} + + {% if inline_admin_form.needs_explicit_pk_field %} + {{ inline_admin_form.pk_field.field }} + {% endif %} + {% if inline_admin_form.fk_field %} + {{ inline_admin_form.fk_field.field }} + {% endif %} +
+ {% endfor %} +
+ +
diff --git a/src/registrar/templates/django/admin/domain_information_change_form.html b/src/registrar/templates/django/admin/domain_information_change_form.html index f58ee2239..c5b0d54b8 100644 --- a/src/registrar/templates/django/admin/domain_information_change_form.html +++ b/src/registrar/templates/django/admin/domain_information_change_form.html @@ -9,7 +9,9 @@ {% include "django/admin/includes/domain_information_fieldset.html" %} Use detail_table_fieldset as an example, or just extend it. + + original_object is just a variable name for "DomainInformation" or "DomainRequest" {% endcomment %} - {% include "django/admin/includes/detail_table_fieldset.html" %} + {% include "django/admin/includes/detail_table_fieldset.html" with original_object=original %} {% endfor %} {% endblock %} diff --git a/src/registrar/templates/django/admin/domain_request_change_form.html b/src/registrar/templates/django/admin/domain_request_change_form.html index 3b4fa7283..1c8ce2633 100644 --- a/src/registrar/templates/django/admin/domain_request_change_form.html +++ b/src/registrar/templates/django/admin/domain_request_change_form.html @@ -13,8 +13,10 @@ {% include "django/admin/includes/domain_information_fieldset.html" %} Use detail_table_fieldset as an example, or just extend it. + + original_object is just a variable name for "DomainInformation" or "DomainRequest" {% endcomment %} - {% include "django/admin/includes/detail_table_fieldset.html" %} + {% include "django/admin/includes/detail_table_fieldset.html" with original_object=original %} {% endfor %} {% endblock %} @@ -116,8 +118,8 @@ -

- Requested domain: {{ original.requested_domain.name }} +

+ Requested domain: {{ original.requested_domain.name }}

{{ block.super }} diff --git a/src/registrar/templates/django/admin/includes/detail_table_fieldset.html b/src/registrar/templates/django/admin/includes/detail_table_fieldset.html index f346ee155..fb7303352 100644 --- a/src/registrar/templates/django/admin/includes/detail_table_fieldset.html +++ b/src/registrar/templates/django/admin/includes/detail_table_fieldset.html @@ -5,7 +5,7 @@ This is using a custom implementation fieldset.html (see admin/fieldset.html) {% endcomment %} {% block field_readonly %} - {% with all_contacts=original.other_contacts.all %} + {% with all_contacts=original_object.other_contacts.all %} {% if field.field.name == "other_contacts" %} {% if all_contacts.count > 2 %}
@@ -54,7 +54,7 @@ This is using a custom implementation fieldset.html (see admin/fieldset.html) {% elif field.field.name == "alternative_domains" %}
{% with current_path=request.get_full_path %} - {% for alt_domain in original.alternative_domains.all %} + {% for alt_domain in original_object.alternative_domains.all %} {{ alt_domain }}{% if not forloop.last %}, {% endif %} {% endfor %} {% endwith %} @@ -69,24 +69,21 @@ This is using a custom implementation fieldset.html (see admin/fieldset.html) {% if field.field.name == "creator" %}
- {% include "django/admin/includes/contact_detail_list.html" with user=original.creator no_title_top_padding=field.is_readonly %} -
-
- - {% include "django/admin/includes/user_detail_list.html" with user=original.creator no_title_top_padding=field.is_readonly %} + {% include "django/admin/includes/contact_detail_list.html" with user=original_object.creator no_title_top_padding=field.is_readonly %}
+ {% include "django/admin/includes/user_detail_list.html" with user=original_object.creator no_title_top_padding=field.is_readonly %} {% elif field.field.name == "submitter" %}
- {% include "django/admin/includes/contact_detail_list.html" with user=original.submitter no_title_top_padding=field.is_readonly %} + {% include "django/admin/includes/contact_detail_list.html" with user=original_object.submitter no_title_top_padding=field.is_readonly %}
{% elif field.field.name == "authorizing_official" %}
- {% include "django/admin/includes/contact_detail_list.html" with user=original.authorizing_official no_title_top_padding=field.is_readonly %} + {% include "django/admin/includes/contact_detail_list.html" with user=original_object.authorizing_official no_title_top_padding=field.is_readonly %}
- {% elif field.field.name == "other_contacts" and original.other_contacts.all %} - {% with all_contacts=original.other_contacts.all %} + {% elif field.field.name == "other_contacts" and original_object.other_contacts.all %} + {% with all_contacts=original_object.other_contacts.all %} {% if all_contacts.count > 2 %}
Details @@ -104,7 +101,6 @@ This is using a custom implementation fieldset.html (see admin/fieldset.html) {{ contact.title }} {{ contact.email }} - {{ contact.phone }} diff --git a/src/registrar/templates/django/admin/includes/domain_info_inline_stacked.html b/src/registrar/templates/django/admin/includes/domain_info_inline_stacked.html new file mode 100644 index 000000000..414c485e5 --- /dev/null +++ b/src/registrar/templates/django/admin/includes/domain_info_inline_stacked.html @@ -0,0 +1,6 @@ +{% extends 'admin/stacked.html' %} +{% load i18n static %} + +{% block fieldset %} + {% include "django/admin/includes/detail_table_fieldset.html" with original_object=original_object %} +{% endblock %} diff --git a/src/registrar/templates/django/admin/includes/user_detail_list.html b/src/registrar/templates/django/admin/includes/user_detail_list.html index 829af933a..c9ce1c52a 100644 --- a/src/registrar/templates/django/admin/includes/user_detail_list.html +++ b/src/registrar/templates/django/admin/includes/user_detail_list.html @@ -5,24 +5,27 @@ {% with rejected_requests_count=user.get_rejected_requests_count %} {% with ineligible_requests_count=user.get_ineligible_requests_count %} {% if approved_domains_count|add:active_requests_count|add:rejected_requests_count|add:ineligible_requests_count > 0 %} -
    - {% if approved_domains_count > 0 %} - {# Approved domains #} -
  • Approved domains: {{ approved_domains_count }}
    • - {% endif %} - {% if active_requests_count > 0 %} - {# Active requests #} -
  • Active requests: {{ active_requests_count }}
    • - {% endif %} - {% if rejected_requests_count > 0 %} - {# Rejected requests #} -
  • Rejected requests: {{ rejected_requests_count }}
    • - {% endif %} - {% if ineligible_requests_count > 0 %} - {# Ineligible requests #} -
  • Ineligible requests: {{ ineligible_requests_count }}
    • - {% endif %} -
+
+ +
    + {% if approved_domains_count > 0 %} + {# Approved domains #} +
  • Approved domains: {{ approved_domains_count }}
    • + {% endif %} + {% if active_requests_count > 0 %} + {# Active requests #} +
  • Active requests: {{ active_requests_count }}
    • + {% endif %} + {% if rejected_requests_count > 0 %} + {# Rejected requests #} +
  • Rejected requests: {{ rejected_requests_count }}
    • + {% endif %} + {% if ineligible_requests_count > 0 %} + {# Ineligible requests #} +
  • Ineligible requests: {{ ineligible_requests_count }}
    • + {% endif %} +
+
{% endif %} {% endwith %} {% endwith %} diff --git a/src/registrar/tests/common.py b/src/registrar/tests/common.py index a0b0e774f..07dc08f8a 100644 --- a/src/registrar/tests/common.py +++ b/src/registrar/tests/common.py @@ -1152,6 +1152,18 @@ class MockEppLib(TestCase): ], ) + infoDomainFourHosts = fakedEppObject( + "fournameserversDomain.gov", + cr_date=make_aware(datetime(2023, 5, 25, 19, 45, 35)), + contacts=[], + hosts=[ + "ns1.my-nameserver-1.com", + "ns1.my-nameserver-2.com", + "ns1.cats-are-superior3.com", + "ns1.explosive-chicken-nuggets.com", + ], + ) + infoDomainNoHost = fakedEppObject( "my-nameserver.gov", cr_date=make_aware(datetime(2023, 5, 25, 19, 45, 35)), @@ -1452,7 +1464,9 @@ class MockEppLib(TestCase): ) def mockInfoDomainCommands(self, _request, cleaned): - request_name = getattr(_request, "name", None) + request_name = getattr(_request, "name", None).lower() + + print(request_name) # Define a dictionary to map request names to data and extension values request_mappings = { @@ -1474,7 +1488,8 @@ class MockEppLib(TestCase): "nameserverwithip.gov": (self.infoDomainHasIP, None), "namerserversubdomain.gov": (self.infoDomainCheckHostIPCombo, None), "freeman.gov": (self.InfoDomainWithContacts, None), - "threenameserversDomain.gov": (self.infoDomainThreeHosts, None), + "threenameserversdomain.gov": (self.infoDomainThreeHosts, None), + "fournameserversdomain.gov": (self.infoDomainFourHosts, None), "defaultsecurity.gov": (self.InfoDomainWithDefaultSecurityContact, None), "adomain2.gov": (self.InfoDomainWithVerisignSecurityContact, None), "defaulttechnical.gov": (self.InfoDomainWithDefaultTechnicalContact, None), diff --git a/src/registrar/tests/data/fake_election_domains.csv b/src/registrar/tests/data/fake_election_domains.csv new file mode 100644 index 000000000..4ec005bb1 --- /dev/null +++ b/src/registrar/tests/data/fake_election_domains.csv @@ -0,0 +1 @@ +manualtransmission.gov \ No newline at end of file diff --git a/src/registrar/tests/test_admin.py b/src/registrar/tests/test_admin.py index 9ff59767b..abc6b76aa 100644 --- a/src/registrar/tests/test_admin.py +++ b/src/registrar/tests/test_admin.py @@ -85,6 +85,78 @@ class TestDomainAdmin(MockEppLib, WebTest): ) super().setUp() + @less_console_noise_decorator + def test_contact_fields_on_domain_change_form_have_detail_table(self): + """Tests if the contact fields in the inlined Domain information have the detail table + which displays title, email, and phone""" + + # Create fake creator + _creator = User.objects.create( + username="MrMeoward", + first_name="Meoward", + last_name="Jones", + ) + + # Due to the relation between User <==> Contact, + # the underlying contact has to be modified this way. + _creator.contact.email = "meoward.jones@igorville.gov" + _creator.contact.phone = "(555) 123 12345" + _creator.contact.title = "Treat inspector" + _creator.contact.save() + + # Create a fake domain request + domain_request = completed_domain_request(status=DomainRequest.DomainRequestStatus.IN_REVIEW, user=_creator) + domain_request.approve() + _domain_info = DomainInformation.objects.filter(domain=domain_request.approved_domain).get() + domain = Domain.objects.filter(domain_info=_domain_info).get() + + p = "adminpass" + self.client.login(username="superuser", password=p) + response = self.client.get( + "/admin/registrar/domain/{}/change/".format(domain.pk), + follow=True, + ) + + # Make sure the page loaded, and that we're on the right page + self.assertEqual(response.status_code, 200) + self.assertContains(response, domain.name) + + # Check that the fields have the right values. + # == Check for the creator == # + + # Check for the right title, email, and phone number in the response. + # We only need to check for the end tag + # (Otherwise this test will fail if we change classes, etc) + self.assertContains(response, "Treat inspector") + self.assertContains(response, "meoward.jones@igorville.gov") + self.assertContains(response, "(555) 123 12345") + + # Check for the field itself + self.assertContains(response, "Meoward Jones") + + # == Check for the submitter == # + self.assertContains(response, "mayor@igorville.gov") + + self.assertContains(response, "Admin Tester") + self.assertContains(response, "(555) 555 5556") + self.assertContains(response, "Testy2 Tester2") + + # == Check for the authorizing_official == # + self.assertContains(response, "testy@town.com") + self.assertContains(response, "Chief Tester") + self.assertContains(response, "(555) 555 5555") + + # Includes things like readonly fields + self.assertContains(response, "Testy Tester") + + # == Test the other_employees field == # + self.assertContains(response, "testy2@town.com") + self.assertContains(response, "Another Tester") + self.assertContains(response, "(555) 555 5557") + + # Test for the copy link + self.assertContains(response, "usa-button__clipboard") + @patch("registrar.admin.DomainAdmin._get_current_date", return_value=date(2024, 1, 1)) def test_extend_expiration_date_button(self, mock_date_today): """ @@ -1509,7 +1581,7 @@ class TestDomainRequestAdmin(MockEppLib): # Since we're using client to mock the request, we can only test against # non-interpolated values - expected_content = "Requested domain:" + expected_content = "Requested domain:" expected_content2 = '' expected_content3 = '
' not_expected_content = "submit-row-wrapper--analyst-view>" @@ -1538,7 +1610,7 @@ class TestDomainRequestAdmin(MockEppLib): # Since we're using client to mock the request, we can only test against # non-interpolated values - expected_content = "Requested domain:" + expected_content = "Requested domain:" expected_content2 = '' expected_content3 = '
' self.assertContains(request, expected_content) diff --git a/src/registrar/tests/test_management_scripts.py b/src/registrar/tests/test_management_scripts.py index 34178e262..26161b272 100644 --- a/src/registrar/tests/test_management_scripts.py +++ b/src/registrar/tests/test_management_scripts.py @@ -7,6 +7,9 @@ from django.test import TestCase from registrar.models import ( User, Domain, + DomainRequest, + Contact, + Website, DomainInvitation, TransitionDomain, DomainInformation, @@ -18,7 +21,284 @@ from django.core.management import call_command from unittest.mock import patch, call from epplibwrapper import commands, common -from .common import MockEppLib, less_console_noise +from .common import MockEppLib, less_console_noise, completed_domain_request +from api.tests.common import less_console_noise_decorator + + +class TestPopulateOrganizationType(MockEppLib): + """Tests for the populate_organization_type script""" + + def setUp(self): + """Creates a fake domain object""" + super().setUp() + + # Get the domain requests + self.domain_request_1 = completed_domain_request( + name="lasers.gov", + generic_org_type=DomainRequest.OrganizationChoices.FEDERAL, + is_election_board=True, + status=DomainRequest.DomainRequestStatus.IN_REVIEW, + ) + self.domain_request_2 = completed_domain_request( + name="readysetgo.gov", + generic_org_type=DomainRequest.OrganizationChoices.CITY, + status=DomainRequest.DomainRequestStatus.IN_REVIEW, + ) + self.domain_request_3 = completed_domain_request( + name="manualtransmission.gov", + generic_org_type=DomainRequest.OrganizationChoices.TRIBAL, + status=DomainRequest.DomainRequestStatus.IN_REVIEW, + ) + self.domain_request_4 = completed_domain_request( + name="saladandfries.gov", + generic_org_type=DomainRequest.OrganizationChoices.TRIBAL, + is_election_board=True, + status=DomainRequest.DomainRequestStatus.IN_REVIEW, + ) + + # Approve all three requests + self.domain_request_1.approve() + self.domain_request_2.approve() + self.domain_request_3.approve() + self.domain_request_4.approve() + + # Get the domains + self.domain_1 = Domain.objects.get(name="lasers.gov") + self.domain_2 = Domain.objects.get(name="readysetgo.gov") + self.domain_3 = Domain.objects.get(name="manualtransmission.gov") + self.domain_4 = Domain.objects.get(name="saladandfries.gov") + + # Get the domain infos + self.domain_info_1 = DomainInformation.objects.get(domain=self.domain_1) + self.domain_info_2 = DomainInformation.objects.get(domain=self.domain_2) + self.domain_info_3 = DomainInformation.objects.get(domain=self.domain_3) + self.domain_info_4 = DomainInformation.objects.get(domain=self.domain_4) + + def tearDown(self): + """Deletes all DB objects related to migrations""" + super().tearDown() + + # Delete domains and related information + Domain.objects.all().delete() + DomainInformation.objects.all().delete() + DomainRequest.objects.all().delete() + User.objects.all().delete() + Contact.objects.all().delete() + Website.objects.all().delete() + + @less_console_noise_decorator + def run_populate_organization_type(self): + """ + This method executes the populate_organization_type command. + + The 'call_command' function from Django's management framework is then used to + execute the populate_organization_type command with the specified arguments. + """ + with patch( + "registrar.management.commands.utility.terminal_helper.TerminalHelper.query_yes_no_exit", # noqa + return_value=True, + ): + call_command("populate_organization_type", "registrar/tests/data/fake_election_domains.csv") + + def assert_expected_org_values_on_request_and_info( + self, + domain_request: DomainRequest, + domain_info: DomainInformation, + expected_values: dict, + ): + """ + This is a helper function that tests the following conditions: + 1. DomainRequest and DomainInformation (on given objects) are equivalent + 2. That generic_org_type, is_election_board, and organization_type are equal to passed in values + + Args: + domain_request (DomainRequest): The DomainRequest object to test + + domain_info (DomainInformation): The DomainInformation object to test + + expected_values (dict): Container for what we expect is_electionboard, generic_org_type, + and organization_type to be on DomainRequest and DomainInformation. + Example: + expected_values = { + "is_election_board": False, + "generic_org_type": DomainRequest.OrganizationChoices.CITY, + "organization_type": DomainRequest.OrgChoicesElectionOffice.CITY, + } + """ + + # Test domain request + with self.subTest(field="DomainRequest"): + self.assertEqual(domain_request.generic_org_type, expected_values["generic_org_type"]) + self.assertEqual(domain_request.is_election_board, expected_values["is_election_board"]) + self.assertEqual(domain_request.organization_type, expected_values["organization_type"]) + + # Test domain info + with self.subTest(field="DomainInformation"): + self.assertEqual(domain_info.generic_org_type, expected_values["generic_org_type"]) + self.assertEqual(domain_info.is_election_board, expected_values["is_election_board"]) + self.assertEqual(domain_info.organization_type, expected_values["organization_type"]) + + def do_nothing(self): + """Does nothing for mocking purposes""" + pass + + def test_request_and_info_city_not_in_csv(self): + """ + Tests what happens to a city domain that is not defined in the CSV. + + Scenario: A domain request (of type city) is made that is not defined in the CSV file. + When a domain request is made for a city that is not listed in the CSV, + Then the `is_election_board` value should remain False, + and the `generic_org_type` and `organization_type` should both be `city`. + + Expected Result: The `is_election_board` and `generic_org_type` attributes should be unchanged. + The `organization_type` field should now be `city`. + """ + + city_request = self.domain_request_2 + city_info = self.domain_request_2 + + # Make sure that all data is correct before proceeding. + # Since the presave fixture is in effect, we should expect that + # is_election_board is equal to none, even though we tried to define it as "True" + expected_values = { + "is_election_board": False, + "generic_org_type": DomainRequest.OrganizationChoices.CITY, + "organization_type": DomainRequest.OrgChoicesElectionOffice.CITY, + } + self.assert_expected_org_values_on_request_and_info(city_request, city_info, expected_values) + + # Run the populate script + try: + self.run_populate_organization_type() + except Exception as e: + self.fail(f"Could not run populate_organization_type script. Failed with exception: {e}") + + # All values should be the same + self.assert_expected_org_values_on_request_and_info(city_request, city_info, expected_values) + + def test_request_and_info_federal(self): + """ + Tests what happens to a federal domain after the script is run (should be unchanged). + + Scenario: A domain request (of type federal) is processed after running the populate_organization_type script. + When a federal domain request is made, + Then the `is_election_board` value should remain None, + and the `generic_org_type` and `organization_type` fields should both be `federal`. + + Expected Result: The `is_election_board` and `generic_org_type` attributes should be unchanged. + The `organization_type` field should now be `federal`. + """ + federal_request = self.domain_request_1 + federal_info = self.domain_info_1 + + # Make sure that all data is correct before proceeding. + # Since the presave fixture is in effect, we should expect that + # is_election_board is equal to none, even though we tried to define it as "True" + expected_values = { + "is_election_board": None, + "generic_org_type": DomainRequest.OrganizationChoices.FEDERAL, + "organization_type": DomainRequest.OrgChoicesElectionOffice.FEDERAL, + } + self.assert_expected_org_values_on_request_and_info(federal_request, federal_info, expected_values) + + # Run the populate script + try: + self.run_populate_organization_type() + except Exception as e: + self.fail(f"Could not run populate_organization_type script. Failed with exception: {e}") + + # All values should be the same + self.assert_expected_org_values_on_request_and_info(federal_request, federal_info, expected_values) + + def test_request_and_info_tribal_add_election_office(self): + """ + Tests if a tribal domain in the election csv changes organization_type to TRIBAL - ELECTION + for the domain request and the domain info + """ + + # Set org type fields to none to mimic an environment without this data + tribal_request = self.domain_request_3 + tribal_request.organization_type = None + tribal_info = self.domain_info_3 + tribal_info.organization_type = None + with patch.object(DomainRequest, "sync_organization_type", self.do_nothing): + with patch.object(DomainInformation, "sync_organization_type", self.do_nothing): + tribal_request.save() + tribal_info.save() + + # Make sure that all data is correct before proceeding. + expected_values = { + "is_election_board": False, + "generic_org_type": DomainRequest.OrganizationChoices.TRIBAL, + "organization_type": None, + } + self.assert_expected_org_values_on_request_and_info(tribal_request, tribal_info, expected_values) + + # Run the populate script + try: + self.run_populate_organization_type() + except Exception as e: + self.fail(f"Could not run populate_organization_type script. Failed with exception: {e}") + + tribal_request.refresh_from_db() + tribal_info.refresh_from_db() + + # Because we define this in the "csv", we expect that is election board will switch to True, + # and organization_type will now be tribal_election + expected_values["is_election_board"] = True + expected_values["organization_type"] = DomainRequest.OrgChoicesElectionOffice.TRIBAL_ELECTION + + self.assert_expected_org_values_on_request_and_info(tribal_request, tribal_info, expected_values) + + def test_request_and_info_tribal_doesnt_remove_election_office(self): + """ + Tests if a tribal domain in the election csv changes organization_type to TRIBAL_ELECTION + when the is_election_board is True, and generic_org_type is Tribal when it is not + present in the CSV. + + To avoid overwriting data, the script should not set any domain specified as + an election_office (that doesn't exist in the CSV) to false. + """ + + # Set org type fields to none to mimic an environment without this data + tribal_election_request = self.domain_request_4 + tribal_election_info = self.domain_info_4 + tribal_election_request.organization_type = None + tribal_election_info.organization_type = None + with patch.object(DomainRequest, "sync_organization_type", self.do_nothing): + with patch.object(DomainInformation, "sync_organization_type", self.do_nothing): + tribal_election_request.save() + tribal_election_info.save() + + # Make sure that all data is correct before proceeding. + # Because the presave fixture is in place when creating this, we should expect that the + # organization_type variable is already pre-populated. We will test what happens when + # it is not in another test. + expected_values = { + "is_election_board": True, + "generic_org_type": DomainRequest.OrganizationChoices.TRIBAL, + "organization_type": None, + } + self.assert_expected_org_values_on_request_and_info( + tribal_election_request, tribal_election_info, expected_values + ) + + # Run the populate script + try: + self.run_populate_organization_type() + except Exception as e: + self.fail(f"Could not run populate_organization_type script. Failed with exception: {e}") + + # If we don't define this in the "csv", but the value was already true, + # we expect that is election board will stay True, and the org type will be tribal, + # and organization_type will now be tribal_election + expected_values["organization_type"] = DomainRequest.OrgChoicesElectionOffice.TRIBAL_ELECTION + tribal_election_request.refresh_from_db() + tribal_election_info.refresh_from_db() + self.assert_expected_org_values_on_request_and_info( + tribal_election_request, tribal_election_info, expected_values + ) class TestPopulateFirstReady(TestCase): diff --git a/src/registrar/tests/test_views_domain.py b/src/registrar/tests/test_views_domain.py index 064c5efdb..3a5ce7e7b 100644 --- a/src/registrar/tests/test_views_domain.py +++ b/src/registrar/tests/test_views_domain.py @@ -5,7 +5,7 @@ from django.conf import settings from django.urls import reverse from django.contrib.auth import get_user_model -from .common import MockSESClient, create_user # type: ignore +from .common import MockEppLib, MockSESClient, create_user # type: ignore from django_webtest import WebTest # type: ignore import boto3_mocking # type: ignore @@ -71,11 +71,14 @@ class TestWithDomainPermissions(TestWithUser): # that inherit this setUp self.domain_dnssec_none, _ = Domain.objects.get_or_create(name="dnssec-none.gov") + self.domain_with_four_nameservers, _ = Domain.objects.get_or_create(name="fournameserversDomain.gov") + self.domain_information, _ = DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_dsdata) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_multdsdata) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_dnssec_none) + DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_with_four_nameservers) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_with_ip) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_just_nameserver) DomainInformation.objects.get_or_create(creator=self.user, domain=self.domain_on_hold) @@ -98,6 +101,11 @@ class TestWithDomainPermissions(TestWithUser): domain=self.domain_dnssec_none, role=UserDomainRole.Roles.MANAGER, ) + UserDomainRole.objects.get_or_create( + user=self.user, + domain=self.domain_with_four_nameservers, + role=UserDomainRole.Roles.MANAGER, + ) UserDomainRole.objects.get_or_create( user=self.user, domain=self.domain_with_ip, @@ -727,7 +735,7 @@ class TestDomainManagers(TestDomainOverview): self.assertContains(home_page, self.domain.name) -class TestDomainNameservers(TestDomainOverview): +class TestDomainNameservers(TestDomainOverview, MockEppLib): def test_domain_nameservers(self): """Can load domain's nameservers page.""" page = self.client.get(reverse("domain-dns-nameservers", kwargs={"pk": self.domain.id})) @@ -974,6 +982,117 @@ class TestDomainNameservers(TestDomainOverview): page = result.follow() self.assertContains(page, "The name servers for this domain have been updated") + def test_domain_nameservers_can_blank_out_first_or_second_one_if_enough_entries(self): + """Nameserver form submits successfully with 2 valid inputs, even if the first or + second entries are blanked out. + + Uses self.app WebTest because we need to interact with forms. + """ + + nameserver1 = "" + nameserver2 = "ns2.igorville.gov" + nameserver3 = "ns3.igorville.gov" + valid_ip = "" + valid_ip_2 = "128.0.0.2" + valid_ip_3 = "128.0.0.3" + nameservers_page = self.app.get(reverse("domain-dns-nameservers", kwargs={"pk": self.domain.id})) + session_id = self.app.cookies[settings.SESSION_COOKIE_NAME] + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + nameservers_page.form["form-0-server"] = nameserver1 + nameservers_page.form["form-0-ip"] = valid_ip + nameservers_page.form["form-1-server"] = nameserver2 + nameservers_page.form["form-1-ip"] = valid_ip_2 + nameservers_page.form["form-2-server"] = nameserver3 + nameservers_page.form["form-2-ip"] = valid_ip_3 + with less_console_noise(): # swallow log warning message + result = nameservers_page.form.submit() + + # form submission was a successful post, response should be a 302 + self.assertEqual(result.status_code, 302) + self.assertEqual( + result["Location"], + reverse("domain-dns-nameservers", kwargs={"pk": self.domain.id}), + ) + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + nameservers_page = result.follow() + self.assertContains(nameservers_page, "The name servers for this domain have been updated") + + nameserver1 = "ns1.igorville.gov" + nameserver2 = "" + nameserver3 = "ns3.igorville.gov" + valid_ip = "128.0.0.1" + valid_ip_2 = "" + valid_ip_3 = "128.0.0.3" + session_id = self.app.cookies[settings.SESSION_COOKIE_NAME] + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + nameservers_page.form["form-0-server"] = nameserver1 + nameservers_page.form["form-0-ip"] = valid_ip + nameservers_page.form["form-1-server"] = nameserver2 + nameservers_page.form["form-1-ip"] = valid_ip_2 + nameservers_page.form["form-2-server"] = nameserver3 + nameservers_page.form["form-2-ip"] = valid_ip_3 + with less_console_noise(): # swallow log warning message + result = nameservers_page.form.submit() + + # form submission was a successful post, response should be a 302 + self.assertEqual(result.status_code, 302) + self.assertEqual( + result["Location"], + reverse("domain-dns-nameservers", kwargs={"pk": self.domain.id}), + ) + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + nameservers_page = result.follow() + self.assertContains(nameservers_page, "The name servers for this domain have been updated") + + def test_domain_nameservers_can_blank_out_first_and_second_one_if_enough_entries(self): + """Nameserver form submits successfully with 2 valid inputs, even if the first and + second entries are blanked out. + + Uses self.app WebTest because we need to interact with forms. + """ + + # We need to start with a domain with 4 nameservers otherwise the formset in the test environment + # will only have 3 forms + nameserver1 = "" + nameserver2 = "" + nameserver3 = "ns3.igorville.gov" + nameserver4 = "ns4.igorville.gov" + valid_ip = "" + valid_ip_2 = "" + valid_ip_3 = "" + valid_ip_4 = "" + nameservers_page = self.app.get( + reverse("domain-dns-nameservers", kwargs={"pk": self.domain_with_four_nameservers.id}) + ) + + session_id = self.app.cookies[settings.SESSION_COOKIE_NAME] + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + + # Minimal check to ensure the form is loaded correctly + self.assertEqual(nameservers_page.form["form-0-server"].value, "ns1.my-nameserver-1.com") + self.assertEqual(nameservers_page.form["form-3-server"].value, "ns1.explosive-chicken-nuggets.com") + + nameservers_page.form["form-0-server"] = nameserver1 + nameservers_page.form["form-0-ip"] = valid_ip + nameservers_page.form["form-1-server"] = nameserver2 + nameservers_page.form["form-1-ip"] = valid_ip_2 + nameservers_page.form["form-2-server"] = nameserver3 + nameservers_page.form["form-2-ip"] = valid_ip_3 + nameservers_page.form["form-3-server"] = nameserver4 + nameservers_page.form["form-3-ip"] = valid_ip_4 + with less_console_noise(): # swallow log warning message + result = nameservers_page.form.submit() + + # form submission was a successful post, response should be a 302 + self.assertEqual(result.status_code, 302) + self.assertEqual( + result["Location"], + reverse("domain-dns-nameservers", kwargs={"pk": self.domain_with_four_nameservers.id}), + ) + self.app.set_cookie(settings.SESSION_COOKIE_NAME, session_id) + nameservers_page = result.follow() + self.assertContains(nameservers_page, "The name servers for this domain have been updated") + def test_domain_nameservers_form_invalid(self): """Nameserver form does not submit with invalid data. diff --git a/src/registrar/utility/csv_export.py b/src/registrar/utility/csv_export.py index 949b0adcd..8787f9e74 100644 --- a/src/registrar/utility/csv_export.py +++ b/src/registrar/utility/csv_export.py @@ -87,10 +87,10 @@ def parse_row_for_domain( if security_email.lower() in invalid_emails: security_email = "(blank)" - if domain_info.federal_type and domain_info.generic_org_type == DomainRequest.OrganizationChoices.FEDERAL: - domain_type = f"{domain_info.get_generic_org_type_display()} - {domain_info.get_federal_type_display()}" + if domain_info.federal_type and domain_info.organization_type == DomainRequest.OrgChoicesElectionOffice.FEDERAL: + domain_type = f"{domain_info.get_organization_type_display()} - {domain_info.get_federal_type_display()}" else: - domain_type = domain_info.get_generic_org_type_display() + domain_type = domain_info.get_organization_type_display() # create a dictionary of fields which can be included in output FIELDS = { @@ -319,9 +319,9 @@ def parse_row_for_requests(columns, request: DomainRequest): requested_domain_name = request.requested_domain.name if request.federal_type: - request_type = f"{request.get_generic_org_type_display()} - {request.get_federal_type_display()}" + request_type = f"{request.get_organization_type_display()} - {request.get_federal_type_display()}" else: - request_type = request.get_generic_org_type_display() + request_type = request.get_organization_type_display() # create a dictionary of fields which can be included in output FIELDS = { @@ -399,7 +399,7 @@ def export_data_type_to_csv(csv_file): # Coalesce is used to replace federal_type of None with ZZZZZ sort_fields = [ - "generic_org_type", + "organization_type", Coalesce("federal_type", Value("ZZZZZ")), "federal_agency", "domain__name", @@ -432,7 +432,7 @@ def export_data_full_to_csv(csv_file): ] # Coalesce is used to replace federal_type of None with ZZZZZ sort_fields = [ - "generic_org_type", + "organization_type", Coalesce("federal_type", Value("ZZZZZ")), "federal_agency", "domain__name", @@ -465,13 +465,13 @@ def export_data_federal_to_csv(csv_file): ] # Coalesce is used to replace federal_type of None with ZZZZZ sort_fields = [ - "generic_org_type", + "organization_type", Coalesce("federal_type", Value("ZZZZZ")), "federal_agency", "domain__name", ] filter_condition = { - "generic_org_type__icontains": "federal", + "organization_type__icontains": "federal", "domain__state__in": [ Domain.State.READY, Domain.State.DNS_NEEDED, @@ -601,7 +601,6 @@ def get_sliced_domains(filter_condition): def get_sliced_requests(filter_condition): """Get filtered requests counts sliced by org type and election office.""" - requests = DomainRequest.objects.all().filter(**filter_condition).distinct() requests_count = requests.count() federal = requests.filter(generic_org_type=DomainRequest.OrganizationChoices.FEDERAL).distinct().count()