Reapply "Merge branch 'main' into za/2348-csv-export-org-member-domain-export"

This reverts commit 8becad8186.
This commit is contained in:
zandercymatics 2024-08-01 09:53:59 -06:00
parent 8becad8186
commit 86f040e4b7
No known key found for this signature in database
GPG key ID: FF4636ABEC9682B7
9 changed files with 221 additions and 54 deletions

View file

@ -14,13 +14,29 @@ assignees: abroddrick
## Installation
There are several tools we use locally that you will need to have.
- [ ] [Install the cf CLI v7](https://docs.cloudfoundry.org/cf-cli/install-go-cli.html#pkg-mac) for the ability to deploy
There are several tools we use locally that you will need to have.
- [ ] [Cloudfoundry CLI](https://docs.cloudfoundry.org/cf-cli/install-go-cli.html#pkg-mac) Note: If you are on Windows the cli will be under `cf8` or `cf7` depending on which version you install.
- If you are using Windows, installation information can be found [here](https://github.com/cloudfoundry/cli/wiki/V8-CLI-Installation-Guide#installers-and-compressed-binaries)
- Alternatively, for Windows, [consider using chocolately](https://community.chocolatey.org/packages/cloudfoundry-cli/7.2.0)
- [ ] Make sure you have `gpg` >2.1.7. Run `gpg --version` to check. If not, [install gnupg](https://formulae.brew.sh/formula/gnupg)
- Alternatively, you can skip this step and [use ssh keys](#setting-up-commit-signing-with-ssh) instead
- [ ] Install the [Github CLI](https://cli.github.com/)
- [ ] [GPG](https://gnupg.org/download/)
- Make sure you have `gpg` >2.1.7. Run `gpg --version` to check. If not, [install gnupg](https://formulae.brew.sh/formula/gnupg)
- This may not work on DHS devices. Alternatively, you can [use ssh keys](#setting-up-commit-signing-with-ssh) instead.
- [ ] Docker Community Edition*
- [ ] Git*
- [ ] VSCode (our preferred editor)*
- [ ] Github Desktop* or the Github CLI*
The following tools are optional but recommended. For DHS devices, these can be requested through the DHS IT portal:
- [ ] Slack Desktop App**
- [ ] Python 3.10*
- [ ] NodeJS (latest version available)*
- [ ] Putty*
- [ ] Windows Subsystem for Linux*
* Must be requested through DHS IT portal on DHS devices
** Downloadable via DHS Software Center
## Access
@ -37,7 +53,12 @@ cf login -a api.fr.cloud.gov --sso
**Note:** As mentioned in the [Login documentation](https://developers.login.gov/testing/), the sandbox Login account is different account from your regular, production Login account. If you have not created a Login account for the sandbox before, you will need to create a new account first.
- [ ] Optional- add yourself as a codeowner if desired. See the [Developer readme](https://github.com/cisagov/getgov/blob/main/docs/developer/README.md) for how to do this and what it does.
Follow the [.gov onboarding dev setup instructions](https://docs.google.com/document/d/1ukbpW4LSqkb_CCt8LWfpehP03qqfyYfvK3Fl21NaEq8/edit#heading=h.94jwfwkpkhdx). Confirm you successfully set up the following accounts:
- [ ] Identity sandbox accounts - 1 superuser access account and 1 analyst access account.
- [ ] Login.gov account to access stable
**Optional**
- [ ] Add yourself as a codeowner if desired. See the [Developer readme](https://github.com/cisagov/getgov/blob/main/docs/developer/README.md) for how to do this and what it does.
### Steps for the onboarder
- [ ] Add the onboardee to cloud.gov org (cisa-dotgov)
@ -124,3 +145,19 @@ Additionally, consider a gpg key manager like Kleopatra if you run into issues w
We have three types of environments: stable, staging, and sandbox. Stable (production)and staging (pre-prod) get deployed via tagged release, and developer sandboxes are given to get.gov developers to mess around in a production-like environment without disrupting stable or staging. Each sandbox is namespaced and will automatically be deployed too when the appropriate branch syntax is used for that space in an open pull request. There are several things you need to setup to make the sandbox work for a developer.
All automation for setting up a developer sandbox is documented in the scripts for [creating a developer sandbox](../../ops/scripts/create_dev_sandbox.sh) and [removing a developer sandbox](../../ops/scripts/destroy_dev_sandbox.sh). A Cloud.gov organization administrator will have to perform the script in order to create the sandbox.
## Known Issues
### SSL Verification Failure
Some developers using Government Furnished Equipment (GFE) have problems using tools such as git and pip due to SSL verification failurse. This happens because GFE has a custom certificate chain installed, but these tools use their own certificate bundles. As a result, when they try to verify an ssl connection, they cannot and so the connection fails. To resolve this in pip you can use --use-feature=truststore to direct pip to use the local certificate store. If you are running into this issue when using git on windows, run ```git config --global http.sslbackend schannel```.
If you are running into these issues in a docker container you will need to export the root certificate and pull it into the container. Ask another developer how to do this properly.
### Puppeteer Download Error
When building the node image either individually or with docker compose, there may be an error caused by a node package call puppeteer. This can be resolved by adding `ENV PUPPETEER_SKIP_DOWNLOAD=true` to [node.Dockerfile](../../src/node.Dockerfile) after the COPY command.
### Checksum Error
There is an unresolved issue with python package installation that occurs after the above SSL Verification failure has been resolved. It often manifests as a checksum error, where the hash of a download .whl file (python package) does not match the expected value. This appears to be because pythonhosted.org is cutting off download connections to some devices for some packages (the behavior is somewhat inconsistent). We have outstanding issues with PyPA and DHS IT to fix this. In the meantime we have a [workaround](#developing-using-docker).
## Developing Using Docker
While we have unresolved issues with certain devices, you can pull a pre-built docker image from matthewswspence/getgov-base that comes with all the needed packages installed. To do this, you will need to change the very first line in the main [Dockerfile](../../src/Dockerfile) to `FROM matthewswspence/getgov-base:latest`. Note: this change will need to be reverted before any branch can be merged. Additionally, this will only resolve the [checksum error](#checksum-error), you will still need to resolve any other issues through the listed instructions. We are actively working to resolve this inconvenience.

View file

@ -1169,6 +1169,8 @@ document.addEventListener('DOMContentLoaded', function() {
const statusIndicator = document.querySelector('.domain__filter-indicator');
const statusToggle = document.querySelector('.usa-button--filter');
const noPortfolioFlag = document.getElementById('no-portfolio-js-flag');
const portfolioElement = document.getElementById('portfolio-js-value');
const portfolioValue = portfolioElement ? portfolioElement.getAttribute('data-portfolio') : null;
/**
* Loads rows in the domains list, as well as updates pagination around the domains list
@ -1178,10 +1180,15 @@ document.addEventListener('DOMContentLoaded', function() {
* @param {*} order - the sort order {asc, desc}
* @param {*} scroll - control for the scrollToElement functionality
* @param {*} searchTerm - the search term
* @param {*} portfolio - the portfolio id
*/
function loadDomains(page, sortBy = currentSortBy, order = currentOrder, scroll = scrollToTable, status = currentStatus, searchTerm = currentSearchTerm) {
function loadDomains(page, sortBy = currentSortBy, order = currentOrder, scroll = scrollToTable, status = currentStatus, searchTerm = currentSearchTerm, portfolio = portfolioValue) {
// fetch json of page of domains, given params
fetch(`/get-domains-json/?page=${page}&sort_by=${sortBy}&order=${order}&status=${status}&search_term=${searchTerm}`)
let url = `/get-domains-json/?page=${page}&sort_by=${sortBy}&order=${order}&status=${status}&search_term=${searchTerm}`
if (portfolio)
url += `&portfolio=${portfolio}`
fetch(url)
.then(response => response.json())
.then(data => {
if (data.error) {

View file

@ -0,0 +1,38 @@
# Generated by Django 4.2.10 on 2024-07-25 12:45
import django.contrib.postgres.fields
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
("registrar", "0113_user_portfolio_user_portfolio_additional_permissions_and_more"),
]
operations = [
migrations.AlterField(
model_name="user",
name="portfolio_additional_permissions",
field=django.contrib.postgres.fields.ArrayField(
base_field=models.CharField(
choices=[
("view_all_domains", "View all domains and domain reports"),
("view_managed_domains", "View managed domains"),
("view_member", "View members"),
("edit_member", "Create and edit members"),
("view_all_requests", "View all requests"),
("view_created_requests", "View created requests"),
("edit_requests", "Create and edit requests"),
("view_portfolio", "View organization"),
("edit_portfolio", "Edit organization"),
],
max_length=50,
),
blank=True,
help_text="Select one or more additional permissions.",
null=True,
size=None,
),
),
]

View file

@ -77,11 +77,6 @@ class User(AbstractUser):
VIEW_ALL_DOMAINS = "view_all_domains", "View all domains and domain reports"
VIEW_MANAGED_DOMAINS = "view_managed_domains", "View managed domains"
# EDIT_DOMAINS is really self.domains. We add is hear and leverage it in has_permission
# so we have one way to test for portfolio and domain edit permissions
# Do we need to check for portfolio domains specifically?
# NOTE: A user on an org can currently invite a user outside the org
EDIT_DOMAINS = "edit_domains", "User is a manager on a domain"
VIEW_MEMBER = "view_member", "View members"
EDIT_MEMBER = "edit_member", "Create and edit members"
@ -269,11 +264,6 @@ class User(AbstractUser):
def _has_portfolio_permission(self, portfolio_permission):
"""The views should only call this function when testing for perms and not rely on roles."""
# EDIT_DOMAINS === user is a manager on a domain (has UserDomainRole)
# NOTE: Should we check whether the domain is in the portfolio?
if portfolio_permission == self.UserPortfolioPermissionChoices.EDIT_DOMAINS and self.domains.exists():
return True
if not self.portfolio:
return False
@ -287,21 +277,14 @@ class User(AbstractUser):
return self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_PORTFOLIO)
def has_domains_portfolio_permission(self):
return (
self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_ALL_DOMAINS)
or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_MANAGED_DOMAINS)
# or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.EDIT_DOMAINS)
)
def has_edit_domains_portfolio_permission(self):
return self._has_portfolio_permission(User.UserPortfolioPermissionChoices.EDIT_DOMAINS)
return self._has_portfolio_permission(
User.UserPortfolioPermissionChoices.VIEW_ALL_DOMAINS
) or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_MANAGED_DOMAINS)
def has_domain_requests_portfolio_permission(self):
return (
self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_ALL_REQUESTS)
or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_CREATED_REQUESTS)
# or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.EDIT_REQUESTS)
)
return self._has_portfolio_permission(
User.UserPortfolioPermissionChoices.VIEW_ALL_REQUESTS
) or self._has_portfolio_permission(User.UserPortfolioPermissionChoices.VIEW_CREATED_REQUESTS)
def has_view_all_domains_permission(self):
"""Determines if the current user can view all available domains in a given portfolio"""

View file

@ -2,7 +2,6 @@
<section class="section--outlined domain-requests" id="domain-requests">
<div class="grid-row">
<!-- Use portfolio_base_permission when merging into 2366 and then delete this comment -->
{% if portfolio is None %}
<div class="mobile:grid-col-12 desktop:grid-col-6">
<h2 id="domain-requests-header" class="flex-6">Domain requests</h2>

View file

@ -1,11 +1,15 @@
{% load static %}
<section class="section--outlined domains{% if portfolio is not None %} margin-top-0{% endif %}" id="domains">
<div class="section--outlined__header margin-bottom-3 {% if portfolio is None %} section--outlined__header--no-portfolio justify-content-space-between{% else %} grid-row{% endif %}">
<!-- Use portfolio_base_permission when merging into 2366 then delete this comment -->
<div class="grid-row">
{% if portfolio is None %}
<h2 id="domains-header" class="display-inline-block">Domains</h2>
<span class="display-none" id="no-portfolio-js-flag"></span>
<div class="mobile:grid-col-12 desktop:grid-col-6">
<h2 id="domains-header" class="flex-6">Domains</h2>
</div>
<span class="display-none" id="no-portfolio-js-flag"></span>
{% else %}
<!-- Embedding the portfolio value in a data attribute -->
<span id="portfolio-js-value" data-portfolio="{{ portfolio.id }}"></span>
{% endif %}
<div class="section--outlined__search {% if portfolio %} mobile:grid-col-12 desktop:grid-col-6{% endif %}">
<section aria-label="Domains search component" class="margin-top-2">
@ -47,7 +51,6 @@
</div>
{% endif %}
</div>
<!-- Use portfolio_base_permission when merging into 2366 then delete this comment -->
{% if portfolio %}
<div class="display-flex flex-align-center">
<span class="margin-right-2 margin-top-neg-1 usa-prose text-base-darker">Filter by</span>
@ -151,7 +154,6 @@
<th data-sortable="name" scope="col" role="columnheader">Domain name</th>
<th data-sortable="expiration_date" scope="col" role="columnheader">Expires</th>
<th data-sortable="state_display" scope="col" role="columnheader">Status</th>
<!-- Use portfolio_base_permission when merging into 2366 then delete this comment -->
{% if portfolio %}
<th data-sortable="suborganization" scope="col" role="columnheader">Suborganization</th>
{% endif %}

View file

@ -1292,7 +1292,6 @@ class TestUser(TestCase):
1. Returns False when a user does not have a portfolio
2. Returns True when user has direct permission
3. Returns True when user has permission through a role
4. Returns True EDIT_DOMAINS when user does not have the perm but has UserDomainRole
Note: This tests _get_portfolio_permissions as a side effect
"""
@ -1304,11 +1303,9 @@ class TestUser(TestCase):
user_can_view_all_domains = self.user.has_domains_portfolio_permission()
user_can_view_all_requests = self.user.has_domain_requests_portfolio_permission()
user_can_edit_domains = self.user.has_edit_domains_portfolio_permission()
self.assertFalse(user_can_view_all_domains)
self.assertFalse(user_can_view_all_requests)
self.assertFalse(user_can_edit_domains)
self.user.portfolio = portfolio
self.user.save()
@ -1316,11 +1313,9 @@ class TestUser(TestCase):
user_can_view_all_domains = self.user.has_domains_portfolio_permission()
user_can_view_all_requests = self.user.has_domain_requests_portfolio_permission()
user_can_edit_domains = self.user.has_edit_domains_portfolio_permission()
self.assertTrue(user_can_view_all_domains)
self.assertFalse(user_can_view_all_requests)
self.assertFalse(user_can_edit_domains)
self.user.portfolio_roles = [User.UserPortfolioRoleChoices.ORGANIZATION_ADMIN]
self.user.save()
@ -1328,11 +1323,9 @@ class TestUser(TestCase):
user_can_view_all_domains = self.user.has_domains_portfolio_permission()
user_can_view_all_requests = self.user.has_domain_requests_portfolio_permission()
user_can_edit_domains = self.user.has_edit_domains_portfolio_permission()
self.assertTrue(user_can_view_all_domains)
self.assertTrue(user_can_view_all_requests)
self.assertFalse(user_can_edit_domains)
UserDomainRole.objects.all().get_or_create(
user=self.user, domain=self.domain, role=UserDomainRole.Roles.MANAGER
@ -1340,11 +1333,9 @@ class TestUser(TestCase):
user_can_view_all_domains = self.user.has_domains_portfolio_permission()
user_can_view_all_requests = self.user.has_domain_requests_portfolio_permission()
user_can_edit_domains = self.user.has_edit_domains_portfolio_permission()
self.assertTrue(user_can_view_all_domains)
self.assertTrue(user_can_view_all_requests)
self.assertTrue(user_can_edit_domains)
Portfolio.objects.all().delete()

View file

@ -1,4 +1,4 @@
from registrar.models import UserDomainRole, Domain
from registrar.models import UserDomainRole, Domain, DomainInformation, Portfolio
from django.urls import reverse
from .test_views import TestWithUser
from django_webtest import WebTest # type: ignore
@ -15,15 +15,25 @@ class GetDomainsJsonTest(TestWithUser, WebTest):
self.domain1 = Domain.objects.create(name="example1.com", expiration_date="2024-01-01", state="unknown")
self.domain2 = Domain.objects.create(name="example2.com", expiration_date="2024-02-01", state="dns needed")
self.domain3 = Domain.objects.create(name="example3.com", expiration_date="2024-03-01", state="ready")
self.domain4 = Domain.objects.create(name="example4.com", expiration_date="2024-03-01", state="ready")
# Create UserDomainRoles
UserDomainRole.objects.create(user=self.user, domain=self.domain1)
UserDomainRole.objects.create(user=self.user, domain=self.domain2)
UserDomainRole.objects.create(user=self.user, domain=self.domain3)
# Create Portfolio
self.portfolio = Portfolio.objects.create(creator=self.user, organization_name="Example org")
# Add domain3 and domain4 to portfolio
DomainInformation.objects.create(creator=self.user, domain=self.domain3, portfolio=self.portfolio)
DomainInformation.objects.create(creator=self.user, domain=self.domain4, portfolio=self.portfolio)
def tearDown(self):
super().tearDown()
UserDomainRole.objects.all().delete()
DomainInformation.objects.all().delete()
Portfolio.objects.all().delete()
super().tearDown()
@less_console_noise_decorator
def test_get_domains_json_unauthenticated(self):
@ -104,6 +114,82 @@ class GetDomainsJsonTest(TestWithUser, WebTest):
)
self.assertEqual(svg_icon_expected, svg_icons[i])
@less_console_noise_decorator
def test_get_domains_json_with_portfolio(self):
"""Test that an authenticated user gets the list of 2 domains for portfolio."""
response = self.app.get(reverse("get_domains_json"), {"portfolio": self.portfolio.id})
self.assertEqual(response.status_code, 200)
data = response.json
# Check pagination info
self.assertEqual(data["page"], 1)
self.assertFalse(data["has_next"])
self.assertFalse(data["has_previous"])
self.assertEqual(data["num_pages"], 1)
# Check the number of domains
self.assertEqual(len(data["domains"]), 2)
# Expected domains
expected_domains = [self.domain3, self.domain4]
# Extract fields from response
domain_ids = [domain["id"] for domain in data["domains"]]
names = [domain["name"] for domain in data["domains"]]
expiration_dates = [domain["expiration_date"] for domain in data["domains"]]
states = [domain["state"] for domain in data["domains"]]
state_displays = [domain["state_display"] for domain in data["domains"]]
get_state_help_texts = [domain["get_state_help_text"] for domain in data["domains"]]
action_urls = [domain["action_url"] for domain in data["domains"]]
action_labels = [domain["action_label"] for domain in data["domains"]]
svg_icons = [domain["svg_icon"] for domain in data["domains"]]
# Check fields for each domain
for i, expected_domain in enumerate(expected_domains):
self.assertEqual(expected_domain.id, domain_ids[i])
self.assertEqual(expected_domain.name, names[i])
self.assertEqual(expected_domain.expiration_date, expiration_dates[i])
self.assertEqual(expected_domain.state, states[i])
# Parsing the expiration date from string to date
parsed_expiration_date = parse_date(expiration_dates[i])
expected_domain.expiration_date = parsed_expiration_date
# Check state_display and get_state_help_text
self.assertEqual(expected_domain.state_display(), state_displays[i])
self.assertEqual(expected_domain.get_state_help_text(), get_state_help_texts[i])
self.assertEqual(reverse("domain", kwargs={"pk": expected_domain.id}), action_urls[i])
# Check action_label
user_domain_role_exists = UserDomainRole.objects.filter(
domain_id=expected_domains[i].id, user=self.user
).exists()
action_label_expected = (
"View"
if not user_domain_role_exists
or expected_domains[i].state
in [
Domain.State.DELETED,
Domain.State.ON_HOLD,
]
else "Manage"
)
self.assertEqual(action_label_expected, action_labels[i])
# Check svg_icon
svg_icon_expected = (
"visibility"
if expected_domains[i].state
in [
Domain.State.DELETED,
Domain.State.ON_HOLD,
]
else "settings"
)
self.assertEqual(svg_icon_expected, svg_icons[i])
@less_console_noise_decorator
def test_get_domains_json_search(self):
"""Test search."""

View file

@ -6,6 +6,8 @@ from django.contrib.auth.decorators import login_required
from django.urls import reverse
from django.db.models import Q
from registrar.models.domain_information import DomainInformation
logger = logging.getLogger(__name__)
@ -14,9 +16,9 @@ def get_domains_json(request):
"""Given the current request,
get all domains that are associated with the User object"""
domain_ids = request.user.get_user_domain_ids()
domain_ids = get_domain_ids_from_request(request)
objects = Domain.objects.filter(id__in=domain_ids)
objects = Domain.objects.filter(id__in=domain_ids).select_related("domain_info__sub_organization")
unfiltered_total = objects.count()
objects = apply_search(objects, request)
@ -27,7 +29,7 @@ def get_domains_json(request):
page_number = request.GET.get("page")
page_obj = paginator.get_page(page_number)
domains = [serialize_domain(domain) for domain in page_obj.object_list]
domains = [serialize_domain(domain, request.user) for domain in page_obj.object_list]
return JsonResponse(
{
@ -42,6 +44,21 @@ def get_domains_json(request):
)
def get_domain_ids_from_request(request):
"""Get domain ids from request.
If portfolio specified, return domain ids associated with portfolio.
Otherwise, return domain ids associated with request.user.
"""
portfolio = request.GET.get("portfolio")
if portfolio:
domain_infos = DomainInformation.objects.filter(portfolio=portfolio)
return domain_infos.values_list("domain_id", flat=True)
else:
user_domain_roles = UserDomainRole.objects.filter(user=request.user)
return user_domain_roles.values_list("domain_id", flat=True)
def apply_search(queryset, request):
search_term = request.GET.get("search_term")
if search_term:
@ -93,7 +110,7 @@ def apply_sorting(queryset, request):
return queryset.order_by(sort_by)
def serialize_domain(domain):
def serialize_domain(domain, user):
suborganization_name = None
try:
domain_info = domain.domain_info
@ -105,6 +122,9 @@ def serialize_domain(domain):
domain_info = None
logger.debug(f"Issue in domains_json: We could not find domain_info for {domain}")
# Check if there is a UserDomainRole for this domain and user
user_domain_role_exists = UserDomainRole.objects.filter(domain_id=domain.id, user=user).exists()
return {
"id": domain.id,
"name": domain.name,
@ -113,7 +133,11 @@ def serialize_domain(domain):
"state_display": domain.state_display(),
"get_state_help_text": domain.get_state_help_text(),
"action_url": reverse("domain", kwargs={"pk": domain.id}),
"action_label": ("View" if domain.state in [Domain.State.DELETED, Domain.State.ON_HOLD] else "Manage"),
"action_label": (
"View"
if not user_domain_role_exists or domain.state in [Domain.State.DELETED, Domain.State.ON_HOLD]
else "Manage"
),
"svg_icon": ("visibility" if domain.state in [Domain.State.DELETED, Domain.State.ON_HOLD] else "settings"),
"suborganization": suborganization_name,
}