The console will have 2 different "updatable things":
- only ADMINs (GAE-admins and users in the support G-Suite group) can change the things in the "admin settings" tab (currently just the allowed TLDs)
- only OWNERs can change things from the other tabs: WHOIS info, certificates, whitelisted IPs, contacts etc.
Also, all ADMINs are now OWNERS of "non-REAL" registrars. Meaning - we're only
preventing ADMINs from editing "REAL" registrars (usually in production).
Specifically, OTE registrars on sandbox are NOT "REAL", meaning ADMINS will
still be able to update them.
This only changes the backend (registrar-settings endpoint). As-is, the console
website will still make ADMINs *think* they can change everything, but if they
try - they will get an error.
Changing the frontend will happen in the next CL - because I want to get this
out this release cycle and getting JS reviewed takes a long time :(
TESTED=deployed to alpha, and saw I can't update fields even as admin on REAL
registrars, but could change it on non-REAL registrars. Also checked that I can
update the allowed TLDs on REAL registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222698270
EppConsoleAction still "manually" checks access by going over the
RegistrarContacts. We need it to use AuthenticatedRegistrarAccessor just like
every other part of the registrar console.
We still need to remove the (now unneeded) login EPP sent by the console, but that's left for a followup CL.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222404208
This is a temporary measure until we implement access control for Support.
Once we implement access control, we will only block Support from removing TLDs
on production.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222180321
It is starting to be used in more places than just ur/server/registrar. Even now it's used in the RDAP, and we are going to start using it for the registrar-xhr endpoint meaning it will be used in EPP flows as well.
Also logically - this is part of the request authentication.
While moving - we also refactor it to make it easier to use in tests. Instead of mocking, we will be able to create instances with arbitrary roles.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221645055
After this CL, "support" accounts (accounts that are part of the "support" G-Suite group) will the same access to the registrar console as GCP "admins". However, they don't won't have access to the GCP project itself.
We could give them their own Role in the future (say SUPPORT) and give them different access than "admins", but right now we don't need it and YAGNI or something :)
NOTE: we identify users by their email (they need to be logged in to a google account). I don't know if that's best practice, since I guess different google accounts might have the same email address. However, G-Suite groups' membership is by email so there's not much we can do about it if we want to use G-Suite groups.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=220804273
This tab will set the "allowedTlds", but might have other functionality in the
future.
It is based on (branches from) the security-settings tab, because I'm copying the functionality of the "whitelisted IPs" to the "allowed TLDs": they are both lists of "arbitrary" strings that you can remove from and add to.
There are a lot of moving parts in this CL, because of how all the different elements need to interact, and how intertwined they are (for example, we need to disable the admin-settings view for non admins both in the soy and in the JS code)
It's really time to refactor the console given all we've learned... :/
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=220373443
Cardinality of this metric:
clientId: there are currently 650 (on sandbox, because of OTE), and 200 on production.
explicitClientId: 2
roles: 2 now, might be 3 soon if we add vendors
status: 2
So we're talking about a cardinality of 2,000-8,000. Less when you consider that registrars only seldom actually need to access the console (certainly not daily or even weekly).
Compare with, e.g., the /epp/processing_time from the above EppMetrics.java which has:
Epp commands: 26 (manual counting)
client IDs: 200 on prod
status: the actual status CODE of the command. Can have many values, but looking at the past few weeks' metrics I counted 20
Note that not every command results in every status. Looking a few weeks back we can see around 80-100 (commands+status) combination.
buckets: 16
so that's over 250,000-1,000,000 cardinality, on a very high-volume metric.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218699280
This CL removes the "READ vs UPDATE" feature completely. Now anyone with access
has full read+write access.
We still keep track of which role a user has (did they get access "explicitly"
because they are an "allowed access" contact? Or do they have access because
they are admins?) for the logs and UI, and also so we could in the (very near)
future have features only available to admins.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218169608
This allows us to inject it with Dagger and avoid using InjectRule to set it
in unit tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217571475
Whenever a registrar is changed via the registrar console, we send out a
notification of that change.
Since we're going to allow Admins and soon Vendors to use the console in
addition to the registrars, it becomes important to know who actually performed
the changes if the registrars complain.
In addition, we will now send notifications for changes in Sandbox since we're
going to actually allow registrars to update sandbox data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217539534
SessionUtil is renames AuthenticatedRegistrarAccessor, as it's used to access a registrar for an authenticated user.
It will now be injected with the AuthResult instead of receiving it in every function call, since there's only one "legal" AuthResult to use.
The AccessType names are changed from READ_ONLY/READ_WRITE to READ/UPDATE, as it was confusing that a user could have both READ_ONLY AND READ_WRITE access to the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216958306
We want to be able to view / test / debug how the registrar console looks for our clients.
However, we don't want to accidentally change the data for registrars, especially in a "non-accountable" way (where we later don't know who did that change)
So we do 2 things here:
- Add a "mode" (read-only and read-write) to the getRegistrarForUser function. We set it according to what we want to do with the registrar. Currently, read-write is only requested for the "update" RegistrarSetting action. Admins will have read-only access to all registrars, but read-write access only to the "admin registrar" (or whatever registrar they are contacts for).
- Support an undocumented "clientId=XXX" query param that replaces the "guessClientIdForUser" function in the original page load. We can then set it when we want to view a different account.
We also change the navigation links on the HTML page to preserve the query.
-------------------------
This might be used also for a better user experience for our clients, especially those with multiple "clientId"s (some registrar entities have multiple "registrar" objects)
Currently, they have to have a separate user for each clientId, and only have one user allowed which has both read and write permissions.
Using this change, we can give them the possibility to add users on their own, some with read-only access (to view billing information without being able to change anything), and use a single user for all their clientIds.
-------------------------
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215480610
RegistrarSettingsAction is a JSON in / JSON out endpoint, meaning the reply is consumed as JSON.
The current state is that if an error occurs, there are two possible replies:
- a JSON error reply is sent out, or
- a 402 HTML reply is sent out with the exception.getMessage()
The difference is only - do we actively catch the exception to translate it to JSON or not.
This fix catches ALL exceptions and translates them to JSON format. Note that there's no security change by giving the getMessage in the JSON reply since we were returning that anyway (in the HTML).
In addition - changed the "gaeUserId" to "user.getEmail" as the identifier, since it's clearer to the users who see that error - and I do want to transition to a more "email identifier" way of checking access (since that's what users put in the registrar contact info)
This too isn't leaking new information because
- the initial HTML page load already gives the user's email, and
- the logs already log the user's email for every request
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215213807
Replaced the plethora of inter winding access functions and inputs in SessionUtils with just 2 functions, that both accept the same type for the user (AuthResult):
guessRegistrarForUser: given an AuthResult, finds a registrar that they have access to. If none is found - a ForbiddenException is thrown.
getRegistrarForUser[Cached]: (maybe should be called getRegistrarOnBehalfOfUser?) given an AuthResult and a clientId, loads and returns the registrar ONLY IF the user has access to it. Otherwise throws a ForbiddenException.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214630657
This is an intermediate CL, part of the Registrar Console cleanup.
TL;DR:
- the current state: resource.js points to a resource TYPE on the server (only registrars can be resources right now), but the specific resource is selected based on the user (we select the "first resource of this type that the user has access to)
- new state: resource.js points to a SPECIFIC resource (TYPE + ID).
In this CL the server still chooses the resource like before (first one that user has access to) but we make sure the returned resource is the same one we requested.
In a subsequent CL we will use the requested ID to load the resource, and then make sure the user has access to that resource.
---------------------------
When loading the RegistrarConsole HTML page, the server determines which clientId belongs to the user ("guesses" it by looking for the first registrar that has this user as contact). It sends the relevant clientId back with the page load.
However, this information isn't currently used in the JS requests to read / update the registrar. Instead, currently the client ID is guessed again for each JS access to the server. It is also saved again in the client's "session" cookie.
As a result, it is theoretically possible to have the JS access a different clientID than the original page load (not likely, since it requires a single user registered for multiple registrars AND that the contacts change for the original registrar).
So our goal is to only have a single clientID "value" instead of the 3 we currently have for JS requests (the one from the initial page load, the one saved in the session cookie, the one guessed on the JS request)
As a first step, we send over the "initial page load" clientId on every JS request, and make sure the "session + guessed" value is equal to that one. Later we will remove the "session+guessed" values from the RegistrarSettings, using the "initial page load" clientID instead.
In addition to the "nicer code" implications, having the clientID from the initial page load always used means it'll be easy to have a clientID selection option for users who have access to multiple clientIDs (such as admins)
SECURITY NOTE:the choice of clientID has no security implication since we make sure the user has access to the clientID no matter how we actually choose the clientID on every single server request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214459506
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).
This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.
This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
We never launched this, don't planning on launching it now anyway, and it's rotted over the past two years anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202993577
Also adds Guy to list of CONTRIBUTORS and removes Xooglers (who definitely won't be contributing more code).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202476218
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
An admin that isn't associated with any clientId, will default to the
registryAdminClientId.
However, if we set the registryAdminClientId as the session's
CLIENT_ID_ATTRIBUTE, the next time we access the server we have a client-id
attribute we aren't associated with - which returns a "403 Registrar Console
access revoked" error (the assumption is - we were associated with it before
but aren't anymore)
To fix this - we just add all admins as "hasAccessTo" registryAdminClientId, even if it's not in the contacts. This will let admins stay on the admin registrar, without affecting where they log-in initially if they are also contacts in different registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402856
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.
No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.
Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
This is a 'yellow' Flogger migration CL. Yellow CLs should be mostly safe
but include changes that are notable for one reason or another. Manual
intervention may be required to address small issues.
The comments in this CL indicate cases where suggested code changes
should be double checked, or even modified. There may even be cases where
files outside this CL are affected by changes to things such as logger
visibility. However if a change does not have an associated comment then
it should be safe.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097990
This is in preparation for merging and then removing
RegistrarPremiumPriceAckAction.
This includes:
test that the data the UI sent isn't stale
---------------------------------------------
Our system is "read, modify, write". However, if between the "read" and the "write" someone else changed the registry, my write will undo their change even if I didn't touch any of their fields.
To solve that - we use the "lastUpdateTime" timestamp of the registrar. the UI reads it with the rest of the data, and sends it back on "write". We will now make sure the registrar currently in datastore has the same timestamp.
support premium-price-ack flag
---------------------------------
Add support for reading and writing this flag. We still won't be using it - that's in a followup CL, but we support it.
support changing the URL
------------------------
Add changing the URL in the UI, under the "whois" section
Will replace the Ack endpoint with this (and remove that endpoint) in a followup CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192154078
The registrar security console failed because it assumed the email is a
required field for the registrar, but it isn't (at least - create_registrar
doesn't require an email, and update_registrar lets you remove the email).
Fixed by allowing it to *remain* unset if it was unset originally, but if it was set - it's required.
There are more fixes needed, but they aren't related to the email, so they will wait for the next CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191623034
Caching turns out to be an anti-pattern for the console. If we use it, changes from the user just get obliterated by the older, cached version the next time the console refreshes (and it happens to refresh after every update). Caching is also not very useful here, as the amount of database access driven by the console is very small.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190650931
Implement a checkbox in the "Resources" tab to allow registrars to toggle
their "premium price ack required" flag.
Tested:
Verfied the console functionality by hand. I've started work on an
automated test, but we can't actually test those from blaze and the
kokoro tests are way too time-consuming to be practical for development, so
we're going to have to either find a way to run those locally outside of
the normal process or make do without a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190212177
SoySyntaxException is an abstract exception type and is never even declared to be thrown (all declarations about this changed about 2 years ago). So places catching it should either change to catch SoyCompilationException, or do nothing and let it propagate.
Tested:
TAP sample presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185050724
This fixes up the following problems:
1. Using string concatenation instead of the formatting variant methods.
2. Logging or swallowing exception messages without logging the exception
itself (this swallows the stack trace).
3. Unnecessary logging on re-thrown exceptions.
4. Unnecessary use of formatting variant methods when not necessary.
5. Complicated logging statements involving significant processing not being
wrapped inside of a logging level check.
6. Redundant logging both of an exception itself and its message (this is
unnecessary duplication).
7. Use of the base Logger class instead of our FormattingLogger class.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182419837
Last commit did not pick up all the changes because MOE incorrectly attributed some changes to the wrong commit. This commit should reconcile these. Also picked up some changes to how hamcrest library is depended upon in BUILD file, which should have been included in previous commits.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=177637931
This was a surprisingly involved change. Some of the difficulties included
java.util.Optional purposely not being Serializable (so I had to move a
few Optionals in mapreduce classes to @Nullable) and having to add the Truth
Java8 extension library for assertion support.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171863777
This allows admins to access the registrar console without needing to be added
as a registrar contact. If they are a registrar contact, then that registrar
takes precedence.
Tested=In Alpha
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164182623
When the registrar console code determines that a user has not logged in, it redirects to a login page. But when authenticating as an internal request (which should never happen), the redirection code encountered an exception, resulting in a 500 error.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=163867018
This makes the code more understandable from callsites, and also forces
users of this function to deal with the situation where the registrar
with a given client ID might not be present (it was previously silently
NPEing from some of the callsites).
This also adds a test helper method loadRegistrar(clientId) that retains
the old functionality for terseness in tests. It also fixes some instances
of using the load method with the wrong cachedness -- some uses in high-
traffic situations (WHOIS) that should have caching, but also low-traffic
reporting that don't benefit from caching so might as well always be
current.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162990468
We want to be safer and more explicit about the authentication needed by the many actions that exist.
As such, we make the 'auth' parameter required in @Action (so it's always clear who can run a specific action) and we replace the @Auth with an enum so that only pre-approved configurations that are aptly named and documented can be used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162210306
