Cardinality of this metric:
clientId: there are currently 650 (on sandbox, because of OTE), and 200 on production.
explicitClientId: 2
roles: 2 now, might be 3 soon if we add vendors
status: 2
So we're talking about a cardinality of 2,000-8,000. Less when you consider that registrars only seldom actually need to access the console (certainly not daily or even weekly).
Compare with, e.g., the /epp/processing_time from the above EppMetrics.java which has:
Epp commands: 26 (manual counting)
client IDs: 200 on prod
status: the actual status CODE of the command. Can have many values, but looking at the past few weeks' metrics I counted 20
Note that not every command results in every status. Looking a few weeks back we can see around 80-100 (commands+status) combination.
buckets: 16
so that's over 250,000-1,000,000 cardinality, on a very high-volume metric.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218699280
This CL removes the "READ vs UPDATE" feature completely. Now anyone with access
has full read+write access.
We still keep track of which role a user has (did they get access "explicitly"
because they are an "allowed access" contact? Or do they have access because
they are admins?) for the logs and UI, and also so we could in the (very near)
future have features only available to admins.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=218169608
This allows us to inject it with Dagger and avoid using InjectRule to set it
in unit tests.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217571475
Whenever a registrar is changed via the registrar console, we send out a
notification of that change.
Since we're going to allow Admins and soon Vendors to use the console in
addition to the registrars, it becomes important to know who actually performed
the changes if the registrars complain.
In addition, we will now send notifications for changes in Sandbox since we're
going to actually allow registrars to update sandbox data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217539534
SessionUtil is renames AuthenticatedRegistrarAccessor, as it's used to access a registrar for an authenticated user.
It will now be injected with the AuthResult instead of receiving it in every function call, since there's only one "legal" AuthResult to use.
The AccessType names are changed from READ_ONLY/READ_WRITE to READ/UPDATE, as it was confusing that a user could have both READ_ONLY AND READ_WRITE access to the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216958306
We want to be able to view / test / debug how the registrar console looks for our clients.
However, we don't want to accidentally change the data for registrars, especially in a "non-accountable" way (where we later don't know who did that change)
So we do 2 things here:
- Add a "mode" (read-only and read-write) to the getRegistrarForUser function. We set it according to what we want to do with the registrar. Currently, read-write is only requested for the "update" RegistrarSetting action. Admins will have read-only access to all registrars, but read-write access only to the "admin registrar" (or whatever registrar they are contacts for).
- Support an undocumented "clientId=XXX" query param that replaces the "guessClientIdForUser" function in the original page load. We can then set it when we want to view a different account.
We also change the navigation links on the HTML page to preserve the query.
-------------------------
This might be used also for a better user experience for our clients, especially those with multiple "clientId"s (some registrar entities have multiple "registrar" objects)
Currently, they have to have a separate user for each clientId, and only have one user allowed which has both read and write permissions.
Using this change, we can give them the possibility to add users on their own, some with read-only access (to view billing information without being able to change anything), and use a single user for all their clientIds.
-------------------------
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215480610
RegistrarSettingsAction is a JSON in / JSON out endpoint, meaning the reply is consumed as JSON.
The current state is that if an error occurs, there are two possible replies:
- a JSON error reply is sent out, or
- a 402 HTML reply is sent out with the exception.getMessage()
The difference is only - do we actively catch the exception to translate it to JSON or not.
This fix catches ALL exceptions and translates them to JSON format. Note that there's no security change by giving the getMessage in the JSON reply since we were returning that anyway (in the HTML).
In addition - changed the "gaeUserId" to "user.getEmail" as the identifier, since it's clearer to the users who see that error - and I do want to transition to a more "email identifier" way of checking access (since that's what users put in the registrar contact info)
This too isn't leaking new information because
- the initial HTML page load already gives the user's email, and
- the logs already log the user's email for every request
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215213807
Replaced the plethora of inter winding access functions and inputs in SessionUtils with just 2 functions, that both accept the same type for the user (AuthResult):
guessRegistrarForUser: given an AuthResult, finds a registrar that they have access to. If none is found - a ForbiddenException is thrown.
getRegistrarForUser[Cached]: (maybe should be called getRegistrarOnBehalfOfUser?) given an AuthResult and a clientId, loads and returns the registrar ONLY IF the user has access to it. Otherwise throws a ForbiddenException.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214630657
This is an intermediate CL, part of the Registrar Console cleanup.
TL;DR:
- the current state: resource.js points to a resource TYPE on the server (only registrars can be resources right now), but the specific resource is selected based on the user (we select the "first resource of this type that the user has access to)
- new state: resource.js points to a SPECIFIC resource (TYPE + ID).
In this CL the server still chooses the resource like before (first one that user has access to) but we make sure the returned resource is the same one we requested.
In a subsequent CL we will use the requested ID to load the resource, and then make sure the user has access to that resource.
---------------------------
When loading the RegistrarConsole HTML page, the server determines which clientId belongs to the user ("guesses" it by looking for the first registrar that has this user as contact). It sends the relevant clientId back with the page load.
However, this information isn't currently used in the JS requests to read / update the registrar. Instead, currently the client ID is guessed again for each JS access to the server. It is also saved again in the client's "session" cookie.
As a result, it is theoretically possible to have the JS access a different clientID than the original page load (not likely, since it requires a single user registered for multiple registrars AND that the contacts change for the original registrar).
So our goal is to only have a single clientID "value" instead of the 3 we currently have for JS requests (the one from the initial page load, the one saved in the session cookie, the one guessed on the JS request)
As a first step, we send over the "initial page load" clientId on every JS request, and make sure the "session + guessed" value is equal to that one. Later we will remove the "session+guessed" values from the RegistrarSettings, using the "initial page load" clientID instead.
In addition to the "nicer code" implications, having the clientID from the initial page load always used means it'll be easy to have a clientID selection option for users who have access to multiple clientIDs (such as admins)
SECURITY NOTE:the choice of clientID has no security implication since we make sure the user has access to the clientID no matter how we actually choose the clientID on every single server request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214459506
We have several "consoles" ("views" in the registrar console). They all affect the same resource - the registrar itself.
Currently, each view creates its own "RESTful resource", even though it's the same resource for all of them, meaning we have copied code and copied "URL endpoint" across multiple files.
I assume this was made so that IF one day we have a "view" to another resource, we can easily add it. But we currently don't have any such view, nor plans to add any.
So according to the YAGNI paradigm, it's better to move the resource creation outside of the console.
Also, IF we do add a view to a different resource - it'll still be more readable to have a map from the "view" to the "resource URL endpoint" alongside the existing map from the "view" to the "console"...
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213992967
This fixes the Kokoro tests broken by []
Unfortunately this means that the return type changes from a Map to an Object as well,
so all of the .get() lines need to turn into []. Yay weakly typed languages. And
apparently in some error conditions this method could already return the Object type
anyway, even if you passed in a Map, so it's just poorly designed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213808477
This properly fixes the issue that was suppressed in [] and
just started breaking today thanks to []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213510500
The original thought was that the actions you can do on a resource is:
- create it
- read it
- update it
(I guess you should have "delete" as well, but that isn't currently there)
Although we use "read" and "update", we never use "create". So having it goes against the YAGNI principle :)
Also, it had a bug: when sending a "create", the opt_newId in send_() would
permanentily change the uri of the request, causing any subsequent request to
go to the wrong endpoint.
By removing the "create" we can simplify the rest of the code (the send_() function).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213029499
ModulesService does not provide a great API. Specifically, it doesn't have a
way to get the hostname for a specific service; you have to get the hostname for
a specific version as well. This is very rarely what we want, as we publish new
versions every week and don't expect old ones to hang around for very long, so
a task should execute against whatever the live version is, not whatever the
current version was back when the task was enqueued (especially because that
version might be deleted by now).
This new and improved wrapper API removes the confusion and plays better with
dependency injection to boot. We can also fold in other methods having to do
with App Engine services, whereas ModulesService was quite limited in scope.
This also has the side effect of fixing ResaveEntityAction, which is
currently broken because the tasks it's enqueuing to execute up to 30 days in
the future have the version hard-coded into the hostname, and we typically
delete old versions sooner than that.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206173763
The code stopped working after a recent soy-to-js compiler change.
Impacts include:
- The /registrar#domain page only shows part of the contents.
- RegistrarConsoleScreenshotTest times out.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=204135349
We never launched this, don't planning on launching it now anyway, and it's rotted over the past two years anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202993577
Also adds Guy to list of CONTRIBUTORS and removes Xooglers (who definitely won't be contributing more code).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202476218
This removes the following unnecessary imports:
//third_party/java/activation
//third_party/java/bouncycastle
//third_party/java/bouncycastle_bcpg
//third_party/java/dagger
//third_party/java/dnsjava
//third_party/java/jaxws_api
//third_party/java/jcommander
//third_party/java/joda_money
//third_party/java/joda_time
//third_party/java/json_simple
//third_party/java/junit
//third_party/java/mockito
//third_party/java/re2j
//third_party/java/servlet/servlet_api
//third_party/java/truth:truth8
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/activation,//third_party/java/bouncycastle,//third_party/java/bouncycastle_bcpg,//third_party/java/dagger,//third_party/java/dnsjava,//third_party/java/jaxws_api,//third_party/java/jcommander,//third_party/java/joda_money,//third_party/java/joda_time,//third_party/java/json_simple,//third_party/java/junit,//third_party/java/mockito,//third_party/java/re2j,//third_party/java/servlet/servlet_api,//third_party/java/truth:truth8'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202344774
This removes the following unnecessary imports:
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
[]
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='[],[],[],[],[],[],[],[],[],[],[],[],[],[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202325520
This affects JSR305, JSR330, and Guava annotations.
The exact command run to generate this CL was:
build_cleaner '//third_party/java_src/gtld/...' -c '' --dep_restrictions='//third_party/java/jsr330_inject,//third_party/java/jsr305_annotations,[]'
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202322747
It only needs the error page HTML files; everything else isn't used by endpoints
served by the backend service and only serves to increase build times
(especially compiling all that JS).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=202229514
Now that the large zone re-signing test is complete, we no longer need it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199507075
An admin that isn't associated with any clientId, will default to the
registryAdminClientId.
However, if we set the registryAdminClientId as the session's
CLIENT_ID_ATTRIBUTE, the next time we access the server we have a client-id
attribute we aren't associated with - which returns a "403 Registrar Console
access revoked" error (the assumption is - we were associated with it before
but aren't anymore)
To fix this - we just add all admins as "hasAccessTo" registryAdminClientId, even if it's not in the contacts. This will let admins stay on the admin registrar, without affecting where they log-in initially if they are also contacts in different registrars.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=200402856
This is a 'green' Flogger migration CL. Green CLs are intended to be as
safe as possible and should be easy to review and submit.
No changes should be necessary to the code itself prior to submission,
but small changes to BUILD files may be required.
Changes within files are completely independent of each other, so this CL
can be safely split up for review using tools such as Rosie.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198560170
This is a 'yellow' Flogger migration CL. Yellow CLs should be mostly safe
but include changes that are notable for one reason or another. Manual
intervention may be required to address small issues.
The comments in this CL indicate cases where suggested code changes
should be double checked, or even modified. There may even be cases where
files outside this CL are affected by changes to things such as logger
visibility. However if a change does not have an associated comment then
it should be safe.
For more information, see []
Base CL: 197826149
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097990
This is in preparation for merging and then removing
RegistrarPremiumPriceAckAction.
This includes:
test that the data the UI sent isn't stale
---------------------------------------------
Our system is "read, modify, write". However, if between the "read" and the "write" someone else changed the registry, my write will undo their change even if I didn't touch any of their fields.
To solve that - we use the "lastUpdateTime" timestamp of the registrar. the UI reads it with the rest of the data, and sends it back on "write". We will now make sure the registrar currently in datastore has the same timestamp.
support premium-price-ack flag
---------------------------------
Add support for reading and writing this flag. We still won't be using it - that's in a followup CL, but we support it.
support changing the URL
------------------------
Add changing the URL in the UI, under the "whois" section
Will replace the Ack endpoint with this (and remove that endpoint) in a followup CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192154078
Currently the example whitelist IP is 1.1.1.1/24, which is illegal. Changed to
1.1.1.0/24, which is legal
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191900036
The registrar security console failed because it assumed the email is a
required field for the registrar, but it isn't (at least - create_registrar
doesn't require an email, and update_registrar lets you remove the email).
Fixed by allowing it to *remain* unset if it was unset originally, but if it was set - it's required.
There are more fixes needed, but they aren't related to the email, so they will wait for the next CL
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=191623034
Caching turns out to be an anti-pattern for the console. If we use it, changes from the user just get obliterated by the older, cached version the next time the console refreshes (and it happens to refresh after every update). Caching is also not very useful here, as the amount of database access driven by the console is very small.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190650931
Implement a checkbox in the "Resources" tab to allow registrars to toggle
their "premium price ack required" flag.
Tested:
Verfied the console functionality by hand. I've started work on an
automated test, but we can't actually test those from blaze and the
kokoro tests are way too time-consuming to be practical for development, so
we're going to have to either find a way to run those locally outside of
the normal process or make do without a test.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190212177
Soy is about to require trusted_resource_url in <iframe src="">. This change prepares for that by blessing existing string URLs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=190054283
Soy is about to require trusted_resource_url in <link href="">. This change prepares for that by blessing existing string URLs.
More information: []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=189795596
Soy has historically tolerated map accesses on weakly-typed variables. That is, if a template declared a param $p and then did $p['some_key'] in the template body, Soy would treat $p as a map even if it wasn't statically declared as one.
This situation is changing with [] There are now two map types, `map` and `legacy_object_map`. We are trying to migrate every template in [] from `legacy_object_map` to `map`, leaving Soy with one (improved) map type. Because the two map types generate different JS code, Soy can no longer allow map accesses on weakly-typed variables. (If $p['some_key'] occurs in a template and the type of $p is unknown, Soy would not know what code to generate.)
Every parameter whose static type is unknown (`?`) but which is inferred to be a `legacy_object_map` needs to be migrated to a `map`. We are developing tools for this in [] However, as a first step, we need to migrate the subset of these params that use the legacy SoyDoc syntax to use header param syntax with a static type of `?`. (All params declared in SoyDoc are typed as unknown, and it is a compilation error to mix SoyDoc and header param syntax in the same template, so any template that declares a SoyDoc param that is inferred to be a map needs to migrate to header param syntax.)
This CL was prepared by using the tools in [] to create a list of templates declaring SoyDoc params inferred to be legacy_object_maps. This list was then fed to the existing //third_party/java_src/soy/java/com/google/template/soy/tools:ParamMigrator tool. Since this tool migrates whole files instead of individual templates, the resulting CL is a superset of the migration that is actually required. However, since the SoyDoc param syntax has been deprecated for years, and since there is little risk in migrating from one param style to another, I decided to land the superset.
This migration falls under the LSC described at https://docs.google.com/document/d/1dAl-rDMp3oL0Zh_iSTaiHICwtcbLbVIy1FQ0wXSAaHs.
Tested:
TAP --sample for global presubmit queue
[] passed FOSS tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188879980
SoySyntaxException is an abstract exception type and is never even declared to be thrown (all declarations about this changed about 2 years ago). So places catching it should either change to catch SoyCompilationException, or do nothing and let it propagate.
Tested:
TAP sample presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=185050724
See []for more information.
Created with the tools in []
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184727400
Rosie CL for []/third_party (local approval/rejection).
[]
b/71392935
Tested:
TAP --sample for global presubmit queue
[]
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184412611
This also updates to a newer version of Closure Rules and fixes a protobuf dep
compile issue.
Full description of the change:
Soy supports 2 kinds of loops:
* foreach- for iterating over items in a collection, e.g.
{foreach $item in $list}...{/foreach}
* for - for indexed iteration, e.g. {for $i in range(0, 10)}...{/for}
The reason Soy has two different loops is an accident of history; Soy didn’t use
to have a proper grammar for expressions and so the alternate ‘for...range’
syntax was added to make it possible to write indexed loops. As the grammar has
improved having the two syntaxes is no longer necessary and so we are
eliminating one of them.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182843207
This fixes up the following problems:
1. Using string concatenation instead of the formatting variant methods.
2. Logging or swallowing exception messages without logging the exception
itself (this swallows the stack trace).
3. Unnecessary logging on re-thrown exceptions.
4. Unnecessary use of formatting variant methods when not necessary.
5. Complicated logging statements involving significant processing not being
wrapped inside of a logging level check.
6. Redundant logging both of an exception itself and its message (this is
unnecessary duplication).
7. Use of the base Logger class instead of our FormattingLogger class.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=182419837
*** Reason for rollback ***
Breaks the FOSS build.
We'll reincorporate this change once Closure Rules is properly updated to accommodate it.
*** Original change description ***
Change all foreach loops in Soy templates to use the for loop syntax
Soy supports 2 kinds of loops:
foreach- for iterating over items in a collection e.g. {foreach $item in $list}...{/foreach}
for - for indexed iteration e.g. {for $i in range(0, 10)}...{/for}
The reason Soy has 2 different loops is an accident of history, Soy didn’t use to have a proper grammar for expressions and so the alternate ‘for...range’ syntax was added to make it possible to write indexed loops. As the gramma...
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=180961695
