There is no "google_project" resource managed by terraform, so we are not worried about the new terraform binary destroying/re-creating GAE resources.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=217206226
SessionUtil is renames AuthenticatedRegistrarAccessor, as it's used to access a registrar for an authenticated user.
It will now be injected with the AuthResult instead of receiving it in every function call, since there's only one "legal" AuthResult to use.
The AccessType names are changed from READ_ONLY/READ_WRITE to READ/UPDATE, as it was confusing that a user could have both READ_ONLY AND READ_WRITE access to the same registrar.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216958306
This uses a Dagger-provided map of Keyring implementations, with two currently available,
"KMS" and "Dummy". The active keyring is configured in the YAML file, so we no longer
require MOE directives to choose which one to use for internal/external builds.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216898058
The following issues are addressed:
- XML sanitizer should preserve input encoding. Gradle loses any that is not UTF-8. Bazel loses any that is not ASCII.
- Verify that XML sanitizer works with non-UTF8 encoding
- GpgSystemCommandRule breaks when $TMPDIR env variable is not set
- TestDataHelper throws exception when loading resources if resources are plain files on default file system as opposed to being in a jar file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=216537258
These are simply too costly in their current form now that we are handling double-digit QPS, so at a minimum we'd want to refactor these for batched exports using a background thread (like how Stackdriver metrics work). However, upon further review, that work isn't worth doing if this BigQuery table isn't actually being used for anything, and it seems that we aren't using it anymore given that ICANN transaction reporting no longer requires it.
So the simplest thing to do is simply to get rid of this entirely, and just use a combination of Stackdriver metrics and App Engine logs. The eppMetrics BigQuery table is ~1.2 billion rows and takes up 223 GB, so that's not an insignificant GCP billings saving if we can delete it.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215905466
Note that the check flow does not yet handle any kind of allocation token
handling at all. Step 2 will be to add allocation token handling there, so a
RESERVED_FOR_ANCHOR_TENANT or RESERVED_FOR_SPECIFIC_USE domain will show as
available instead of reserved if the right token is specified using the
extension. Then once that's done, we can use that information to adjust the
price accordingly as well.
Right now the behavior with a domain check is that reserved domains always show
as reserved, even if they're anchor tenants.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215599350
We want to be able to view / test / debug how the registrar console looks for our clients.
However, we don't want to accidentally change the data for registrars, especially in a "non-accountable" way (where we later don't know who did that change)
So we do 2 things here:
- Add a "mode" (read-only and read-write) to the getRegistrarForUser function. We set it according to what we want to do with the registrar. Currently, read-write is only requested for the "update" RegistrarSetting action. Admins will have read-only access to all registrars, but read-write access only to the "admin registrar" (or whatever registrar they are contacts for).
- Support an undocumented "clientId=XXX" query param that replaces the "guessClientIdForUser" function in the original page load. We can then set it when we want to view a different account.
We also change the navigation links on the HTML page to preserve the query.
-------------------------
This might be used also for a better user experience for our clients, especially those with multiple "clientId"s (some registrar entities have multiple "registrar" objects)
Currently, they have to have a separate user for each clientId, and only have one user allowed which has both read and write permissions.
Using this change, we can give them the possibility to add users on their own, some with read-only access (to view billing information without being able to change anything), and use a single user for all their clientIds.
-------------------------
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215480610
All credentials provided by these modules have been
replaced by those in the config/CredentialsModule,
with a new set of Qualifiers. With Dagger 2, a successful
build means that the removal is safe.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215258792
RegistrarSettingsAction is a JSON in / JSON out endpoint, meaning the reply is consumed as JSON.
The current state is that if an error occurs, there are two possible replies:
- a JSON error reply is sent out, or
- a 402 HTML reply is sent out with the exception.getMessage()
The difference is only - do we actively catch the exception to translate it to JSON or not.
This fix catches ALL exceptions and translates them to JSON format. Note that there's no security change by giving the getMessage in the JSON reply since we were returning that anyway (in the HTML).
In addition - changed the "gaeUserId" to "user.getEmail" as the identifier, since it's clearer to the users who see that error - and I do want to transition to a more "email identifier" way of checking access (since that's what users put in the registrar contact info)
This too isn't leaking new information because
- the initial HTML page load already gives the user's email, and
- the logs already log the user's email for every request
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215213807
com_google_truth has a dependency on diffutils that we're not exposing. This
becomes problematic in cases where certain tests fail and the equality check
can't show the difference. If this happens, instead of the original failure
all we see is a failure to load diffutils.
Note that com_google_truth appears to have some other dependencies that we are
also not exposing, but that so far these have not been problematic.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214774587
We now require the URL to be filled out by all registrars and so there is no
default needed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214774054
Looks like a copy-paste error from the reserved list export disclaimer.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214773560
Replaced the plethora of inter winding access functions and inputs in SessionUtils with just 2 functions, that both accept the same type for the user (AuthResult):
guessRegistrarForUser: given an AuthResult, finds a registrar that they have access to. If none is found - a ForbiddenException is thrown.
getRegistrarForUser[Cached]: (maybe should be called getRegistrarOnBehalfOfUser?) given an AuthResult and a clientId, loads and returns the registrar ONLY IF the user has access to it. Otherwise throws a ForbiddenException.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214630657
This is an intermediate CL, part of the Registrar Console cleanup.
TL;DR:
- the current state: resource.js points to a resource TYPE on the server (only registrars can be resources right now), but the specific resource is selected based on the user (we select the "first resource of this type that the user has access to)
- new state: resource.js points to a SPECIFIC resource (TYPE + ID).
In this CL the server still chooses the resource like before (first one that user has access to) but we make sure the returned resource is the same one we requested.
In a subsequent CL we will use the requested ID to load the resource, and then make sure the user has access to that resource.
---------------------------
When loading the RegistrarConsole HTML page, the server determines which clientId belongs to the user ("guesses" it by looking for the first registrar that has this user as contact). It sends the relevant clientId back with the page load.
However, this information isn't currently used in the JS requests to read / update the registrar. Instead, currently the client ID is guessed again for each JS access to the server. It is also saved again in the client's "session" cookie.
As a result, it is theoretically possible to have the JS access a different clientID than the original page load (not likely, since it requires a single user registered for multiple registrars AND that the contacts change for the original registrar).
So our goal is to only have a single clientID "value" instead of the 3 we currently have for JS requests (the one from the initial page load, the one saved in the session cookie, the one guessed on the JS request)
As a first step, we send over the "initial page load" clientId on every JS request, and make sure the "session + guessed" value is equal to that one. Later we will remove the "session+guessed" values from the RegistrarSettings, using the "initial page load" clientID instead.
In addition to the "nicer code" implications, having the clientID from the initial page load always used means it'll be easy to have a clientID selection option for users who have access to multiple clientIDs (such as admins)
SECURITY NOTE:the choice of clientID has no security implication since we make sure the user has access to the clientID no matter how we actually choose the clientID on every single server request.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214459506
We need this to efficiently query which tokens have and have not been redeemed.
I'm also marking it as nullable for consistency with the domain name field (it's
already null for unredeemed tokens).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214366534
*** Reason for rollback ***
Automated tools sometimes don't have default credentials, and can't set them up. We should redo this CL once we figure out the credential thing.
*** Original change description ***
Add metrics for the command used in the registry CLI tool
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
***
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214048616
We have several "consoles" ("views" in the registrar console). They all affect the same resource - the registrar itself.
Currently, each view creates its own "RESTful resource", even though it's the same resource for all of them, meaning we have copied code and copied "URL endpoint" across multiple files.
I assume this was made so that IF one day we have a "view" to another resource, we can easily add it. But we currently don't have any such view, nor plans to add any.
So according to the YAGNI paradigm, it's better to move the resource creation outside of the console.
Also, IF we do add a view to a different resource - it'll still be more readable to have a map from the "view" to the "resource URL endpoint" alongside the existing map from the "view" to the "console"...
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213992967
This fixes the Kokoro tests broken by []
Unfortunately this means that the return type changes from a Map to an Object as well,
so all of the .get() lines need to turn into []. Yay weakly typed languages. And
apparently in some error conditions this method could already return the Object type
anyway, even if you passed in a Map, so it's just poorly designed.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213808477
1. Moved code for the GCP proxy to where the [] proxy code used to live.
3. Corrected reference to the GCP proxy location.
4. Misc changes to make ErrorProne and various tools happy.
+diekmann to LGTM terraform whitelist change.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213630560
This change completes the switch to @DefaultCredential for
all use cases in GAE.
Impacted modules:
- IcannReporting
- CreateCdnsTld command
- LoadSnapshot command.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213511730
This properly fixes the issue that was suppressed in [] and
just started breaking today thanks to []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213510500
We never really used it and it'll be obsolete come Registry 3.0 anyway.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213274520
This is only useful when we used the [] proxy because the GFE requires SNI during handshake in order to request the client certificate. The GCP proxy does not need this (it always requests the client certificate). We do not need to check for its existence.
Also removed the checking of internal headers for ssl cert hash used only by the [] proxy.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213059027
This turns on spec11 reporting in production by adding it to the cron.xml, generating the report and sending an e-mail with a list of all problematic registrations to the associated registrar on the 2nd of each month at 15:00Z (11am EST)
This also tweaks the e-mail template a bit according to suggestions from Bruno.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213031440
The original thought was that the actions you can do on a resource is:
- create it
- read it
- update it
(I guess you should have "delete" as well, but that isn't currently there)
Although we use "read" and "update", we never use "create". So having it goes against the YAGNI principle :)
Also, it had a bug: when sending a "create", the opt_newId in send_() would
permanentily change the uri of the request, causing any subsequent request to
go to the wrong endpoint.
By removing the "create" we can simplify the rest of the code (the send_() function).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213029499
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
Puts the metric in <project>/tools/commands_called
It counts the use of the tool, with the following labels:
- environment
- tool (nomulus/gtech)
- command called (class name)
- success true/false
- from the shell true/false
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212879670
Defines cron job in crash, sandbox and production environments.
Job already exists in alpha.
Job is not added to qa environment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212878436
This is obsoleted by the upcoming Registry 3.0 migration, after which we will be
using neither the App Engine Mapreduce library nor Cloud Datastore.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212864845
Updated Reporting (Beam pipeline), Registrar sync to sheets, and Cloud Dns.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212811185
According to RFC 2046, the body of the multipart contains:
multipart-body := [preamble CRLF]
dash-boundary transport-padding CRLF
body-part *encapsulation
close-delimiter transport-padding
[CRLF epilogue]
The preemble and epilogue are optional, and ignored. However, it's not 100%
explicit whether the CRLFs after the preamble and before the epilogue are
required. The one after the preemble is often not given if there's no preemble,
so it's conceivable that you don't *have* to give the CRLF before the epilogue
if there's no epilogue (it's also enclosed in the [], making it part of the
"optional")
However, it seems that when the TMDB "migrated to the cloud" (as they
describe it) on Aug. 13 2018, they started requiring that CRLF.
TESTED=connected to a TMDB-whitelisted server, used CURL to manually create the
message as we currently send it (without the final CRLF) with junk data and got
the error from the bug. Then sent the exact same message with the additional
CRLF, and got a different error that directly relates to the content of the
junk data.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212637246
Scope changes in delegated credentials require coordinated external changes,
therefore should be separate from those used in the application default
credential.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212488389
Updated the registar contact group management, which is the only
use case for this credential.
Also updated GSuite domain delegated admin access config in admin
dashboard for both sandbox (used by alpha and sandbox) and prod.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212320157
Unlike anchor tenants, these domains can be registered for any number of years,
but only during GA, as third parties cannot register domains pre-GA except
through the anchor tenant program.
Since this is new functionality, unlike creation of anchor tenants, there is no
fallback provided to send codes through the domain authcode; they must be sent
using the allocation token extension.
And note that, like with anchor tenants, providing the domain-specific
allocation token overrides any other reserved types that might apply to that
domain.
No changes are necessary to the domain application create flow because of the
above restriction to GA.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212310701
