The artifacts for each service will be packaged inside a tar file that, when untared, is ready for Spinnaker to deploy to GAE.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226516223
Recent changes cause a bunch of new tests to have runtime conflicts.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226499557
Most dependencies on the Old SDK were switched in [] This is just catching up on some OAuth dependencies that remained and some remaining uses of Old build rules.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226337284
see b/34094769 for context
The webdriver tests don't choose a correct font when we specify "monospace". As a result, we don't render correctly pages that use monospace.
Here we instead explicitly reference a monospace font we know exists in the webdriver: Courier New.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226233831
A few nullable parameters were not marked as nullable, which causes exceptions
to be thrown in debug mode.
This had no effect in the deployed web server, because these assert sanity
checks aren't performed - but on our local test server this failed.
Note that all these fields are checked for "nullness" in the code itself. It's
just an oversight in the declaration.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226187227
SendEmailUtils is a general utility of the web console, and not specifically "only"
to the Registrar console.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226187094
This will only affect the tools service, the primary use case being (1) I go to
create a domain through nomulus tool, realize it's premium, (2) update the
premium list to not include that domain, (3) kill the tools service instance to
wipe out the cached premium value, then (4) create the domain at standard. This
commit eliminates step 3.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226180160
HTML headers can only be sent via JS, we need this change to allow
secure POST form submission.
The form itself will have a hidden "input" tag with the XSRF token in it. This
is how other framework do it as well - see
https://en.wikipedia.org/wiki/Cross-site_request_forgery#Synchronizer_token_pattern
This is in preparation for the OT&E setup page, which will be a simple form
with a "submit" button, so using JS for it is overkill.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226178070
Currently there's a margin on the top, making the textarea be unaligned with
the text naming it. This is annoying on the eye, and will be more annoying in
the OT&E cl that will be added soon.
- So why not just do this change in that CL?
- Because the changes in the Screenshot tests here are irrelevant to that CL
and I found make it harder to actually review the actual screenshots we're
adding there.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226057985
This allows us to run nomulus tool programmatically on environments that do not
allow the 3-legged OAuth authentication flow.
The provided JSON file corresponds to a service account, which must have
GAE admin permission and whose client ID must be whitelisted in the config
file.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226008337
Added a property to show all test output, useful for debugging.
Usage: ./gradlew -PshowAllOutput=true build
Tested:
Verified that the property does the right thing for true and false values
and verified that the build doesn't show test output if it's omitted.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225854375
This is in preparation for having a web-console endpoint to create OTE.
In addition - we streamline the code:
- we remove support for different premium lists
- we remove support for different DNS writers - we never want a "real" DnsWriter for OTE
- we remove support of --eap_only, because we don't need it anymore
- We use a single password for all the Registrars
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225841694
In addition to just making good sense to not have support group for some
environments (local? unittest? crash?) - connecting with G Suit requires
additional permissions that are harder to find.
Specifically, it requires the Json Credentials that just aren't set in the
Dummy Keyring used on some environments.
So we make sure to not even *try* to create the credentials if the support
email isn't set
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225589255
The private repo structure is re-done to mirror that of the public repo to facilitate easy merging.
Also removed steps to tag the private repos. This will likely cause a race condition if both the nomulus and the proxy cloud build are triggered by the same tag. They will both try to tag the private repo with the same tag.
The tagging of the private repo should happen simultaneously with the public repo tagging, and in an out-of-band process as far as the build process is concerned. The build process should not have side effect on its source.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225544063
This is safer and addresses a common source of confusion in the codebase because it's always explicit that the resource returned may not be present, whether because it's soft-deleted when projected to the given time or because it never existed in the first place.
In production code, the presence of the returned value is always checked. In test code, its presence is assumed using .get() where that is expected and convenient, as it not being present will throw an NPE that will cause the test to fail anyway.
Note that the roughly equivalent reloadResourceByForeignKey(), which is widely used in test code, is not having this same treatment applied to it. That is out of the scope of this CL, and has much smaller returns anyway because it's only used in tests (where the unexpected absence of a given resource would just cause the test to fail).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225424002
Modularize the code for DNS count reporting to allow it to be customized for
more flexible systems.
Tested:
Uploaded to alpha with hacks to allow admin initiating and logging from the
DnsCountQueryCoordinatorModule, verified that the provider function is invoked and
that the action runs successfully.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225225587
Nomulus tool will be using OAuth to communicate to GAE endpoints exclusively starting with next week's release. As far as I can tell that is the only part of the system that uses Google-internal auth). We can therefore remove the it after next week's release.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225221102
If the user runs "nomulus -e [ENV] login --remote", an URL will be provided, the user then can visit the URL on any machine (not necessary where the command is run) and copy&paste back the authorization code to complete authorization.
This makes it easy to login on machines where local browsers are not easily accessible.
Also upgraded nebula lint version to 10.3.5.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225198700
Tested:
TAP --sample for global presubmit queue
[] dr-foss-test passes
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225189790
Next up (and a much larger commit) will be giving loadByForeignKey() the same
treatment.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225182377
Note that it's possible to set a config option to disable this functionality
on a per-environment basis (we're disabling it for sandbox), but in general
SSL certificate hashes should be required for increased security.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225053496
This will allow us to perform the OT&E history verification
in the model/ package as well so that it can be used both
by both the UI and the command line tool.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=225007167
This is used to reduce the expiration time of domain(s) by some number of years
(if enough length remains in the registration term to do so). This does not back
out the previously saved BillingEvent entities as they may have already been
sent out and invoiced, so any related refunds must be handled out of band.
In addition to reducing the registration expiration time on the domain itself,
this command writes out a new history entry, one-time poll message informing the
registrar of this change, auto-renew billing event and poll message, and
updates/ends the old auto-renew billing event and poll message.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224999285
Add server end points to backup Datastore using managed-export mechanism.
A cron job is defined in Alpha to run daily exports using this implementation.
Existing backup is left running. The new backups are saved to a new set of
locations:
- GCS bucket: gs://PROJECT-ID-datastore-backups
- Big Query data set: datastore_backups
- Big Query latest back up view name: latest_datastore_backup
Also, the names of Bigquery tables now use the export timestamp
assigned by Datastore. E.g., 2018_12_05T23_56_18_50532_ContactResource,
After the new import mechanism is implemented and the back-restore flow is
tested, we will stop the existing backup runs and deploy the new
implementation to all environments.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224932957
Eventually the Publish action will control daily/monthly sending and provide
the correct threats to email. The goal of this PR is to entirely separate
the "sending email" functionality from the "parsing threat matches"
functionality.
The PublishAction will figure out if the monthly emails should be sent out,
then will ask the Spec11ThreatMatchesParser for the monthly threats (if
appropriate) and the new threat matches for today. It will then pass those
matches and the appropriate email subject+body to the email utils class,
whose only job is to format and send the emails.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224869643
We have a retrier wanting to call a function, retrying on
IllegalStateExceptions (thrown by checkState()).
However, one of the checkStates is called inside a Concurrent.transform, so
when the checkState fails, the resulting IllegalStateException is wrapped in an
UncheckedExecutionException and isn't caught by the retrier.
We unwrap the IllegalStateException to make sure it's caught.
Q: Why not just catch UncheckedExecution exception?
A: Because it might wrap a different exception which we don't want to retry on.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224862573
The lastEppUpdateTime should be updated asynchronously in the situations below:
- Implicit transfer success after 5 day pending transfer period
- Implicit end of any grace period
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224831791
It'd be nice if we can separate out the tool to its own package and reduce the transitive dependencies that it pulls in. However since the entire core project is a dependency of the tool, it doesn't make any difference as we'd be pulling in core and all its transitive dependencies as well.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224821931
When a connection to the proxy using the PROXY protocol (https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) comes from an IP address that the external load balancer does not recognize, make the source IP 0.0.0.0. This way an appropriate WHOIS quota can be configured for this kind of connections.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224583547
This is for consistency, mostly the LocalDate fields added in []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224525451
Add a sendSpec11Email parameter that allows us to only send the email on
one run per month. Next, we will compute the diffs between the daily runs
and send daily emails with those diffs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224404653
Removed duplicate build files for each service, and also added two tasks that runs deployment/staging for all services.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224359198
We are moving away from using Application Default Credentials generated by "gcloud auth application-default login" in our code base and consolidate on using self-managed credentials provided from AuthModule.
One of the remaining dependencies on the ADCs is from beam pipeline deployment commands, which by default use the ADCs to talk to GCS and upload the jar files and templates. In this CL, we explicitly provide the locally created credential to the Options used in deployments.
Also moved all credential qualifiers to CredentialModule, and removed @AppEngineAdminApiCredential, which is no longer used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224199812
