Commit graph

219 commits

Author SHA1 Message Date
Shicong Huang
168aca3c41 Upgrade Postgres realted packages to latest version (#861) 2020-11-06 10:14:44 -05:00
Weimin Yu
9ddde4799c Upgrade error-prone to 3.3.4 (#848)
* Upgrade error-prone to 3.3.4

This would fix the failure with openjdk 11.0.9 in
3.3.3.

Fixed new antipatterns raised by the new version:
- Replaced unnecessary lambdas with methods.
- Switched wait/sleep calls to equivalent methods using java.time types
- Types inheriting Object.toString() should not be assigned to string
parameter in logging statements.
2020-10-23 11:17:57 -04:00
Shicong Huang
7f87938942 Add a command to generate ER diagram for SQL schema (#839)
* Add a command to generate ER diagram for SQL schema

* Add graphviz as runtime dependency

* Update ER diagrams for #838
2020-10-15 17:31:43 -04:00
Shicong Huang
f18c33dc1f Add SchemaCrawler as dependency (#833) 2020-10-09 15:02:11 -04:00
Weimin Yu
bc89963b3d Minor python changes (#832)
* Minor python changes

Use dataclasses instead of attrs. The former is part of the standard lib
while the latter may need to be installed separately.

Also added python3 to the list of prerequisites.
2020-10-09 14:50:21 -04:00
sarahcaseybot
35ebe371ba Add a CertificateChecker class (#793)
* CertificateChecker with checks for expiration and key length

* Add validity length check

* Get rid of hard-coded constants and DSA checks

* add files that for some reason weren't included in last commit

* Rename violations and other fixes

* Add displayMessage to CertificateViolation enum

* Switch violations from an enum to a class

* small changes

* Get rid of ECDSA checks

* add checks for old validity length

* Change error message for validity length
2020-10-06 15:47:42 -04:00
Lai Jiang
8d0c0e7fb2 Add log4j-core as a runtime dependency (#817)
Without it we kept getting the following warning:

ERROR StatusLogger Log4j2 could not find a logging implementation. Please add log4j-core to the classpath. Using SimpleLogger to log to the console...
2020-09-24 19:59:39 -04:00
Michael Muller
d0144290bb Update user-facing documentation (#662)
* Update user-facing documentation

Give our docs a complete overhaul to account for changes in the system,
notably the requirement to configure postgresql.

* Fix dangling sentence.

* Merge branch 'master' into admin-docs
2020-08-17 14:46:32 -04:00
Lai Jiang
6a5dbf8817 Update dependencies to get rid of some illegal reflection warnings (#768)
For some inexplicable reasons I have to move the javax.mail package one
spot up to avoid its classes being shadowed by those provided in the
appengine package...
2020-08-14 16:53:22 -04:00
Weimin Yu
206adffb21 Update BEAM SDK to work with Java 11 (#762)
* Update BEAM SDK to work with Java 11

Upgraded BEAM dependencies to 2.23.0.

Updated Spec11 and invoice pipelines:
- Added the required region parameter.
- Removed the workaround code for staging.

Verified that staging is successful in alpha:
./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_spec11_pipeline --project domain-registry-alpha'

and

./nom_build :core:registryTool --args='-e alpha --sql_access_info "gs://..." deploy_invoicing_pipeline'
2020-08-10 20:56:08 -04:00
Lai Jiang
f6749ad663 Enable Java 11 features (#737)
* Enable Java 11 features

As of this commit Java 11 must be used to build. The generated bytecode
is still at Java 8 due to App Engine task queue limit.

Also fixed a bug where the included google-java-format jar file is not
used, requiring the user to install it separately.

See: https://cloud.google.com/appengine/docs/standard/java/taskqueue/push
2020-08-03 16:08:29 -04:00
Ben McIlwain
d290b82566 Upgrade prober & proxy packages from JUnit 4 to JUnit 5 (#734)
* Upgrade prober & proxy packages from JUnit 4 to JUnit 5
2020-08-01 11:34:58 -04:00
Lai Jiang
d6cc43277e Some more changes to make it possible to use Java 11 to compile (#732)
Add the class paths of the source files generated by annotation processors to
the javadoc task's class path so that it doesn't complain about missing
Dagger classes.

Also remove empty <p> tags in all generated source files, because jaxb
genrerates files in multiple locations.

Lastly, for unkown reasons when the source level is set to > 8, the core
subproject throws a warning about a Gradle internal annotator processor
that only supports up to Java 8 and cause the Java compliation to fail because
we set -Werror on all java compliation tasks. I don't think there is a
strong reason that we set -Werror anyway, so this commit removes it.
2020-07-31 17:21:28 -04:00
Ben McIlwain
0d857f8291 Get rid of all remaining JUnit 4 usages except in prober & proxy (#731)
* Get rid of all remaining JUnit 4 usages except in prober & proxy subprojects

Caveat: Test suites aren't yet implemented in JUnit 5 so we still use the ones
from JUnit 5 in the core subproject.

* Fix some build errors
2020-07-30 20:29:00 -04:00
Lai Jiang
fba8af0485 Migrate the documentation package to Java 11 (#729)
* Migrate the documentation package to Java 11

The old Doclet API is deprected and removed in Java 12. This commit
changes the documentation package to use the new recommended API.
However it is not a drop-in replacement and there are non-idiomatic
usages all over the place. I think it is eaiser to keep the current code
logic and kind of shoehorn in the new API than starting afresh as the
return on investment of a do-over is not great.

Also note that the docs package is disabled as of this commit because we
are still using Java 8 to compile which lacks the new API. Once we
switch our toolchains to Java 11 (but still compiling Java 8 bytecode)
we can re-enable this package.

TESTED=ran `./gradlew :docs:test` locally with the documentation package
enabled.
2020-07-30 17:12:33 -04:00
Lai Jiang
04f032e226 Move the documentation package to its own subproject (#722)
This makes it easier to later migrate the package to Java 11. If we move
and migrate in a single PR, because of the portion of the contents that
s changed, git will have trouble recognizing that some files are
renamed *and* modified and treat them as distinct files, making code
review difficult.
2020-07-29 13:41:02 -04:00
Ben McIlwain
23310bd688 Rename whitelist -> allow list (#635)
* Rename whitelist -> allow list

* Merge branch 'master' into allowlist-denylist
2020-06-18 18:36:05 -04:00
0xflotus
223832a402 (docs): fixed small errors (#572)
* Update first-steps-tutorial.md

* Update proxy-setup.md (#1)

* Update registrar-faq.md (#2)

* Update first-steps-tutorial.md
2020-05-11 10:01:47 -04:00
Ben McIlwain
9573235ca7 Always validate domain name on allocation token (#498)
* Always validate domain name on allocation token

This is in response to a client-reported error, where they accidentally sent the
wrong domain name on a domain create that included an allocation token. What
should have happened (and that now happens as of this commit) is an error being
thrown that the allocation token does not match the domain name being created.
What happened instead was that, since the incorrectly submitted domain name was
not reserved, the create succeeded (as it would for all creates of unreserved
domains in GA) and the allocation token was redeemed, which is not what you'd
expect.

* Fix tests to reflect changed check behavior
2020-02-27 16:48:37 -05:00
gbrodman
2a205f3340 Check for an empty/null allocation token (#489)
* Check for an empty/null allocation token

In addition, we should be returning an authorization exception for an
invalid token, not a parameter syntax exception. See https://tools.ietf.org/html/draft-ietf-regext-allocation-token-04#section-2.1

* Add comment
2020-02-20 15:50:10 -05:00
Lai Jiang
fbedbbcf60 Remove AUTH_INTERNAL_ONLY (#393)
Endpoints annotated with AUTH_INTERNAL_ONLY used to be accessible
manually with an internal RPC tool that adds App Engine specific HTTP
headers to a request to make it look like it comes from App Engine
(hence internal). This tool is used by admins to hit such endpoints
during debugging, making them effectively AUTH_INTERNAL_OR_ADMIN.

This RPC tool has never been made available outside Google so the open
source admins do not have such ability. A recent change in the RPC tool
made this hack stop working internally as well. This PR replaces all
all occurances of AUTH_INTERNAL_ONLY with AUTH_INTERNAL_OR_ADMIN and
brings the open source build into feature parity with the internal
version.

Also fixed a few issues the router tests.
2019-12-04 12:52:18 -05:00
Ben McIlwain
c34b68331f Update premium and reserved list management docs (#380)
* Update premium and reserved list management docs

They were a little bit out of date.
2019-11-21 16:44:45 -05:00
gbrodman
b3480fb2eb Throw an EPP exception when using discount on premium domains (#351)
We should communicate to the users why this command failed, that they
are not allowed to use discounted allocation tokens on premium domains.
Currently it still fails, but we don't yet tell them why.
2019-11-07 15:30:23 -05:00
Lai Jiang
684bb119db Move terraform and kubernetes folder to be under proxy (#127)
* Move terraform and kubernetes folder to be under proxy

There is no reason for them to be under proxy/src/... any more now that
we have a Gradle-idiomatic folder structure.
2019-06-20 14:28:32 -04:00
Gus Brodman
38cfc9f693 Refactor to be more in line with a standard Gradle project structure 2019-06-13 09:41:11 -04:00
gbrodman
353fb0a35e De-duplicate flow documentation error reasons on a per-code basis
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250894847
2019-06-06 10:51:43 -04:00
gbrodman
ba23e3efaa Split ResourceAlreadyExistsException based on if this client owns the resource
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=250728711
2019-06-06 10:42:38 -04:00
gbrodman
74b5b407fa Validate on-load that an AllocationToken can be used
Check the timing (that is, whether or not we're in a promotion), the allowed registrar client IDs, and the allowed TLDs.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246824080
2019-05-06 16:54:55 -04:00
jianglai
b28e444d7d Update to bazel 0.25.0
Also updated the docker rules to be compatible with the new version of bazel,
and updated the proxy base image to the latest Java 8 version.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=246509816
2019-05-06 16:36:04 -04:00
weiminyu
7ecb153330 Fix incorrect public tutorial on configuration
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=245665006
2019-05-06 16:09:23 -04:00
mcilwain
ff3aeff0ed Remove per-domain nameserver restrictions in reserved lists
This also removes the related setting on the Registry entity. We never used
either of these, and keeping them around in our codebase just adds complexity
for no purpose. We already achieve the goals of this feature by restricting
nameservers on entire TLDs and by registry-locking important domain names.

This is also two fewer things we'll have to worry about carrying over to the new
schema in Registry 3.0.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=243816241
2019-04-16 14:09:27 -04:00
gbrodman
2a18e705a2 Add generic XML syntax testing to a flow test
Adding it to one test is sufficient because we use the same loading logic across all flows.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239506003
2019-03-21 14:58:31 -04:00
gbrodman
6e01d400cd Test UnknownCurrencyEppException in the flow tests
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239279335
2019-03-20 14:25:28 -04:00
jianglai
50e88acddb Update to bazel 0.23.1
Also updated closure rules to GitHub master, as the current release version
(0.80.0) has compatibility issues with bazel 0.23.1.

See: https://github.com/bazelbuild/rules_closure/issues/333

Note that on Debian Testing (Buster) there's currently an issue with OpenSSL that can be temporarily fixed by setting the environment variable OPENSSL_CONF to /etc/ssl.

See: https://fransdejonge.com/2018/12/run-enpass-on-debian-buster-testing/

Lastly, migrated from the --local_resources flag (to be deprecated) to --local_ram_resources and --local_cpu_resources.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=237527805
2019-03-08 18:41:16 -05:00
guyben
847795d58d Remove the web console EPP endpoint
This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.

We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.

Also, this is a security risk, as it allowed to do "billable actions" (creating a new domain for example) with the only authentication being access to the registrar's G Suite account.

This bypassed the certificate, IP whitelist, and EPP password, which is bad.

PUBLIC:
Remove the web console EPP endpoint

This removes the "create Domain/Host/Contact" forms that were supposed to be used instead of regular EPPs for CC-TLD that wanted to support it.

We're removing it because we don't use it and want to reduce unneeded code for the registry 3.0 migration.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=236244195
2019-03-05 14:20:42 -05:00
mcilwain
6a6c7e3b09 Explain SERVER_HOLD part of NAME_COLLISION reservations
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=232713573
2019-02-06 18:01:37 -05:00
mcilwain
5dedc1e889 Delete everything related to RDE import
This code was never finished or fully working anyway.  It would require
substantial reworking for the Registry 3.0 migration because it's closely tied
to the Datastore model and App Engine MapReduce framework, both of which will be
going away.  We can bring back some of these deleted test files as necessary
if/when we rewrite RDE import for the new schema.

On the plus side, in a relational database, RDE import will be much simpler.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=231265578
2019-01-28 16:16:36 -05:00
mmuller
87ab149049 Externalize premium/reserved list tests
Remove the GoogleInternal designator on the premium/reserved list tests, move
the lists themselves under the config/files directory, and fix all of the
surrounding infrastructure.

The lists do not get exported.  Only the "example.txt" list does, allowing the
test to function on the open source build.

TESTED:
  Verified that the gradle build works and that only the example.txt file is exported.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229940659
2019-01-18 15:34:06 -05:00
jianglai
52212748ca Update documentation on nomulus tool authentication
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229780750
2019-01-17 19:22:25 -05:00
mcilwain
580302898d Delete end-date sunrise, landrush, and sunrush phases
This also deletes the associated commands and domain application specific
entities.

We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.

Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
2019-01-10 16:23:35 -05:00
weiminyu
25f3d8544e Update Nomulus open source install guide
appcfg fails if version is not specified.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=227909394
2019-01-08 10:56:53 -05:00
jianglai
9e155f14c0 Add support to bazel 0.21.0
Also fixes the build.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=226493872
2018-12-21 15:55:08 -05:00
jianglai
e4733953d3 Update bazel version to 0.20.0
Also updated distroless base image to include the latest vulnerability fixes.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223975505
2018-12-05 15:56:01 -05:00
mcilwain
11da64bee2 Deny non-active registrars access to non-free EPP commands
Registrars that are PENDING or SUSPENDED should not have access to
EPP commands that cost money, as in either case it's not likely we'd
actually be able to get payment from said registrar. For this reason
we already prevented access to the domain create flow for non-active
registrars. This commit extends that to other commands that cost
money, including renewals, restores, and transfer requests.

Note that implicit autorenews will still occur for suspended
registrars, as in our point-in-time data model there's no good way
to prevent them. So when a registrar is suspended for non-payment,
the game plan is to get all of their domains transferred out to a
registrar that will pay as soon as possible.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=223173677
2018-12-03 19:10:45 -05:00
guyben
5f283ebd09 Use AuthenticatedRegistrarAccessor in EppConsoleAction
EppConsoleAction still "manually" checks access by going over the
RegistrarContacts. We need it to use AuthenticatedRegistrarAccessor just like
every other part of the registrar console.

We still need to remove the (now unneeded) login EPP sent by the console, but that's left for a followup CL.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222404208
2018-12-03 18:51:40 -05:00
jianglai
a0aeec6f2f Update bazel version to 0.19.2
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=222309479
2018-12-03 18:48:28 -05:00
jianglai
4a9b715020 Update bazel version to 0.17.2
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=215603517
2018-10-08 16:59:29 -04:00
jianglai
f4d392eed9 Update to bazel 0.17.1
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213271889
2018-09-20 11:19:36 -04:00
jianglai
8d675a4b8c Remove checking of SNI headers
This is only useful when we used the [] proxy because the GFE requires SNI during handshake in order to request the client certificate. The GCP proxy does not need this (it always requests the client certificate). We do not need to check for its existence.

Also removed the checking of internal headers for ssl cert hash used only by the [] proxy.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213059027
2018-09-14 21:34:44 -04:00
mcilwain
8ddbf88151 Remove the deprecated MISTAKEN_PREMIUM reservation type
It doesn't do anything that ALLOWED_IN_SUNRISE doesn't do, and there's no point
in having two separate types when we can simply keep track of the semantic
difference between the two by using different lists (as we have for .soy).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=212997752
2018-09-14 21:28:23 -04:00