Commit graph

59 commits

Author SHA1 Message Date
mcilwain
91febfd945 Add a nomulus tool command to get allocation tokens
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=239624232
2019-03-21 15:05:12 -04:00
weiminyu
acbd23fa64 Remove deprecated Datastore backup code
Removed three Action classes and the CheckSnapshot command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=230545631
2019-01-24 19:58:54 -05:00
weiminyu
9014165148 Add a Nomulus command to list Datastore operations
Command would be useful when checking the status of daily
backup jobs.

Only two modes are supported: list all operations, or list those
that started within the past period as specified by command line
option.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=229239783
2019-01-14 16:31:18 -05:00
mcilwain
580302898d Delete end-date sunrise, landrush, and sunrush phases
This also deletes the associated commands and domain application specific
entities.

We haven't used any of these TLD phases since early 2015 and have no
intent to do so in the future, so it makes sense to delete them now so we
don't have to carry them through the Registry 3.0 migration.

Note that, while there are data model changes, there should be no required
data migrations. The fields and entities being removed will simply remain
as orphans. I confirmed that the removed types (such as the SUNRUSH_ADD
GracePeriodType) are no longer used in production data, and left types
that are still used, e.g. BillingEvent.Flag.LANDRUSH or
HistoryEntry.Type.ALLOCATE.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228752843
2019-01-10 16:23:35 -05:00
weiminyu
4e71421c81 Support datastore restore in Nomulus tool
Two commands are being added:
- ImportDatastoreCommand starts an async import operation.
  User may choose to wait until import completes or quit
  immediately.
- GetOperationStatusCommand checks the status of an operation.
  It may be used to check the status of an operation started by
  ImportDatastoreCommand.

Both commands communicate with Datastore admin api directly, without
going through the Registry server.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=228400059
2019-01-10 16:23:35 -05:00
mcilwain
f58211402a Add an unrenew_domain command to nomulus tool
This is used to reduce the expiration time of domain(s) by some number of years
(if enough length remains in the registration term to do so). This does not back
out the previously saved BillingEvent entities as they may have already been
sent out and invoiced, so any related refunds must be handled out of band.

In addition to reducing the registration expiration time on the domain itself,
this command writes out a new history entry, one-time poll message informing the
registrar of this change, auto-renew billing event and poll message, and
updates/ends the old auto-renew billing event and poll message.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224999285
2018-12-12 13:22:34 -05:00
shicong
66d98c8d66 Add SetNumInstancesCommand in Nomulus tool to adjust the number of instances
for a given service and version at runtime.

Note that this CL only supports the adjustment for a given service and version. I will add another functionality to let this command be able to detect all non-live versions automatically and apply the adjustment.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=221092001
2018-11-12 14:51:40 -05:00
mcilwain
49e14387e7 Add nomulus command for deleting AllocationTokens
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=214459480
2018-10-03 11:54:10 -04:00
mcilwain
399cde1291 Add a command-line tool suitable for tech support usage
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=213634869
2018-09-20 11:19:36 -04:00
mcilwain
7b87ba41c7 Delete all Limited Release Program (LRP) code
We never used it and don't have any plans to use it going forward. All
conceivable parts of its functionality that we might use going forward have
already been subsumed into allocation tokens, which are a simpler way of
handling the same use case that are also standards-compliant.

Also gets rid of the hideous ANCHOR_ prefix on anchor tenant EPP authcodes
that was only ever necessary because of overloading the authcode for
anchor tenant creation. Going forward it'll be based on allocation tokens,
so there's no risk of conflicts.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209418194
2018-08-20 14:28:16 -04:00
mmuller
e3977024f3 Create a nomulus "curl" command
Create a command to send arbitrary, authenticated HTTP requests to the backend
and remove the existing commands that are basically just wrappers around this.

Tested:
  In addition to the unit tests, verified both get and post requests against
  alpha.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=207756509
2018-08-10 13:46:48 -04:00
larryruili
d199b383e5 Add preliminary spec11 monthly pipeline
This adds the scaffolding for a basic Spec11 pipeline- it gathers all domains from all time for a given project and counts how many there are. I've factored out a few common utilities for beam pipelines to avoid excessive duplication.

Future CLs will:
- Actually process domains via the SafeBrowsing API
- Generate a real spec11 report
- Template queries based on the input YearMonth
- Abstract more commonalities across beam pipelines to reduce boilerplate when adding new pipelines.

TESTED: FOSS test passed, and ran successfully on alpha

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=205997741
2018-08-10 13:44:25 -04:00
mcilwain
6706b99828 Delete remnants of registrar-level credits
We never fully used this stuff but definitely no longer use it following our
recent billing refactor. It's confusing to retain all of these entities and
commands given that none of them are actually used by anything.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201978094
2018-06-27 15:28:52 -04:00
mcilwain
aa11d5681f Delete ancient registrar activity report command
We haven't used this in years and I doubt it's still even functional. It's using
the legacy BigQuery tables and the pre-Registry 2.0 schema, for starters.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=201960046
2018-06-27 15:28:52 -04:00
mcilwain
897690119e Add a nomulus renew_domain command
This may come in handy when dealing with URS requests.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=199332085
2018-06-06 15:16:09 -04:00
mcilwain
92190f8699 Merge the check_domain_fee nomulus command into check_domain
I'm tired of running check_domain only to then realize I needed to run
check_domain_fee instead because the domain ended up being premium. We require
the use of the fee extension to register domains on all of our TLDs anyway so we
might as well always be sending the fee extension when checking domains
too. There's no additional harm in sending along the fee extension (it only
sends back additional information that is often useful), so just make that the
default and remove the extra unnecessary command.

Note that check_domain_claims can't be merged in too because it fundamentally
works differently. It doesn't query the availability of domain names for
registration, just whether they're trademarked.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=198097326
2018-05-30 12:18:54 -04:00
mcilwain
c242a4d08f Add nomulus count_domains command
This efficiently counts domains without having to load them (as opposed to the
existing list_domains command which does load then).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=194559095
2018-05-05 23:37:11 -04:00
mcilwain
0923c89981 Fix verb order in nomulus tool domain check commands
This makes them consistent with every other command, which uses the format
verb_noun.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=192301468
2018-04-10 17:04:07 -04:00
mmuller
f1c29633fb Add a "shell" pseudo-command to nomulus tool
Add the "shell" command which lets you run multiple other command in a single
session, sparing you the initialization costs for all but the first of them.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=188712815
2018-03-19 18:22:01 -04:00
larryruili
5f218b4a8b Delete obsolete billing code
Now that we've verified the new Beam billing pipeline works, we can delete the
old manual commands we used to use.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=184707182
2018-02-20 15:09:45 -05:00
mcilwain
646dcecd7e Create GenerateAllocationTokens nomulus tool command
This creates a specified number of tokens of a given schema, with a dryrun option
to not persist them.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=181403775
2018-01-19 14:26:56 -05:00
mcilwain
c2ed7429d3 Delete AuctionStatusCommand
It's untested, we suspect it's not actually working properly, and we don't
intend to ever need to use anything having to do with auctions ever again.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178815580
2017-12-13 12:43:45 -05:00
larryruili
026f249955 Add Dataflow path to generate overall monthly invoice
This forks off the results of the billing_events query from Bigquery and generates the overall invoice for the month.

This also turns the ad hoc beam binary into a proper Nomulus command, allowing us to @Inject from the RegsistryToolComponent and thus template the BillingEvent Bigquery query.

Next steps:
- Add the beam package to the open source build
  - That will likely require updates to the open source jars, since I'm using features from 2.2.0 which was only just released.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=178798943
2017-12-13 12:43:45 -05:00
mcilwain
6eb0d8689d Add domain_unlock nomulus command
This command is used by registry operators to apply registry locks to
domain names.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176577240
2017-11-23 01:09:07 -05:00
mcilwain
f041b1bac0 Add domain_lock nomulus command
This command is used by registry operators to apply registry locks to
domain names.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=176549874
2017-11-21 18:54:42 -05:00
mmuller
44df5da771 Add "create_cdns_tld" command to nomulus
Automated g4 rollback of changelist 171011304.  Please see history below, the original CL was []  Rolling back and re-mailing
so I can get a readability review on it.

*** Reason for rollback ***

Re-opening for readability review.

*** Original change description ***

Automated g4 rollback of changelist 170906329.

*** Reason for rollback ***

Forgot to send this for readability review.

*** Original change description ***

Add "create_cdns_tld" command to nomulus

Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171161371
2017-10-10 12:09:41 -04:00
mmuller
a9ecccf672 Automated g4 rollback of changelist 170906329.
*** Reason for rollback ***

Forgot to send this for readability review.

*** Original change description ***

Add "create_cdns_tld" command to nomulus

Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

b/67367533 created to clean this up, along with...

***

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=171011304
2017-10-04 16:16:45 -04:00
mmuller
6e84401271 Add "create_cdns_tld" command to nomulus
Add a command to allow us to create the managed zone for a new TLD in cloud
dns.

Note: this implementation is problematic, it's currently doing its own thing
as far as credentials and http transport, making it unusable with the []
channel and the credentials generated by login.  Unfortunately, fixing the
plumbing to make it work right is difficult, and we have an immediate need for
this functionality.

b/67367533 created to clean this up, along with related warts in the codebase.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=170906329
2017-10-04 16:16:45 -04:00
bbilbo
bbdf9bfc38 Refactor CreateDomainCommand and add UpdateDomainCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=160282921
2017-07-10 11:22:39 -04:00
nickfelt
c41cf9142a Remove FixDomainTransferDataCommand
Scrap tool is no longer needed.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159500743
2017-06-21 09:53:03 -04:00
nickfelt
fc78fd7d4a Update transfer data fixing command to fix one more domain
I happened to notice that one domain affected by b/33289763 had existing transfer data, so in that case the data got overwritten to look like a server cancellation (a year after the transfer request, so obviously wrong) rather than creating invalid transfer data, and hence wasn't picked up in the previous fixes.  See b/33289763#comment32 for more details.

This CL renames the RemoveDomainTransferDataCommand used to fix the errors to FixDomainTransferDataCommand and changes its behavior to instead reset the TransferData to a server-approval result.  It also changes it to load domains by ROID because loadByForeignKey() fails for deleted domains.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=159273225
2017-06-21 09:49:53 -04:00
mmuller
b733443eab Create nomulus tool restore command
Create a nomulus tool restoreCommitLogs command.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157103177
2017-06-05 18:17:09 -04:00
mountford
ddb0f5692e Add golden files for request component routing maps
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=157026188
2017-06-05 18:17:09 -04:00
larryruili
b08e317505 Add scrap tool for populating required registrar fields
This tool finds all registrars that have a null name, or a null localizedAddress field, and populates them as follows:

- The registrar's name is updated to its clientId
- The registrar's empty localizedAddress fields are replaced with fake data

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=155739946
2017-05-17 12:01:49 -04:00
mountford
8bb66dd365 Add delete_host nomulus tool command
This is needed to delete an invalid host resulting from a previous bug.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=154757425
2017-05-03 11:08:02 -04:00
guyben
47870f98eb Add UpdateKmsKeyringCommand to update keys in the KMS Keyring
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=152253868
2017-04-10 13:26:43 -04:00
guyben
c3df4e26a3 Add GetKeyringSecretCommand to retrieve keyring secrets.
Cloned from CL 149476124 by 'g4 patch'.
Original change by shikhman@shikhman:registry-secrets-2:897:citc on 2017/03/07 15:37:09.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151950797
2017-04-05 10:13:45 -04:00
mountford
62c7a3935a Add nomulus tool command to delete a TLD
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151863632
2017-04-05 10:10:38 -04:00
mmuller
5ffb424682 Add a "loadtest" command to nomulus tool
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151034721
2017-03-27 13:16:35 -04:00
nickfelt
aa35d74900 Remove UpdateCreditsCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=151023241
2017-03-27 13:12:00 -04:00
mmuller
5614760d53 Implement login/logout commands
Refactor the auth code into its own dagger module, add tests and use the new interfaces to implement the login and logout commands.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=149108266
2017-03-07 13:45:11 -05:00
ctingue
a5932c0fc3 Drop FixDomainNameserverKeysCommand
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=148688861
2017-03-07 13:16:34 -05:00
mcilwain
32b236e940 Delete the unsafe DeleteEppResourceCommand
It was somewhat unsafe to run because it bypassed some of the checks
that are usually run when attempting to delete EPP resources. The
DeleteDomainCommand is a recommended replacement that uses SOY templates
to delete a domain using EPP. Similar commands to delete hosts and
contacts can be written if required.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147634146
2017-02-17 12:16:27 -05:00
mcilwain
cdadb54acd Refer to Datastore everywhere correctly by its capitalized form
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147479683
2017-02-17 12:12:12 -05:00
ctingue
335b3f0cf8 Scrap tool to fix DomainResource keys to deleted nameservers
There are about 25 active domains that have keys that point to deleted nameservers. In all cases, there are active nameservers with the same FQDN that these domains should be pointing to. Given the domains in question (pulled via BigQuery, see the bug), update the domain with the correct keys whenever a deleted host is found.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=147346660
2017-02-14 12:08:16 -05:00
mmuller
b70f57b7c7 Update copyright year on all license headers
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=146111211
2017-02-02 16:27:22 -05:00
mountford
7cf29366bc Add scrap tool to remove transfer data
We have wound up with a few domains with invalid transfer data; the transfer status is SERVER_CANCELLED, but the other data is missing. This tool should set the transfer data for the specified domain back to null in the database.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=140883792
2016-12-06 11:52:46 -05:00
cgoldfeder
419a04bc26 Command to resave datastore entities by websafe key
While I am doing this, promote LoadAndResaveEntityCommand from
javascrap to the main tool since it's repeatedly been useful, and
rename it and update its documentation to better reflect the
difference between that command and the one I am adding.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=139460841
2016-11-28 11:23:29 -05:00
shikhman
f76bc70f91 Preserve test logs and test summary output for Kokoro CI runs
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135494972
2016-10-14 16:57:43 -04:00
mcilwain
3b02d77ceb Rename 'registry_tool' to 'nomulus'
This changes everything with external visibility beyond the codebase
(i.e. the name of the compiled binary and the documentation that refers
to it). It does not change a lot of things internal to the codebase,
i.e. the "RegistryTool" class didn't change its name. We can rename that
in a subsequent CL if we want to.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=135022087
2016-10-04 09:59:54 -04:00