mirror of
https://github.com/google/nomulus.git
synced 2025-04-29 19:47:51 +02:00
2294c77306
This will replace the ExpandRecurringBillingEventsAction, which has a couple of issues: 1) The action starts with too many Recurrings that are later filtered out because their expanded OneTimes are not actually in scope. This is due to the Recurrings not recording its latest expanded event time, and therefore many Recurrings that are not yet due for renewal get included in the initial query. 2) The action works in sequence, which exacerbated the issue in 1) and makes it very slow to run if the window of operation is wider than one day, which in turn makes it impossible to run any catch-up expansions with any significant gap to fill. 3) The action only expands the recurrence when the billing times because due, but most of its logic works on event time, which is 45 days before billing time, making the code hard to reason about and error-prone. This has led to b/258822640 where a premature optimization intended to fix 1) caused some autorenwals to not be expanded correctly when subsequent manual renews within the autorenew grace period closed the original recurrece. As a result, the new pipeline addresses the above issues in the following way: 1) Update the recurrenceLastExpansion field on the Recurring when a new expansion occurs, and narrow down the Recurrings in scope for expansion by only looking for the ones that have not been expanded for more than a year. 2) Make it a Beam pipeline so expansions can happen in parallel. The Recurrings are grouped into batches in order to not overwhelm the database with writes for each expansion. 3) Create new expansions when the event time, as opposed to billing time, is within the operation window. This streamlines the logic and makes it clearer and easier to reason about. This also aligns with how other (cancelllable) operations for which there are accompanying grace periods are handled, when the corresponding data is always speculatively created at event time. Lastly, doing this negates the need to check if the expansion has finished running before generating the monthly invoices, because the billing events are now created not just-in-time, but 45 days in advance. Note that this PR only adds the pipeline. It does not switch the default behavior to using the pipeline, which is still done by ExpandRecurringBillingEventsAction. We will first use this pipeline to generate missing billing events and domain histories caused by b/258822640. This also allows us to test it in production, as it backfills data that will not affect ongoing invoice generation. If anything goes wrong, we can always delete the generated billing events and domain histories, based on the unique "reason" in them. This pipeline can only run after we switch to use SQL sequence based ID allocation, introduced in #1831. |
||
|---|---|---|
| .. | ||
| builder | ||
| rollback | ||
| schema-deployer | ||
| schema-verifier | ||
| build_nomulus_for_env.sh | ||
| cloudbuild-delete.yaml | ||
| cloudbuild-deploy.yaml | ||
| cloudbuild-dev-resource.yaml | ||
| cloudbuild-kythe.yaml | ||
| cloudbuild-nomulus.yaml | ||
| cloudbuild-proxy.yaml | ||
| cloudbuild-release.yaml | ||
| cloudbuild-schema-deploy.yaml | ||
| cloudbuild-schema-verify.yaml | ||
| cloudbuild-sync.yaml | ||
| cloudbuild-tag.yaml | ||
| stage_beam_pipeline.sh | ||