Commit graph

5 commits

Author SHA1 Message Date
larryruili
74ced1e907 Refactor ICANN reporting and billing into common package
This moves the default yearMonth logic into a common ReportingModule, rather than the coarse-scoped BackendModule, which may not want the default parameter extraction logic, as well as moving the 'yearMonth' parameter constant to the common package it's used in. This also provides a basis for future consolidation of the ReportingEmailUtils and BillingEmailUtils classes, which have modest overlap.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=183130311
2018-02-01 21:56:06 -05:00
larryruili
38abe9fa48 Optimize activity report SQL
We only log FLOW-LOG-SIGNATURE-METADATA from one place- FlowRunner. As a
result, we can swap the generalized regex for a prefix-only regex, saving a
<strong>lot</strong> of processing for our epp query (which is the most
expensive of the bunch).

I've also changed the test dates from 2017-05 to 2017-06, allowing us to copy-paste
the test data into Bigquery to verify their function. The reason for 2017-06 in particular is because June was the first month that populated all the metadata necessary to generate these reports.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=165391715
2017-08-29 16:22:27 -04:00
larryruili
477617eec9 Add activity report generation code
This adds Bigquery API client code to generate the activity reports from our
now standardSQL queries. The naming mirrors that of RDE (Staging generates the
reports and uploads them to GCS).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=164656344
2017-08-29 15:53:33 -04:00
larryruili
33eb5f1c87 Upgrade activity reporting queries to StandardSQL
This also brings the SQL template parameters in-line with the anticipated Bigquery dataset format, and switches from DateTime to the more appropriate LocalDate (since we only need monthly granularity).

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=162747692
2017-08-01 16:56:12 -04:00
larryruili
4887811fc3 Add activity reporting SQL query generation code
This allows us to have a modular view of all tables used in activity reporting, to facilitate generating reports in BigQuery.

-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=161849007
2017-08-01 16:23:31 -04:00