This is for consistency, mostly the LocalDate fields added in []
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224525451
Add a sendSpec11Email parameter that allows us to only send the email on
one run per month. Next, we will compute the diffs between the daily runs
and send daily emails with those diffs.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224404653
We are moving away from using Application Default Credentials generated by "gcloud auth application-default login" in our code base and consolidate on using self-managed credentials provided from AuthModule.
One of the remaining dependencies on the ADCs is from beam pipeline deployment commands, which by default use the ADCs to talk to GCS and upload the jar files and templates. In this CL, we explicitly provide the locally created credential to the Options used in deployments.
Also moved all credential qualifiers to CredentialModule, and removed @AppEngineAdminApiCredential, which is no longer used.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=224199812
This adds the terminal step of the Spec11 pipeline- processing the output of
the Beam pipeline to send an e-mail to each registrar informing them of
identified 'bad urls.'
This also factors out methods common between invoicing (which uses similar beam pipeline tools) and spec11 to the common superpackage ReportingModule + ReportingUtils classes.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=210932496
This changes the BigQuery input to the fields we ultimately want (fqdn,
registrarName, registrarEmailAddress) and the output to a structured POJO
holding the results from the API. This POJO is then converted to its final text output, i.e.:
Map from registrar e-mail to list of threat-detected subdomains:
{"registrarEmail": "c@fake.com", "threats": [{"url": "a.com", "threatType": "MALWARE"}]}
{"registrarEmail": "d@fake.com", "threats": [{"url": "x.com", "threatType": "MALWARE"}, {"url": "y.com", "threatType": "MALWARE"}]}
This gives us all the data we want in a JSON structured format, to be acted upon downstream by the to-be-constructed PublishSpec11ReportAction. Ideally, we would send an e-mail directly from the beam pipeline, but this is only possible through third-party providers (as opposed to app engine itself).
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=209416880
This adds actual subdomain verification via the SafeBrowsing API to the Spec11
pipeline, as well as on-the-fly KMS decryption via the GenerateSpec11Action to
securely store our API key in source code.
Testing the interaction becomes difficult due to serialization requirements, and will be significantly expanded in the next cl. For now, it verifies basic end-to-end pipeline behavior.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=208092942
It broke because I forgot to add the new spec11 packages to gtld.
-------------
Created by MOE: https://github.com/google/moe
MOE_MIGRATED_REVID=206021827
