{"push_id":1130361807,"size":2,"distinct_size":2,"ref":"refs/heads/e10s-jnothing","head":"d8933a13dddcdb96d1825ecaeb987a591e32faa8","before":"170c9110b722ecbe8220451fd4028d675776558e","commits":[{"sha":"ca1d281ba07818c27ed1c520749b51bc4df0b4a3","author":{"email":"2d045cf25aece9c2053bc34b8b7d19180f0a2b11@mozilla.com","name":"Mark Reid"},"message":"Rewrite \"Main Summary job\" to use SparkSQL schema.\n\n* Refactor \"Main Summary\" to drop avro, use \"Views\".\r\n\r\nPer bug 1272334, refactor the Main Summary code to use Spark SQL types\r\ndirectly (rather than using Avro Schema), and switch from the \"Derived\r\nStream\" approach to the \"Views\" approach.\r\n\r\nThis incorporates the \"utils.Telemetry\" code in #62.\r\n\r\nProcessing a single day's data on a 20-node cluster (April 1, 2016 was\r\nmy reference date) takes a bit more than 90 minutes with this current\r\ncode. Using the DerivedStreams approach from a few commits back takes\r\nabout half that amount of time.\r\n\r\nConsumer-facing changes in the v3 dataset include:\r\n- Renaming the \"submission_date_s3\" field to \"submission_date\"\r\n- Changing the type of the \"sample_id\" field to a string due to\r\n  using it as an s3 partitioning field.\r\n\r\n* Fix the S3 partitioning logic.\r\n\r\n* Re-enable the tests for \"Longitudinal\".\r\n\r\nThe MainSummaryView serialization doesn't run when using\r\nparquet-avro 1.8.1, so disable that for now. Also adjust\r\nthe .sbtopts file to avoid an OOM when running tests.\r\n\r\n* Remove a spurious print stmt","distinct":true,"url":"https://api.github.com/repos/chutten/telemetry-batch-view/commits/ca1d281ba07818c27ed1c520749b51bc4df0b4a3"},{"sha":"d8933a13dddcdb96d1825ecaeb987a591e32faa8","author":{"email":"e4e4f6cc692beabc62f4695439533d9dab7b2f69@mozilla.com","name":"Chris H-C"},"message":"e10sCohort and e10sEnabled can occasionally be JNothing\n\nThey shouldn't be, but they are.","distinct":true,"url":"https://api.github.com/repos/chutten/telemetry-batch-view/commits/d8933a13dddcdb96d1825ecaeb987a591e32faa8"}]}
{"actor":{"display_login":"chutten"}}