{"push_id":2909485809,"size":1,"distinct_size":1,"ref":"refs/heads/828-centralize-event-publication","head":"d65ad802a8e8faf4df16508b6c4555bf5692696b","before":"f1c72660c897c21bbbc2d39cea95e2a3ca48cc6c","commits":[{"sha":"d65ad802a8e8faf4df16508b6c4555bf5692696b","author":{"name":"Andres Arana","email":"443a0f1b59528d1b89de5af44e0f1ee6a8fc8ed4@gmail.com"},"message":"Centralizes event publication\n\nThe general architecture here is to use subdags to modularize the\ndifferent events that we publish.\n\nA new version of `pipe-tools` is used as well. It includes a new feature\nwhich allows us to define a base config when building subpipelines. This\nlet's us configure generic stuff on the parent pipeline (such as project\nid, standard buckets and so on) while being able to override some of\nthose or add new config options in the sub pipelines.\n\nThe fishing events pipeline requires us to parametrize\nthe inference buffer days, a setting that's also used in\n`pipe-features`. To do this, we'll pull directly from the global\n`INFERENCE_BUFFER_DAYS` airflow variable. We can't use the general\nstrategy we do with other global variables where we use the `{{\nvar.value.GLOBAL_VARIABLE }}` value in a local\n`pipe_events.fishing.local_variable` variable here, since we use this\nparameter when building the start and end dates of the task and this is\nalready being interpolated. This means that we would need to recursively\ninterpolate variables, and airflow doesn't do that.","distinct":true,"url":"https://api.github.com/repos/GlobalFishingWatch/pipe-events/commits/d65ad802a8e8faf4df16508b6c4555bf5692696b"}]}
{"actor":{"display_login":"andres-arana"}}