{"push_id":2561422289,"size":14,"distinct_size":14,"ref":"refs/heads/periodic-import","head":"fdd78f6cc4f0d20ace9ab1ffa66e4dbe9e02e200","before":"3700b344ce417da000be78aa03415898a336700f","commits":[{"sha":"40f132284665df817317bc10a851457ecb5be5b9","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(chore) Cache pre-commit to speedup build on CircleCI","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/40f132284665df817317bc10a851457ecb5be5b9"},{"sha":"b278180db2a7d05a633cf0825817e3f592c7c864","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(refactor) Move startup code to more appropriate location","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/b278180db2a7d05a633cf0825817e3f592c7c864"},{"sha":"d1ff3693479fe2e666c2f7de91ce15956a5cc24b","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(feat) Use aiohttp rather than gunicorn\n\nPros\n\n  - Event-loop model is usually good for high performance networking. There\n    will be some data transformation, but mostly this code will just be\n    shifting data about.\n  - The code will be doing 2 main things: intermittantly pulling from services,\n    and accepting data in real time. Suspect it will be easier to reason about\n    any sort of race conditions knowing that nothing can interrupt other code,\n    which would be the case with threads/processes. [Even with threads, things\n    would be a bit \"interrupted\"-y, in-spite of the GIL.]\n  - Quick to get going, i.e. single simple route, which is all current plan\n    calls for.\n  - Multiple routes can be added without it getting too hacky, if we ever need\n    to\n  - The API seems \"appropriately light but powerful\", so I suspect when there\n    is more code, should be straightforward to make low-level changes.\n  - It's very straightfoward to startup from Python code. Gunicorn can do it, but\n    it feels more of a hack/brittle. This will be integrated with a cron-type\n    poll of services, and so it gives a bit more confidence that we won't have to\n    work around / \"fight\" its API.\n  - Supports incoming and outgoing requests, so will have a single dependency\n    and single mental model for asynchronously...\n      - Pulling data from services\n      - Data from services being pushed\n      - Pushing data to ES\n  - If we want to use longer-term connections in future, it does support\n    websockets out of the box.\n\nCons (but deemed to be minor)\n\n  - Out of the box it doesn't support parallelism, i.e. everything will be on 1\n    core. However, I suspect this won't be needed any time soon (mostly will be\n    network-bound/bound by the speed of other services?), there are alternatives\n    e.g. firing up separate instances, and it _can_ be combined with Gunicorn if\n    ever needed, so this door isn't closing.\n\nThe logging format chosen here is based on the default from\nhttps://docs.aiohttp.org/en/stable/logging.html, but with X-Forwarded-For\nadded. For now, it just logs to stdout, since that's what it was doing before.\n\nThe logging format is not configurable by environment variable, since there is\nno evidence this needs to be configured on a per-environment basis.\n\nThis commit is \"almost\" a refactor in that GET \"/\" responds with a 200, but\nthere are differences in startup, logging, as well as how other URLs respond.\nThe default port is now also 8080 rather than 8000.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/d1ff3693479fe2e666c2f7de91ce15956a5cc24b"},{"sha":"798073c2dad45e3d63dda5d4dfd14d5c02d1e77f","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(refactor) Separate to functions","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/798073c2dad45e3d63dda5d4dfd14d5c02d1e77f"},{"sha":"4ac40cd1094a898a63959042871c83489d45a7ba","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(feat) Only run application if running as main\n\nThis should allow testing of parts of the application without having to fire up\na separate process (as is the case with the single current test), or doing an\nimport mid-test.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/4ac40cd1094a898a63959042871c83489d45a7ba"},{"sha":"94947aec6aa566e7002a38e53dbc45fa9f3fcdd2","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(refactor) More accurate name to be able to distinguish from other tests\n\nMost tests are unlikely to fire up the server via a process: there is likely to\nbe some mocking involved.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/94947aec6aa566e7002a38e53dbc45fa9f3fcdd2"},{"sha":"c7083663d770580b29a90d288ef7342eed10b34f","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(feat) Allow server to itself be called from the event loop\n\nThis is so (hopefully) will be able to test `run_application`. This does put a\nbit more code outside of `run_application`, but as long as there is some higher\nlevel testing, this should be fine.\n\nThere is less graceful shutdown on iterrupt here. This should be thought about.\nHowever, since the system has to handle ungraceful shutdown _anyway_, say, if\nthe surrounding container just dies, then spending time keeping graceful\nshutdown may not be worthwile. This isn't user-facing, so reducing chance of a\nuser seeing an error isn't a concern.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/c7083663d770580b29a90d288ef7342eed10b34f"},{"sha":"d2c89b57a4a5d3b51d0d8fd1c51ac46673e6e3f6","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(refactor) Make test for http re-usable","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/d2c89b57a4a5d3b51d0d8fd1c51ac46673e6e3f6"},{"sha":"a0f94bff4068989fd19422e179eb0de6dbe27a33","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(fix) Text for http connection is less racy","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/a0f94bff4068989fd19422e179eb0de6dbe27a33"},{"sha":"6faee8e2da2d42df0e04cc6de5e30a26c0d95b66","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(test) Test also for connection errors, e.g. if port isn't listening","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/6faee8e2da2d42df0e04cc6de5e30a26c0d95b66"},{"sha":"245d3474d889f28700a27f3c34237dad9f20459a","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(refactor) Test for http connectivity uses event loop\n\nThis is so it can use used in other tests to not block the event loop","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/245d3474d889f28700a27f3c34237dad9f20459a"},{"sha":"e55f3094efd6e0a8d0c0ccc2e02dc49c05df6664","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(test) Test application in the same process as the tests\n\nThis is to start building a suite a tests that would be able to run quicker or\nmock (where appropriate) implementation details.\n\nThe patching here is so we have access to the runner to be able to shut the server\ndown properly so ports are closed for other tests.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/e55f3094efd6e0a8d0c0ccc2e02dc49c05df6664"},{"sha":"00fbb788a4e85a2981f88bf62a7db3b65e31c5a1","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(feat) Always use current python to trigger process\n\nThis is to allow the tests to be triggered from a Virtual Env that might have a\ndifferent python version to the one in the path.","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/00fbb788a4e85a2981f88bf62a7db3b65e31c5a1"},{"sha":"fdd78f6cc4f0d20ace9ab1ffa66e4dbe9e02e200","author":{"name":"Michal Charemza","email":"a2df1e659c9fd2578de0a26565357cb273292eeb@charemza.name"},"message":"(feat) Simple polling\n\nAt this point it doesn't do anything with the results of the poll","distinct":true,"url":"https://api.github.com/repos/uktrade/activity-stream/commits/fdd78f6cc4f0d20ace9ab1ffa66e4dbe9e02e200"}]}
{"actor":{"display_login":"michalc"}}