{"action":"opened","issue":{"url":"https://api.github.com/repos/mortendahl/tf-encrypted/issues/307","repository_url":"https://api.github.com/repos/mortendahl/tf-encrypted","labels_url":"https://api.github.com/repos/mortendahl/tf-encrypted/issues/307/labels{/name}","comments_url":"https://api.github.com/repos/mortendahl/tf-encrypted/issues/307/comments","events_url":"https://api.github.com/repos/mortendahl/tf-encrypted/issues/307/events","html_url":"https://github.com/mortendahl/tf-encrypted/issues/307","id":383925195,"node_id":"MDU6SXNzdWUzODM5MjUxOTU=","number":307,"title":"Integration testing against different tensorflow versions.","user":{"login":"justin1121","id":1417322,"node_id":"MDQ6VXNlcjE0MTczMjI=","avatar_url":"https://avatars1.githubusercontent.com/u/1417322?v=4","gravatar_id":"","url":"https://api.github.com/users/justin1121","html_url":"https://github.com/justin1121","followers_url":"https://api.github.com/users/justin1121/followers","following_url":"https://api.github.com/users/justin1121/following{/other_user}","gists_url":"https://api.github.com/users/justin1121/gists{/gist_id}","starred_url":"https://api.github.com/users/justin1121/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/justin1121/subscriptions","organizations_url":"https://api.github.com/users/justin1121/orgs","repos_url":"https://api.github.com/users/justin1121/repos","events_url":"https://api.github.com/users/justin1121/events{/privacy}","received_events_url":"https://api.github.com/users/justin1121/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-11-23T20:54:40Z","updated_at":"2018-11-23T20:54:40Z","closed_at":null,"author_association":"COLLABORATOR","body":"With the addition of custom operations and their related shared object, we need to build a shared object for each version of tensorflow we want to support. In PR https://github.com/mortendahl/tf-encrypted/pull/304 we've implicitly set up support for tf versions 1.9+. (1.10, 1.11, 1.12). \r\n\r\nA big question we have is whether we want to actually support all these versions or just the latest. For this issue I'm going to assume we're support all these versions. To do this we need to introduce some sort of integration testing to test against all of these versions. There are a few different options for what we can do here.\r\n\r\n1. Set up a testing framework that runs on every commit.\r\n\r\nThere are some major downsides to this option. Right now tests will need to be run on four different version of tensorflow that could take 5 to 10 minutes for each version of tf. This feedback loop would be to slow for PRs and development. There are some things we could do to speed this up. Cache versions of the shared objects, etc.\r\n\r\nOne idea here is to only test a subset of tests on every commit to speed up the testing. We could combine this with the below nightly builds suggestion to make sure everything gets tested thoroughly.\r\n\r\n2. Nightly builds\r\n\r\nDo nightly builds off master where we test, build and release nightly versions. This way some of the time can be offloaded to later in the day and bugs can still be caught in a timely manner. This would need to be automated somehow and maybe we wouldn't necessarily need to do a full release.\r\n\r\n3. Do a full test only on release\r\n\r\nPerhaps we can just put off testing thoroughly until the very last moment and test everything once we have a release candidate ready to ship. The major downside with this is that if we have a release day scheduled and some major bug isn't discovered until then it could mess up the release day!!"}}
{"actor":{"display_login":"justin1121"}}