{"action":"created","issue":{"url":"https://api.github.com/repos/python/mypy/issues/1292","repository_url":"https://api.github.com/repos/python/mypy","labels_url":"https://api.github.com/repos/python/mypy/issues/1292/labels{/name}","comments_url":"https://api.github.com/repos/python/mypy/issues/1292/comments","events_url":"https://api.github.com/repos/python/mypy/issues/1292/events","html_url":"https://github.com/python/mypy/pull/1292","id":140763686,"number":1292,"title":"The mother of all pull requests: add --incremental","user":{"login":"gvanrossum","id":2894642,"avatar_url":"https://avatars.githubusercontent.com/u/2894642?v=3","gravatar_id":"","url":"https://api.github.com/users/gvanrossum","html_url":"https://github.com/gvanrossum","followers_url":"https://api.github.com/users/gvanrossum/followers","following_url":"https://api.github.com/users/gvanrossum/following{/other_user}","gists_url":"https://api.github.com/users/gvanrossum/gists{/gist_id}","starred_url":"https://api.github.com/users/gvanrossum/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gvanrossum/subscriptions","organizations_url":"https://api.github.com/users/gvanrossum/orgs","repos_url":"https://api.github.com/users/gvanrossum/repos","events_url":"https://api.github.com/users/gvanrossum/events{/privacy}","received_events_url":"https://api.github.com/users/gvanrossum/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-03-14T18:47:57Z","updated_at":"2016-03-16T21:02:52Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/python/mypy/pulls/1292","html_url":"https://github.com/python/mypy/pull/1292","diff_url":"https://github.com/python/mypy/pull/1292.diff","patch_url":"https://github.com/python/mypy/pull/1292.patch"},"body":"tl;dr: Please review in this order: semanal.py+tests; nodes.py+types.py; fixup.py; build.py; main.py.\r\n\r\nSee https://paper.dropbox.com/doc/Mypy-incremental-checking-ifj77IF7eNNySxhXYAaBe#:h=Technical-debt for some technical debt (read the paper from the top for a more complete overview).\r\n\r\nDespite that technical debt I'd like to merge this into master in the current state, because at this point it works well enough and in particular I've tested it with typeshed's runtests.py script as well as with the selection of files in the internal Dropbox server repo. Some considerations:\r\n\r\n- The new (SCC + topsort) algorithm for determining dependencies is always used\r\n- The incremental (cache) feature is only used when `-i/--incremental` is used\r\n- After merging this would require additional real-world testing before we can do a release\r\n- My own biggest concern is the hack for the sqlalchemy cycle: https://github.com/gvanrossum/mypy/commit/ede81849a636e88d4f048472bc290224aa0cc7a0\r\n\r\nDon't be fooled by the large number of commits! There are a lot of dead ends and U-turns that you can ignore. Most of the changes are limited to a small number of files:\r\n\r\n- build.py: the new dependency manager (I ripped out the old one)\r\n- nodes.py, types.py: added [de]serialize() methods to many classes\r\n- fixup.py: new module to fix up symbol tables after deserialization\r\n- main.py: added the `-i/--incremental` option\r\n- semanal.py: a fix for anonymous namedtuples needed by serialization\r\n- some tests added/changed for the namedtuple fix\r\n- .gitignore the .mypy_cache directory (though not even sure I want this)\r\n\r\nSomehow GitHub refuses to show the diffs for build.py. Sorry about this; I guess you have to view them locally. You could also try to view the incremental diffs to this file starting with https://github.com/python/mypy/commit/0b4ef76db49c0cb3719ef7d3c6688bcfc2c81b3a. (Also note that the dates GitHub lists for the diffs are bogus -- they represent the most recent rebase. The actual work started over a month ago.)\r\n\r\nSome questions about how to merge this monster into master. I've got a feeling that the detailed commits from my branch are not that useful. I propose the following order, as separate commits to master:\r\n\r\n1. the semanal.py changes and related tests\r\n2. (de)serialization methods in nodes.py, types.py\r\n3. the rest: fixup.py, build.py, main.py, .gitignore\r\n\r\nThis may also represent a reasonable review order."},"comment":{"url":"https://api.github.com/repos/python/mypy/issues/comments/197548387","html_url":"https://github.com/python/mypy/pull/1292#issuecomment-197548387","issue_url":"https://api.github.com/repos/python/mypy/issues/1292","id":197548387,"user":{"login":"JukkaL","id":1107911,"avatar_url":"https://avatars.githubusercontent.com/u/1107911?v=3","gravatar_id":"","url":"https://api.github.com/users/JukkaL","html_url":"https://github.com/JukkaL","followers_url":"https://api.github.com/users/JukkaL/followers","following_url":"https://api.github.com/users/JukkaL/following{/other_user}","gists_url":"https://api.github.com/users/JukkaL/gists{/gist_id}","starred_url":"https://api.github.com/users/JukkaL/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/JukkaL/subscriptions","organizations_url":"https://api.github.com/users/JukkaL/orgs","repos_url":"https://api.github.com/users/JukkaL/repos","events_url":"https://api.github.com/users/JukkaL/events{/privacy}","received_events_url":"https://api.github.com/users/JukkaL/received_events","type":"User","site_admin":false},"created_at":"2016-03-16T21:02:51Z","updated_at":"2016-03-16T21:02:51Z","body":"Agreed, I'm not suggesting to implement anything fancy right now, but there are some ideas I had for potential future things.\r\n\r\nDavid's suggestion is great -- we should definitely do that."}}