{"action":"created","issue":{"url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/609","repository_url":"https://api.github.com/repos/open-source-economics/Tax-Calculator","labels_url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/609/labels{/name}","comments_url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/609/comments","events_url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/609/events","html_url":"https://github.com/open-source-economics/Tax-Calculator/issues/609","id":135852409,"number":609,"title":"Remove unused variables from puf.csv file","user":{"login":"martinholmer","id":12170745,"avatar_url":"https://avatars.githubusercontent.com/u/12170745?v=3","gravatar_id":"","url":"https://api.github.com/users/martinholmer","html_url":"https://github.com/martinholmer","followers_url":"https://api.github.com/users/martinholmer/followers","following_url":"https://api.github.com/users/martinholmer/following{/other_user}","gists_url":"https://api.github.com/users/martinholmer/gists{/gist_id}","starred_url":"https://api.github.com/users/martinholmer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martinholmer/subscriptions","organizations_url":"https://api.github.com/users/martinholmer/orgs","repos_url":"https://api.github.com/users/martinholmer/repos","events_url":"https://api.github.com/users/martinholmer/events{/privacy}","received_events_url":"https://api.github.com/users/martinholmer/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-02-23T20:16:52Z","updated_at":"2016-02-29T20:58:36Z","closed_at":null,"body":"Tax-Calculator pull request #603 adds a Records.UNUSED_READ_VARS set which contains variables that are in the Records.VALID_READ_VARS set, but are not used in the tax calculations in any way.  These unused variables do not have an array created for them in the Records call object.  It would be more efficient, and less confusing to newcomers, if these unused variables were removed from the ```puf.csv``` file.\r\n\r\nThe removing of these variables is probably best done in the taxdata repository as a new step in the final-prep/puf-cps-processing.py script.\r\n\r\nThe size reduction in the ```puf.csv``` file would be substantial as shown by the following tabulations:\r\n```\r\n$ python\r\nPython 2.7.11 |Anaconda 2.3.0 (x86_64)| (default, Dec  6 2015, 18:57:58) \r\n[GCC 4.2.1 (Apple Inc. build 5577)] on darwin\r\n>>> from taxcalc import Records\r\n>>> print len(Records.VALID_READ_VARS)\r\n228\r\n>>> print len(Records.UNUSED_READ_VARS)\r\n121\r\n>>> quit()\r\n$ \r\n```\r\n\r\nSo, more than half of the variables in the ```puf.csv``` file are never used in the Tax-Calculator."},"comment":{"url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/comments/190386824","html_url":"https://github.com/open-source-economics/Tax-Calculator/issues/609#issuecomment-190386824","issue_url":"https://api.github.com/repos/open-source-economics/Tax-Calculator/issues/609","id":190386824,"user":{"login":"martinholmer","id":12170745,"avatar_url":"https://avatars.githubusercontent.com/u/12170745?v=3","gravatar_id":"","url":"https://api.github.com/users/martinholmer","html_url":"https://github.com/martinholmer","followers_url":"https://api.github.com/users/martinholmer/followers","following_url":"https://api.github.com/users/martinholmer/following{/other_user}","gists_url":"https://api.github.com/users/martinholmer/gists{/gist_id}","starred_url":"https://api.github.com/users/martinholmer/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martinholmer/subscriptions","organizations_url":"https://api.github.com/users/martinholmer/orgs","repos_url":"https://api.github.com/users/martinholmer/repos","events_url":"https://api.github.com/users/martinholmer/events{/privacy}","received_events_url":"https://api.github.com/users/martinholmer/received_events","type":"User","site_admin":false},"created_at":"2016-02-29T20:58:36Z","updated_at":"2016-02-29T20:58:36Z","body":"Amy said:\r\n> Martin, I haven't gone though the UNUSED_READ_VARS list thoroughly, but my first intuition is to keep those variables in ```puf.csv```, and have them accessible through Records class. My argument in general is that our users might want to see these variables for checking/testing purposes, or might even create tax logic with these currently unused variables. One example is variable ```filer```, which documents filers and non-filers tag for 2009. Some users might want to compare any later year non-filers with 09. If we removed this variable from ```puf.csv```, users could not even extract this variable through our Python interface, which will not be convenient for them.\r\n>\r\n> What do you think?\r\n\r\nWell, you (and Matt who made a similar point on the phone) are correct in that my proposal to remove the unused variables from ```puf.csv``` has pros and cons.  Perhaps there are a few variables that could be used in tax-logic enhancements (both you and Matt offered only one: ```filer```), but my impression is that most of the unused variables are just junk.  For example, why are ```s008``` and ```s009``` in the ```puf.csv``` file?  In what kind of situation would these variables be used?\r\n\r\nI think you and Matt are way underestimating the confusion factor with new users/contributors.  I'm an experienced user of the Python Tax-Calculator and I'm confused as to why **more than half** of the ```puf.csv``` variables are used nowhere in the tax-calculation logic.\r\n\r\nIf new users get sophisticated enough to want to program a tax reform that uses variables that are currently unused in Tax-Calculator, they will certainly be sophisticated enough to ask OSPC for an expanded version of ```puf.csv``` that includes the extra variables they need to program their reform.  And in that case, there will be no need to distribute that expanded version of ```puf.csv``` to anybody else unless this sophisticated user's new reform gets merged into the OSPC Git Hub master branch.  Remember your example assumes that this sophisticated user has paid IRS to use the 2009 SOI PUF, so they will know what is available in that dataset that is not in our pared-down ```puf.csv``` file.\r\n\r\nSo, on balance, I can see your point about including the ```filer``` variable, but I don't find that one case to be a compelling argument to keeping the other 120 unused variables in the ```puf.csv``` file.  Maybe I'm missing something?  Which of the other 120 unused variables do you think should be keep in the ```puf.csv``` file even though they are currently not being used?  And why?\r\n\r\n@Amy-Xu @MattHJensen @feenberg @GoFroggyRun \r\n"}}