{"action":"created","issue":{"url":"https://api.github.com/repos/martanne/vis/issues/166","repository_url":"https://api.github.com/repos/martanne/vis","labels_url":"https://api.github.com/repos/martanne/vis/issues/166/labels{/name}","comments_url":"https://api.github.com/repos/martanne/vis/issues/166/comments","events_url":"https://api.github.com/repos/martanne/vis/issues/166/events","html_url":"https://github.com/martanne/vis/issues/166","id":133528993,"number":166,"title":"Improve Regression Test Suite","user":{"login":"martanne","id":5736493,"avatar_url":"https://avatars.githubusercontent.com/u/5736493?v=3","gravatar_id":"","url":"https://api.github.com/users/martanne","html_url":"https://github.com/martanne","followers_url":"https://api.github.com/users/martanne/followers","following_url":"https://api.github.com/users/martanne/following{/other_user}","gists_url":"https://api.github.com/users/martanne/gists{/gist_id}","starred_url":"https://api.github.com/users/martanne/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/martanne/subscriptions","organizations_url":"https://api.github.com/users/martanne/orgs","repos_url":"https://api.github.com/users/martanne/repos","events_url":"https://api.github.com/users/martanne/events{/privacy}","received_events_url":"https://api.github.com/users/martanne/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/martanne/vis/labels/help%20wanted","name":"help wanted","color":"159818"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":10,"created_at":"2016-02-14T10:34:59Z","updated_at":"2016-02-28T16:46:54Z","closed_at":null,"body":"For some time there exists basic testing infrastructure on a [separate branch](https://github.com/martanne/vis/tree/test/test). Unfortunately it currently lacks proper test cases.\r\n\r\nAs mentioned [elsewhere](https://github.com/martanne/vis/issues/78#issuecomment-183460684) having a proper test suite would be extremely helpful to guarantee stability and make sure fixed bugs are not reintroduced. Ideally every bugfix / newly introduced feature would have an accompanying test suite entry. The test suite should at all times pass without failure.\r\n\r\nAlso working on the test suite is a good way to contribute to the project without necessarily having to delve into the C code base.\r\n\r\nThere currently exists 3 different kind of tests (each with a README in the corresponding sub directory):\r\n\r\n1. [test/vim](https://github.com/martanne/vis/tree/test/test/vim) these are simple black box tests, comparing vim compatibility. They are completely independent of vis and could possibly be reused by other vim-like editors. The idea is to start the editor with a given file, feed it keyboard input as would normally be done by a human and compare the resulting file content to the one produced by vim.\r\n\r\n2. [test/vis](https://github.com/martanne/vis/tree/test/test/vis) these are similar to 1) but test vis features which are either not present in vim core (e.g. multiple cursors) or have deliberately different behavior. Instead of comparing the final file content to the one produced by vim it is compared to a fixed expected result file.\r\n\r\n3. [test/core](https://github.com/martanne/vis/tree/test/test/core) these are C unit tests for core data structures used in vis. Writing tests for those is a good way to become familiar with the core APIs of vis.\r\n\r\nFor types 1 and 2 a good way to start would be to go through all operators / motions and make sure they:\r\n\r\n* work when given a count\r\n* are repeatable with `.` as expected\r\n* (apply to all cursors, only for type 2)\r\n\r\nThis approach would ensure that tests are nicely categorized by functionality.\r\n\r\nAnother good starting point would be to go through he commit log and past closed issues (like #160 #152 #149 #144 #141 #131 #123 #118 #113 #92 #91 #89 #45 #44) and add test cases for those.\r\n\r\nAdding some vimgolf entries (not necessarily their solution) might also be an option, but remember the goal is to test existing editor functionality."},"comment":{"url":"https://api.github.com/repos/martanne/vis/issues/comments/189900736","html_url":"https://github.com/martanne/vis/issues/166#issuecomment-189900736","issue_url":"https://api.github.com/repos/martanne/vis/issues/166","id":189900736,"user":{"login":"erf","id":1562523,"avatar_url":"https://avatars.githubusercontent.com/u/1562523?v=3","gravatar_id":"","url":"https://api.github.com/users/erf","html_url":"https://github.com/erf","followers_url":"https://api.github.com/users/erf/followers","following_url":"https://api.github.com/users/erf/following{/other_user}","gists_url":"https://api.github.com/users/erf/gists{/gist_id}","starred_url":"https://api.github.com/users/erf/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/erf/subscriptions","organizations_url":"https://api.github.com/users/erf/orgs","repos_url":"https://api.github.com/users/erf/repos","events_url":"https://api.github.com/users/erf/events{/privacy}","received_events_url":"https://api.github.com/users/erf/received_events","type":"User","site_admin":false},"created_at":"2016-02-28T16:46:54Z","updated_at":"2016-02-28T16:46:54Z","body":"It compiles OK now, but **test 2** fails since `/etc/shadow` does not exist."}}