{"action":"created","comment":{"url":"https://api.github.com/repos/awesomeWM/awesome/pulls/comments/57407709","id":57407709,"diff_hunk":"@@ -0,0 +1,243 @@\n+# Module is designed to allow the Awesome documentation examples to be tested.\n+#\n+# It shim enough of Awesome C API to allow code to be executed without an\n+# actual X server or running Awesome process. These test are not genuine\n+# integration tests, but they are the next best thing.\n+#\n+# As secondary goals, this module also generate images of the test result where\n+# relevant. Those images are used by the documentation and help developpers\n+# track user interface regressions and glitches. Finally, it also help to find\n+# broken code.\n+cmake_minimum_required(VERSION 3.0.0)\n+\n+# Get and update the LUA_PATH so the scripts can be executed without Awesome\n+execute_process(COMMAND lua -e print\\(package.path\\) OUTPUT_VARIABLE \"LUA_PATH_\")\n+\n+# Add the main awesome lau libraries\n+# set(${LUA_PATH_} \"${SOURCE_DIR}/lib/?;${SOURCE_DIR}/lib/?.lua;${LUA_PATH_}\")\n+set(LUA_PATH2_ \"\\\n+${CMAKE_SOURCE_DIR}/lib/?.lua;\\\n+${CMAKE_SOURCE_DIR}/lib/?/init.lua;\\\n+${CMAKE_SOURCE_DIR}/lib/?;\"\n+)\n+\n+# Add the C API shims\n+# set(${LUA_PATH_} \"${SOURCE_DIR}/lib/?;${SOURCE_DIR}/lib/?.lua;${LUA_PATH_}\")\n+set(LUA_PATH3_ \"\\","path":"tests/examples/CMakeLists.txt","position":26,"original_position":26,"commit_id":"6cf7233eaf8db30e82fd91f572bb23b0b4014ea1","original_commit_id":"6cf7233eaf8db30e82fd91f572bb23b0b4014ea1","user":{"login":"blueyed","id":9766,"avatar_url":"https://avatars.githubusercontent.com/u/9766?v=3","gravatar_id":"","url":"https://api.github.com/users/blueyed","html_url":"https://github.com/blueyed","followers_url":"https://api.github.com/users/blueyed/followers","following_url":"https://api.github.com/users/blueyed/following{/other_user}","gists_url":"https://api.github.com/users/blueyed/gists{/gist_id}","starred_url":"https://api.github.com/users/blueyed/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/blueyed/subscriptions","organizations_url":"https://api.github.com/users/blueyed/orgs","repos_url":"https://api.github.com/users/blueyed/repos","events_url":"https://api.github.com/users/blueyed/events{/privacy}","received_events_url":"https://api.github.com/users/blueyed/received_events","type":"User","site_admin":false},"body":"Same as above.","created_at":"2016-03-25T00:27:42Z","updated_at":"2016-03-25T00:27:42Z","html_url":"https://github.com/awesomeWM/awesome/pull/773#discussion_r57407709","pull_request_url":"https://api.github.com/repos/awesomeWM/awesome/pulls/773","_links":{"self":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/comments/57407709"},"html":{"href":"https://github.com/awesomeWM/awesome/pull/773#discussion_r57407709"},"pull_request":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/773"}}},"pull_request":{"url":"https://api.github.com/repos/awesomeWM/awesome/pulls/773","id":63847725,"html_url":"https://github.com/awesomeWM/awesome/pull/773","diff_url":"https://github.com/awesomeWM/awesome/pull/773.diff","patch_url":"https://github.com/awesomeWM/awesome/pull/773.patch","issue_url":"https://api.github.com/repos/awesomeWM/awesome/issues/773","number":773,"state":"open","locked":false,"title":"[WIP] tests: Add a new GUI testing framework","user":{"login":"Elv13","id":340384,"avatar_url":"https://avatars.githubusercontent.com/u/340384?v=3","gravatar_id":"","url":"https://api.github.com/users/Elv13","html_url":"https://github.com/Elv13","followers_url":"https://api.github.com/users/Elv13/followers","following_url":"https://api.github.com/users/Elv13/following{/other_user}","gists_url":"https://api.github.com/users/Elv13/gists{/gist_id}","starred_url":"https://api.github.com/users/Elv13/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Elv13/subscriptions","organizations_url":"https://api.github.com/users/Elv13/orgs","repos_url":"https://api.github.com/users/Elv13/repos","events_url":"https://api.github.com/users/Elv13/events{/privacy}","received_events_url":"https://api.github.com/users/Elv13/received_events","type":"User","site_admin":false},"body":"This framework allow widgets and user interface related code to be tested.\r\n\r\nThis should close the gap in our testing pipeline. It sits between unit testing and the full integration testing.\r\n\r\n#### It has the following features:\r\n\r\n * Generate documentation examples\r\n * Generate SVG screenshots of widgets\r\n * Generate dynamic ldoc content\r\n * Emulate just enough of the C API for the tests to run\r\n * Tests corner cases like weird multi screen that are harder to test with Xephyr\r\n * provide a template mechanism to generate different kind of images. Each module can have their own template or inherit from the upper module.\r\n\r\n#### Drawbacks:\r\n\r\n * With a thousand tests, the build get very slow, so should only be enabled on the CI. I still have to test parallel test generation. Given the bottleneck is mostly cmake and fork/pipe, I could technically generate a test list, then run 16 in parallel. With the 60 tests I already wrote, it add ~7 seconds to the build.\r\n * It is currently 1 test per file. This cause tons of files to be created. The reason behind this is to clean the state at every run. For example, tests can set random virtual screen setup, but I don't want to have to write code to rollback everything.\r\n\r\n#### WIP:\r\n\r\n * Some commits for svg generation are missing, I don't like the changes I had to do in `wibox.hierarchy`. I will rewrite them.\r\n * Integrate psychon coverall command from the `gears.shape` prototype.\r\n * Eventually run some regression testing on the SVG files. A simple vector path comparison should detect regressions/changes. A github bot could report them in the pull request with before and after images in the comment.\r\n\r\n** more commits will be added in the coming days**\r\n\r\n@blueyed Do you have any feature you want in this before I get too far?","created_at":"2016-03-23T08:44:17Z","updated_at":"2016-03-25T00:27:42Z","closed_at":null,"merged_at":null,"merge_commit_sha":"25d87050f8a7b4691949384d5698e97ef3dbf92d","assignee":null,"milestone":null,"commits_url":"https://api.github.com/repos/awesomeWM/awesome/pulls/773/commits","review_comments_url":"https://api.github.com/repos/awesomeWM/awesome/pulls/773/comments","review_comment_url":"https://api.github.com/repos/awesomeWM/awesome/pulls/comments{/number}","comments_url":"https://api.github.com/repos/awesomeWM/awesome/issues/773/comments","statuses_url":"https://api.github.com/repos/awesomeWM/awesome/statuses/6cf7233eaf8db30e82fd91f572bb23b0b4014ea1","head":{"label":"Elv13:add_testing_framework","ref":"add_testing_framework","sha":"6cf7233eaf8db30e82fd91f572bb23b0b4014ea1","user":{"login":"Elv13","id":340384,"avatar_url":"https://avatars.githubusercontent.com/u/340384?v=3","gravatar_id":"","url":"https://api.github.com/users/Elv13","html_url":"https://github.com/Elv13","followers_url":"https://api.github.com/users/Elv13/followers","following_url":"https://api.github.com/users/Elv13/following{/other_user}","gists_url":"https://api.github.com/users/Elv13/gists{/gist_id}","starred_url":"https://api.github.com/users/Elv13/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Elv13/subscriptions","organizations_url":"https://api.github.com/users/Elv13/orgs","repos_url":"https://api.github.com/users/Elv13/repos","events_url":"https://api.github.com/users/Elv13/events{/privacy}","received_events_url":"https://api.github.com/users/Elv13/received_events","type":"User","site_admin":false},"repo":{"id":17379398,"name":"awesome-1","full_name":"Elv13/awesome-1","owner":{"login":"Elv13","id":340384,"avatar_url":"https://avatars.githubusercontent.com/u/340384?v=3","gravatar_id":"","url":"https://api.github.com/users/Elv13","html_url":"https://github.com/Elv13","followers_url":"https://api.github.com/users/Elv13/followers","following_url":"https://api.github.com/users/Elv13/following{/other_user}","gists_url":"https://api.github.com/users/Elv13/gists{/gist_id}","starred_url":"https://api.github.com/users/Elv13/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Elv13/subscriptions","organizations_url":"https://api.github.com/users/Elv13/orgs","repos_url":"https://api.github.com/users/Elv13/repos","events_url":"https://api.github.com/users/Elv13/events{/privacy}","received_events_url":"https://api.github.com/users/Elv13/received_events","type":"User","site_admin":false},"private":false,"html_url":"https://github.com/Elv13/awesome-1","description":"Semi-official mirror of the awesome window manager from git://git.naquadah.org/awesome.git, updated hourly.","fork":true,"url":"https://api.github.com/repos/Elv13/awesome-1","forks_url":"https://api.github.com/repos/Elv13/awesome-1/forks","keys_url":"https://api.github.com/repos/Elv13/awesome-1/keys{/key_id}","collaborators_url":"https://api.github.com/repos/Elv13/awesome-1/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/Elv13/awesome-1/teams","hooks_url":"https://api.github.com/repos/Elv13/awesome-1/hooks","issue_events_url":"https://api.github.com/repos/Elv13/awesome-1/issues/events{/number}","events_url":"https://api.github.com/repos/Elv13/awesome-1/events","assignees_url":"https://api.github.com/repos/Elv13/awesome-1/assignees{/user}","branches_url":"https://api.github.com/repos/Elv13/awesome-1/branches{/branch}","tags_url":"https://api.github.com/repos/Elv13/awesome-1/tags","blobs_url":"https://api.github.com/repos/Elv13/awesome-1/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/Elv13/awesome-1/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/Elv13/awesome-1/git/refs{/sha}","trees_url":"https://api.github.com/repos/Elv13/awesome-1/git/trees{/sha}","statuses_url":"https://api.github.com/repos/Elv13/awesome-1/statuses/{sha}","languages_url":"https://api.github.com/repos/Elv13/awesome-1/languages","stargazers_url":"https://api.github.com/repos/Elv13/awesome-1/stargazers","contributors_url":"https://api.github.com/repos/Elv13/awesome-1/contributors","subscribers_url":"https://api.github.com/repos/Elv13/awesome-1/subscribers","subscription_url":"https://api.github.com/repos/Elv13/awesome-1/subscription","commits_url":"https://api.github.com/repos/Elv13/awesome-1/commits{/sha}","git_commits_url":"https://api.github.com/repos/Elv13/awesome-1/git/commits{/sha}","comments_url":"https://api.github.com/repos/Elv13/awesome-1/comments{/number}","issue_comment_url":"https://api.github.com/repos/Elv13/awesome-1/issues/comments{/number}","contents_url":"https://api.github.com/repos/Elv13/awesome-1/contents/{+path}","compare_url":"https://api.github.com/repos/Elv13/awesome-1/compare/{base}...{head}","merges_url":"https://api.github.com/repos/Elv13/awesome-1/merges","archive_url":"https://api.github.com/repos/Elv13/awesome-1/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/Elv13/awesome-1/downloads","issues_url":"https://api.github.com/repos/Elv13/awesome-1/issues{/number}","pulls_url":"https://api.github.com/repos/Elv13/awesome-1/pulls{/number}","milestones_url":"https://api.github.com/repos/Elv13/awesome-1/milestones{/number}","notifications_url":"https://api.github.com/repos/Elv13/awesome-1/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/Elv13/awesome-1/labels{/name}","releases_url":"https://api.github.com/repos/Elv13/awesome-1/releases{/id}","deployments_url":"https://api.github.com/repos/Elv13/awesome-1/deployments","created_at":"2014-03-03T20:41:58Z","updated_at":"2014-11-01T22:48:43Z","pushed_at":"2016-03-23T08:26:06Z","git_url":"git://github.com/Elv13/awesome-1.git","ssh_url":"git@github.com:Elv13/awesome-1.git","clone_url":"https://github.com/Elv13/awesome-1.git","svn_url":"https://github.com/Elv13/awesome-1","homepage":"https://awesome.naquadah.org/","size":10763,"stargazers_count":0,"watchers_count":0,"language":"C","has_issues":true,"has_downloads":true,"has_wiki":true,"has_pages":false,"forks_count":0,"mirror_url":null,"open_issues_count":9,"forks":0,"open_issues":9,"watchers":0,"default_branch":"master"}},"base":{"label":"awesomeWM:master","ref":"master","sha":"a40786215bcb38916d324c54179ff93379b9b023","user":{"login":"awesomeWM","id":6843364,"avatar_url":"https://avatars.githubusercontent.com/u/6843364?v=3","gravatar_id":"","url":"https://api.github.com/users/awesomeWM","html_url":"https://github.com/awesomeWM","followers_url":"https://api.github.com/users/awesomeWM/followers","following_url":"https://api.github.com/users/awesomeWM/following{/other_user}","gists_url":"https://api.github.com/users/awesomeWM/gists{/gist_id}","starred_url":"https://api.github.com/users/awesomeWM/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/awesomeWM/subscriptions","organizations_url":"https://api.github.com/users/awesomeWM/orgs","repos_url":"https://api.github.com/users/awesomeWM/repos","events_url":"https://api.github.com/users/awesomeWM/events{/privacy}","received_events_url":"https://api.github.com/users/awesomeWM/received_events","type":"Organization","site_admin":false},"repo":{"id":17378009,"name":"awesome","full_name":"awesomeWM/awesome","owner":{"login":"awesomeWM","id":6843364,"avatar_url":"https://avatars.githubusercontent.com/u/6843364?v=3","gravatar_id":"","url":"https://api.github.com/users/awesomeWM","html_url":"https://github.com/awesomeWM","followers_url":"https://api.github.com/users/awesomeWM/followers","following_url":"https://api.github.com/users/awesomeWM/following{/other_user}","gists_url":"https://api.github.com/users/awesomeWM/gists{/gist_id}","starred_url":"https://api.github.com/users/awesomeWM/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/awesomeWM/subscriptions","organizations_url":"https://api.github.com/users/awesomeWM/orgs","repos_url":"https://api.github.com/users/awesomeWM/repos","events_url":"https://api.github.com/users/awesomeWM/events{/privacy}","received_events_url":"https://api.github.com/users/awesomeWM/received_events","type":"Organization","site_admin":false},"private":false,"html_url":"https://github.com/awesomeWM/awesome","description":"awesome window manager","fork":false,"url":"https://api.github.com/repos/awesomeWM/awesome","forks_url":"https://api.github.com/repos/awesomeWM/awesome/forks","keys_url":"https://api.github.com/repos/awesomeWM/awesome/keys{/key_id}","collaborators_url":"https://api.github.com/repos/awesomeWM/awesome/collaborators{/collaborator}","teams_url":"https://api.github.com/repos/awesomeWM/awesome/teams","hooks_url":"https://api.github.com/repos/awesomeWM/awesome/hooks","issue_events_url":"https://api.github.com/repos/awesomeWM/awesome/issues/events{/number}","events_url":"https://api.github.com/repos/awesomeWM/awesome/events","assignees_url":"https://api.github.com/repos/awesomeWM/awesome/assignees{/user}","branches_url":"https://api.github.com/repos/awesomeWM/awesome/branches{/branch}","tags_url":"https://api.github.com/repos/awesomeWM/awesome/tags","blobs_url":"https://api.github.com/repos/awesomeWM/awesome/git/blobs{/sha}","git_tags_url":"https://api.github.com/repos/awesomeWM/awesome/git/tags{/sha}","git_refs_url":"https://api.github.com/repos/awesomeWM/awesome/git/refs{/sha}","trees_url":"https://api.github.com/repos/awesomeWM/awesome/git/trees{/sha}","statuses_url":"https://api.github.com/repos/awesomeWM/awesome/statuses/{sha}","languages_url":"https://api.github.com/repos/awesomeWM/awesome/languages","stargazers_url":"https://api.github.com/repos/awesomeWM/awesome/stargazers","contributors_url":"https://api.github.com/repos/awesomeWM/awesome/contributors","subscribers_url":"https://api.github.com/repos/awesomeWM/awesome/subscribers","subscription_url":"https://api.github.com/repos/awesomeWM/awesome/subscription","commits_url":"https://api.github.com/repos/awesomeWM/awesome/commits{/sha}","git_commits_url":"https://api.github.com/repos/awesomeWM/awesome/git/commits{/sha}","comments_url":"https://api.github.com/repos/awesomeWM/awesome/comments{/number}","issue_comment_url":"https://api.github.com/repos/awesomeWM/awesome/issues/comments{/number}","contents_url":"https://api.github.com/repos/awesomeWM/awesome/contents/{+path}","compare_url":"https://api.github.com/repos/awesomeWM/awesome/compare/{base}...{head}","merges_url":"https://api.github.com/repos/awesomeWM/awesome/merges","archive_url":"https://api.github.com/repos/awesomeWM/awesome/{archive_format}{/ref}","downloads_url":"https://api.github.com/repos/awesomeWM/awesome/downloads","issues_url":"https://api.github.com/repos/awesomeWM/awesome/issues{/number}","pulls_url":"https://api.github.com/repos/awesomeWM/awesome/pulls{/number}","milestones_url":"https://api.github.com/repos/awesomeWM/awesome/milestones{/number}","notifications_url":"https://api.github.com/repos/awesomeWM/awesome/notifications{?since,all,participating}","labels_url":"https://api.github.com/repos/awesomeWM/awesome/labels{/name}","releases_url":"https://api.github.com/repos/awesomeWM/awesome/releases{/id}","deployments_url":"https://api.github.com/repos/awesomeWM/awesome/deployments","created_at":"2014-03-03T19:56:16Z","updated_at":"2016-03-23T20:52:04Z","pushed_at":"2016-03-23T22:11:23Z","git_url":"git://github.com/awesomeWM/awesome.git","ssh_url":"git@github.com:awesomeWM/awesome.git","clone_url":"https://github.com/awesomeWM/awesome.git","svn_url":"https://github.com/awesomeWM/awesome","homepage":"https://awesomewm.org/","size":15083,"stargazers_count":695,"watchers_count":695,"language":"Lua","has_issues":true,"has_downloads":true,"has_wiki":false,"has_pages":false,"forks_count":104,"mirror_url":null,"open_issues_count":102,"forks":104,"open_issues":102,"watchers":695,"default_branch":"master"}},"_links":{"self":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/773"},"html":{"href":"https://github.com/awesomeWM/awesome/pull/773"},"issue":{"href":"https://api.github.com/repos/awesomeWM/awesome/issues/773"},"comments":{"href":"https://api.github.com/repos/awesomeWM/awesome/issues/773/comments"},"review_comments":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/773/comments"},"review_comment":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/comments{/number}"},"commits":{"href":"https://api.github.com/repos/awesomeWM/awesome/pulls/773/commits"},"statuses":{"href":"https://api.github.com/repos/awesomeWM/awesome/statuses/6cf7233eaf8db30e82fd91f572bb23b0b4014ea1"}}}}