{"action":"created","issue":{"url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/2772","repository_url":"https://api.github.com/repos/NativeScript/nativescript-cli","labels_url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/2772/labels{/name}","comments_url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/2772/comments","events_url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/2772/events","html_url":"https://github.com/NativeScript/nativescript-cli/issues/2772","id":226267903,"node_id":"MDU6SXNzdWUyMjYyNjc5MDM=","number":2772,"title":"tns test <platform> include coverage?","user":{"login":"dtopuzov","id":6651651,"node_id":"MDQ6VXNlcjY2NTE2NTE=","avatar_url":"https://avatars3.githubusercontent.com/u/6651651?v=4","gravatar_id":"","url":"https://api.github.com/users/dtopuzov","html_url":"https://github.com/dtopuzov","followers_url":"https://api.github.com/users/dtopuzov/followers","following_url":"https://api.github.com/users/dtopuzov/following{/other_user}","gists_url":"https://api.github.com/users/dtopuzov/gists{/gist_id}","starred_url":"https://api.github.com/users/dtopuzov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dtopuzov/subscriptions","organizations_url":"https://api.github.com/users/dtopuzov/orgs","repos_url":"https://api.github.com/users/dtopuzov/repos","events_url":"https://api.github.com/users/dtopuzov/events{/privacy}","received_events_url":"https://api.github.com/users/dtopuzov/received_events","type":"User","site_admin":false},"labels":[{"id":110536377,"node_id":"MDU6TGFiZWwxMTA1MzYzNzc=","url":"https://api.github.com/repos/NativeScript/nativescript-cli/labels/feature","name":"feature","color":"0B61A4","default":false},{"id":287416374,"node_id":"MDU6TGFiZWwyODc0MTYzNzQ=","url":"https://api.github.com/repos/NativeScript/nativescript-cli/labels/unit%20testing","name":"unit testing","color":"FF9540","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":11,"created_at":"2017-05-04T12:41:12Z","updated_at":"2018-10-01T21:05:19Z","closed_at":null,"author_association":"MEMBER","body":"_From @NickIliev on June 20, 2016 11:31_\n\n_From @NickIliev on June 20, 2016 10:56_\n\n_From @georgeedwards on June 17, 2016 12:46_\n## Feature Request:\n\nCan the `tns test <platform>` command include coverage reporting as part of it's output? [Karma-Coverage](https://github.com/karma-runner/karma-coverage) seems to be quite a nice way of doing this. To add support for Typescript, adding in [remap-istanbul](https://github.com/SitePen/remap-istanbul) looks to be the most straight forward solution. \n\n_Copied from original issue: NativeScript/NativeScript#2333_\n\n_Copied from original issue: NativeScript/nativescript-cli#1865_\n\n\n_Copied from original issue: NativeScript/nativescript-unit-test-runner#12_"},"comment":{"url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/comments/426062669","html_url":"https://github.com/NativeScript/nativescript-cli/issues/2772#issuecomment-426062669","issue_url":"https://api.github.com/repos/NativeScript/nativescript-cli/issues/2772","id":426062669,"node_id":"MDEyOklzc3VlQ29tbWVudDQyNjA2MjY2OQ==","user":{"login":"rosen-vladimirov","id":8351653,"node_id":"MDQ6VXNlcjgzNTE2NTM=","avatar_url":"https://avatars1.githubusercontent.com/u/8351653?v=4","gravatar_id":"","url":"https://api.github.com/users/rosen-vladimirov","html_url":"https://github.com/rosen-vladimirov","followers_url":"https://api.github.com/users/rosen-vladimirov/followers","following_url":"https://api.github.com/users/rosen-vladimirov/following{/other_user}","gists_url":"https://api.github.com/users/rosen-vladimirov/gists{/gist_id}","starred_url":"https://api.github.com/users/rosen-vladimirov/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rosen-vladimirov/subscriptions","organizations_url":"https://api.github.com/users/rosen-vladimirov/orgs","repos_url":"https://api.github.com/users/rosen-vladimirov/repos","events_url":"https://api.github.com/users/rosen-vladimirov/events{/privacy}","received_events_url":"https://api.github.com/users/rosen-vladimirov/received_events","type":"User","site_admin":false},"created_at":"2018-10-01T21:05:19Z","updated_at":"2018-10-01T21:05:19Z","author_association":"CONTRIBUTOR","body":"Hey all,\r\n\r\nIt's been some time we've not worked on our unit testing story. Most of you, who have tried it, know that it has some issues, but still it can be used to test your application.\r\nCurrently we are working on some other tasks and we may not be able to implement this feature in the next few months. However, we have created a POC that you can use and even improve it and send us a PR.\r\nBefore deep diving in the details described below, please read carefully [this article](https://docs.nativescript.org/tooling/testing) and especially the [Run the tests](https://docs.nativescript.org/tooling/testing#run-the-tests) part.\r\nIt describes what happens when `tns test` command is executed.\r\n\r\n### Karma coverage - enabling the feature\r\nSo back to the coverage feature. \r\n> NOTE: Information below is for a project that already has unit tests, i.e. you have executed `tns create myApp && cd myApp && tns test init`.\r\n\r\nKarma allows you to have coverage information by adding the karma-coverage plugin as devDependency of your project: https://karma-runner.github.io/0.8/config/coverage.html\r\nSo you have to install it with:\r\n```\r\nnpm i --save-dev karma-coverage\r\n```\r\n\r\nAs described in the article, you need to add some code in your karma.config.js. The following configuration should work for your NativeScript application.\r\n```JavaScript\r\nfiles: [\r\n  'app/**/*.js'\r\n],\r\n\r\npreprocessors = {\r\n  'app/**/*.js': ['coverage']\r\n},\r\n\r\nreporters = ['coverage'],\r\n\r\ncoverageReporter: {\r\n  type: 'html',\r\n  dir: 'coverage/',\r\n  includeAllSources: true\r\n},\r\n```\r\n\r\nAt this point you have everything setup, so you can try `tns test <platform> --justlaunch`. (`--justlaunch` will ensure the command exits after the execution of unit test finishes).\r\nAfter that you can check the `<project dir>/coverage` directory. You'll notice that there is some information there, but the generated coverage report is not correct.\r\n\r\n### Discover the issue or why the code coverage is not correct\r\nThe actual issue here is with the execution of the tests. In the previous point we have added a preprocessor - `coverage`. What does it do? Well, it modifies the files that will be served by Karma in a specific way. This specific way in our case is adding of lines inside all files that are used by the karma-coverage to detect how many times each line is called. This is done through `istanbul` package. It is important to notice that the files are modified in-memory, their content is not written anywhere on your hard disk.\r\nSo, we know Karma is working fine and the files are preprocessed, but why the coverage information is not correct on our side? Let's get back to the way NativeScript CLI executes the unit tests.\r\nWhen `tns test <platform>` command is executed CLI does the following:\r\n1. The CLI starts a Karma server on the development machine.\r\n2. The CLI prepares, builds and deploys your project, if not already deployed. If already deployed, the CLI synchronizes changes to the application package.\r\n3. The CLI embeds the NativeScript unit test runner and your host network and Karma configuration in the deployed package.\r\n4. The CLI launches the main module of the NativeScript unit test runner instead of launching the main module of your app.\r\n5. The NativeScript unit test runner uses the embedded network configuration to try to connect to the Karma server on the development machine.\r\n6. When the connection between the NativeScript unit test runner and the Karma server is established, the test runner begins the execution of the unit tests.\r\n7. When the execution completes, the NativeScript unit test runner reports the results to the Karma server.\r\n8. The Karma server reports the results on the command line.\r\n\r\nThere's some additional things that happen on step 6. As you can see [here (this is the code that is actually running on the device)](https://github.com/NativeScript/nativescript-unit-test-runner/blob/7ba5fd4de5fe4a1fdbdea473496dd1ed64cca806/main-view-model.ts#L206-L246) once a connection is established (i.e. after NativeScript CLI had already uploaded the application and all of the files on device), the application on device gets a file called `context.json` from the Karma server.\r\nThis file contains information about the current test run including which files should be included in it. These are all the files that match our `files` array in the `karma.config.js` and some specific files required for testing, like `mocha` and `chai` files in case you've selected to use `mocha` framework for testing. As you can see, the application gets the list of files and executes some logic for each of them. In case the file is part of the application, the application does nothing. \r\nBut in case this is a file from \"node_modules\" (as the paths that are written in the `context.json` are from the local execution of the Karma and they contain `node_modules`, which does not exist on device), the application calls the Karma server to get the content of the file. \r\nGetting back to the way CLI and Karma work, we start to see what might be the issue - CLI had uploaded the original files of the application. Meanwhile the Karma server had preprocessed the local files (the same ones that CLI had already uploaded), but keeps them in memory. Once the application on device starts executing the tests, it does not call Karma server to get the contents of the project files, it calls it only for the files that are not already uploaded by CLI. So the preprocessed files are never taken to the device and so the coverage information cannot be correct.\r\n\r\n### Fixing the coverage issue\r\nSo how to fix this behavior. The easiest way we've found is to overwrite the project files with the content that Karma server will give us. This will allow us to include other preprocessores in case we need. And here's a branch that implements exactly this logic (this is our POC) of this feature:\r\nhttps://github.com/NativeScript/nativescript-unit-test-runner/tree/vladimirov/coverage-poc (check the latest commit)\r\nThis way the files on the device will be overwritten with the preprocessed content and the coverage information that Karma will create inside the local project will be correct.\r\n\r\n### (Potential) Issues with the suggested implementation\r\n1. Using platform specific files will not work - the Karma server will use the files from the local `app` directory, which may have `main-view-model.android.js` and `main-view-model.ios.js` instead of just a single file. Once the application on device starts, it will receive the list of files, including the two platform specific files.\r\nAfter that it will ask Karma to get the content of these two files and will write them in the application with the same names - `main-view-model.android.js` and `main-view-model.ios.js`. Neither file will require these files, as CLI had already uploaded a file called `main-view-model.js`. There are various ways to fix this behavior, for example:\r\n\t- implement a logic once the application gets the file list from Karma and parse the file names based on the platform of the device (you can use [isAndroid/isIOS methods](https://docs.nativescript.org/api-reference/modules/_platform_#isandroid). After that skip the files that are not for this platform and replace the name of the files that are for current platform, i.e. filter the `main-view-model.ios.js` and set the name of the `main-view-model.android.js` to `main-view-model.js`.\r\n\r\n2. Overwritten files may remain in the application after executing `tns test <platform>` and `tns run <platform>` after that as CLI may not transfer all files. Workaround: uninstall the app from device.\r\n\r\n### Notes\r\nThe `nativescript-unit-test-runner` is written in TypeScript, so when you apply changes in it, you have to execute the following steps to get them transpiled:\r\n```\r\ngit clone https://github.com/NativeScript/nativescript-unit-test-runner.git\r\ncd nativescript-unit-test-runner\r\ngit checkout vladimirov/coverage-poc\r\nnpm i\r\n# apply your change in the code\r\n\r\ngrunt pack \r\n```\r\nNOTE: The `grunt pack` command will produce some warnings, but don't worry about them, it will generate a .tgz file that you can install in your application instead of the official version of `nativescript-unit-test-runner`.\r\n\r\n\r\nSo that's it. You should be able to get the coverage information you need. In case anyone is interested in finishing this POC, just drop a line. We'll try to help you with the implementation and any issues you face.\r\n"}}
{"actor":{"display_login":"rosen-vladimirov"}}