{"action":"created","issue":{"url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/202","repository_url":"https://api.github.com/repos/nyanp/tiny-cnn","labels_url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/202/labels{/name}","comments_url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/202/comments","events_url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/202/events","html_url":"https://github.com/nyanp/tiny-cnn/issues/202","id":161667724,"number":202,"title":"Quantization method for conv, deconv and fc layers.","user":{"login":"Wangyida","id":6743242,"avatar_url":"https://avatars.githubusercontent.com/u/6743242?v=3","gravatar_id":"","url":"https://api.github.com/users/Wangyida","html_url":"https://github.com/Wangyida","followers_url":"https://api.github.com/users/Wangyida/followers","following_url":"https://api.github.com/users/Wangyida/following{/other_user}","gists_url":"https://api.github.com/users/Wangyida/gists{/gist_id}","starred_url":"https://api.github.com/users/Wangyida/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Wangyida/subscriptions","organizations_url":"https://api.github.com/users/Wangyida/orgs","repos_url":"https://api.github.com/users/Wangyida/repos","events_url":"https://api.github.com/users/Wangyida/events{/privacy}","received_events_url":"https://api.github.com/users/Wangyida/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/nyanp/tiny-cnn/labels/enhancement","name":"enhancement","color":"84b6eb"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":32,"created_at":"2016-06-22T12:23:42Z","updated_at":"2016-07-04T16:40:30Z","closed_at":null,"body":"## Quantization Method for conv, deconv and fc Layers.\r\nHere I want to implement the quanzization on operation in conv, deconv and fc layers. Much quantization method are included in this paper: Ristretto: [Hardware-Oriented Approximation of Convolutional Neural](https://arxiv.org/pdf/1605.06402v1.pdf)\r\n\r\n### Applicable Method\r\n@nyanp Methods are mainly about 3 aspact:\r\n\r\n1. **Network structure modification**: \r\n [SqueezeNet](http://arxiv.org/pdf/1602.07360v3.pdf), [ENet](https://arxiv.org/pdf/1606.02147v1.pdf)\r\n2. **Binarization**: [XNOR-Net](http://arxiv.org/pdf/1603.05279v2.pdf), [Binary Connect](http://arxiv.org/pdf/1511.00363v3.pdf)\r\n3. **Precision Adjustment**: [Ristretto](http://arxiv.org/pdf/1606.01981v1.pdf)\r\n\r\n### For tiny_cnn Implementaion\r\nSo there I propose some method for further implementation:\r\n\r\n1. To utilize methods about network modification method, Inception concept could solve parameter expanding from large kernel scale (7 * 7, 9 * 9 and 11 * 11) and large amount of channels (AlexNet), This is related to the graph implementation discussed in #201 @edgarriba already give me some advises on the graph style.\r\n2. This will bring an obvious accuracy drop, so I might not going to apply it.\r\n3. @bhack This is the **main task** for me: Quantization to Fixed Point. I think it is better to implement a dynamic fix point data to replace float data, this could be done separately in conv, deconv and fc layers for the kernel file containing forward and backward codes.  It is desirable to group fixed point numbers into groups with constant  the fractional length,  so the number of bits allocated to the fractional part is constant within that group, but different compared to other groups. The fractional length learned to cover\r\na wide dynamic range of output and weight in particular layers. So there will be another worker storage or output channel to contain the fractional length."},"comment":{"url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/comments/230325439","html_url":"https://github.com/nyanp/tiny-cnn/issues/202#issuecomment-230325439","issue_url":"https://api.github.com/repos/nyanp/tiny-cnn/issues/202","id":230325439,"user":{"login":"bhack","id":1710528,"avatar_url":"https://avatars.githubusercontent.com/u/1710528?v=3","gravatar_id":"","url":"https://api.github.com/users/bhack","html_url":"https://github.com/bhack","followers_url":"https://api.github.com/users/bhack/followers","following_url":"https://api.github.com/users/bhack/following{/other_user}","gists_url":"https://api.github.com/users/bhack/gists{/gist_id}","starred_url":"https://api.github.com/users/bhack/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bhack/subscriptions","organizations_url":"https://api.github.com/users/bhack/orgs","repos_url":"https://api.github.com/users/bhack/repos","events_url":"https://api.github.com/users/bhack/events{/privacy}","received_events_url":"https://api.github.com/users/bhack/received_events","type":"User","site_admin":false},"created_at":"2016-07-04T16:40:30Z","updated_at":"2016-07-04T16:40:30Z","body":"@Maratyszcza Neon kernels are interesting for lowpower target (embedded/mobile) and gemmlowp has more complete coverage on Neon. NNpack it is still FMA-AVX2 only."}}
{"actor":{"display_login":"bhack"}}