{"action":"created","issue":{"url":"https://api.github.com/repos/opencv/opencv/issues/6933","repository_url":"https://api.github.com/repos/opencv/opencv","labels_url":"https://api.github.com/repos/opencv/opencv/issues/6933/labels{/name}","comments_url":"https://api.github.com/repos/opencv/opencv/issues/6933/comments","events_url":"https://api.github.com/repos/opencv/opencv/issues/6933/events","html_url":"https://github.com/opencv/opencv/pull/6933","id":165980819,"number":6933,"title":"[GSOC] New camera model for stitching pipeline","user":{"login":"hrnr","id":1316311,"avatar_url":"https://avatars.githubusercontent.com/u/1316311?v=3","gravatar_id":"","url":"https://api.github.com/users/hrnr","html_url":"https://github.com/hrnr","followers_url":"https://api.github.com/users/hrnr/followers","following_url":"https://api.github.com/users/hrnr/following{/other_user}","gists_url":"https://api.github.com/users/hrnr/gists{/gist_id}","starred_url":"https://api.github.com/users/hrnr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hrnr/subscriptions","organizations_url":"https://api.github.com/users/hrnr/orgs","repos_url":"https://api.github.com/users/hrnr/repos","events_url":"https://api.github.com/users/hrnr/events{/privacy}","received_events_url":"https://api.github.com/users/hrnr/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/opencv/opencv/labels/GSoC","name":"GSoC","color":"fbca04"}],"state":"open","locked":false,"assignee":{"login":"alalek","id":4981219,"avatar_url":"https://avatars.githubusercontent.com/u/4981219?v=3","gravatar_id":"","url":"https://api.github.com/users/alalek","html_url":"https://github.com/alalek","followers_url":"https://api.github.com/users/alalek/followers","following_url":"https://api.github.com/users/alalek/following{/other_user}","gists_url":"https://api.github.com/users/alalek/gists{/gist_id}","starred_url":"https://api.github.com/users/alalek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alalek/subscriptions","organizations_url":"https://api.github.com/users/alalek/orgs","repos_url":"https://api.github.com/users/alalek/repos","events_url":"https://api.github.com/users/alalek/events{/privacy}","received_events_url":"https://api.github.com/users/alalek/received_events","type":"User","site_admin":false},"assignees":[{"login":"alalek","id":4981219,"avatar_url":"https://avatars.githubusercontent.com/u/4981219?v=3","gravatar_id":"","url":"https://api.github.com/users/alalek","html_url":"https://github.com/alalek","followers_url":"https://api.github.com/users/alalek/followers","following_url":"https://api.github.com/users/alalek/following{/other_user}","gists_url":"https://api.github.com/users/alalek/gists{/gist_id}","starred_url":"https://api.github.com/users/alalek/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alalek/subscriptions","organizations_url":"https://api.github.com/users/alalek/orgs","repos_url":"https://api.github.com/users/alalek/repos","events_url":"https://api.github.com/users/alalek/events{/privacy}","received_events_url":"https://api.github.com/users/alalek/received_events","type":"User","site_admin":false}],"milestone":null,"comments":49,"created_at":"2016-07-17T15:15:22Z","updated_at":"2016-10-21T16:28:36Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/opencv/opencv/pulls/6933","html_url":"https://github.com/opencv/opencv/pull/6933","diff_url":"https://github.com/opencv/opencv/pull/6933.diff","patch_url":"https://github.com/opencv/opencv/pull/6933.patch"},"body":"**Merge with extra**: https://github.com/opencv/opencv_extra/pull/303\r\n\r\nThis PR contains all work for New camera model for stitching pipeline GSoC 2016  project.\r\n\r\n# GSoC Proposal\r\n\r\nStitching pipeline is a well established code in OpenCV. It provides good results for creating panoramas from camera captured images. Main limitation of stitching pipeline is its expected camera model (perspective transformation). Although this model is fine for many applications working with camera captured images, there are applications which aren't covered by current stitching pipeline.\r\n\r\n## New camera model\r\nDue to physical constraints it is possible for some applications to expect much simpler transform with less degrees of freedom. Those are situations when input data are not subject to perspective transform. The transformation can be much simpler, such as affine transformation. Datasets considered here includes images captured by special hardware (such as book scanners[0] that tries hard to eliminate perspective), maps from laser scanning (produced from different starting points), preprocessed images (where perspective was compensated by other robust means, taking advantage of physical situation, e.g. for book scanners we would use data from calibration to compensate remaining perspective). In all those situations we would like to obtain image mosaic under affine transformation.\r\n\r\nI'd like to introduce new camera model based on affine transformation to stitching pipeline. This would include:\r\n\r\n* New Matcher using affine transformation (cv::estimateRigidTransform) to estimate H\r\n* New Estimator aware of affine model.\r\n* Defining and documenting  this new model for CameraParams (e.g. now translation is always expected to be zero, this might not be true for affine transformation)\r\n* Integration works in compositing part of pipeline (there might be changes necessary depending how we would decide to represent affine model in CameraParams)\r\n* New options for high-level API to be able to use affine model instead of current one simply\r\n* Producing new sample code for stitching pipeline\r\n* Improving current documentation (current documentation does not mention details about current camera model, this might need some clarification)\r\n\r\nI used approach based on affine transformation to merge maps produced by multiple robots [1] for my robotics project. It shows a good results. However, as mentioned earlier applications for this model are much broader than that.\r\n\r\n## Parallelism for FeaturesFinder\r\n\r\nTo make usage of stitching pipeline more comfortable and performant for large number of images, I’d like also to improve FeaturesFinder to allow finding features in parallel. All camera models and other users of FeaturesFinder may take benefit from that. The API could be similar to `FeaturesMatcher::operator ()(features, pairwise_matches, mask)`.\r\n\r\nThis could be with TBB in similar manner as mentioned method in FeaturesMatcher, which is already being used in stitching pipeline so there would be almost no additional overhead in starting new threads in typical scenarios, because these threads are there already for FeaturesMatcher. This change would be fully integrated into high level stitching interface.\r\n\r\nThere might be some changes necessary in finders to ensure thread-safety. Where thread-safety can’t be ensured or it does not make sense (GPU finders), parallelization would be disabled and all images would be processed in serial manner so this method would be always safe to use regardless of underlying finder. This approach is also similar to FeaturesMatcher.\r\n\r\n## Benefits to OpenCV\r\n* New transform options for stitching pipeline\r\n* Performance improvements through parallel processing of image features\r\n\r\n# implemented goals (all + extras)\r\n## new camera model\r\n- [x] affine matcher\r\n- [x] affine estimator \r\n- [x] affine warper\r\n- [x] affine bundle adjusters\r\n- tests for affine stitching\r\n     - [x] basic affine stitching integration test\r\n     - [x] tests for affine matcher\r\n     - [x] integration tests on real-word scans\r\n     - [x] affine warper tests\r\n     - [x] affine bundle adjusters tests\r\n- [x] integrating with high level API (Stitcher)\r\n- [x] stitching_detailed sample\r\n- [x] stitching simple sample\r\n\r\n## parallel feature finding\r\n- [x] parallel API in feature finder\r\n- [x] tests (incl. perf tests) \r\n- [x] integrating with Stitcher\r\n\r\n## implemented extras\r\n- [x]  robust fuctions for affine transform estimations\r\n    - [x] add support for least median robust method\r\n    - [x] tests for LMEDS\r\n    - [x] Levenberg–Marquardt algorithm-based refining for affine estimation functions\r\n    - [x] tests and docs\r\n    - [x] perf tests\r\n- [x] stitching tutorial for high level API\r\n    - [x] add examples of running the samples on testdata in opencv_extra\r\n- [x] fix existing stitching tests with SURF (SURF was disabled)\r\n\r\n# video\r\n[![short video presenting this project](https://img.youtube.com/vi/2xDxd-_6Do8/0.jpg)](https://www.youtube.com/watch?v=2xDxd-_6Do8)\r\n\r\n# other work\r\n\r\nDuring this GSoC I have also coded some related work, that is not going to be included (mostly because we has chosen different approach or the work has been merged under this PR). It is listed here for completeness. \r\n\r\nPRs:\r\n* #6560\r\n* #6609 \r\n* #6615 \r\n* #6642 \r\n\r\ncommits:\r\n* eba30a89737d4ded755f07cff75fd861864cf09a\r\n* 150daa2dc57a258ba61a01e12901518b6b4d98e8"},"comment":{"url":"https://api.github.com/repos/opencv/opencv/issues/comments/255421446","html_url":"https://github.com/opencv/opencv/pull/6933#issuecomment-255421446","issue_url":"https://api.github.com/repos/opencv/opencv/issues/6933","id":255421446,"user":{"login":"hrnr","id":1316311,"avatar_url":"https://avatars.githubusercontent.com/u/1316311?v=3","gravatar_id":"","url":"https://api.github.com/users/hrnr","html_url":"https://github.com/hrnr","followers_url":"https://api.github.com/users/hrnr/followers","following_url":"https://api.github.com/users/hrnr/following{/other_user}","gists_url":"https://api.github.com/users/hrnr/gists{/gist_id}","starred_url":"https://api.github.com/users/hrnr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hrnr/subscriptions","organizations_url":"https://api.github.com/users/hrnr/orgs","repos_url":"https://api.github.com/users/hrnr/repos","events_url":"https://api.github.com/users/hrnr/events{/privacy}","received_events_url":"https://api.github.com/users/hrnr/received_events","type":"User","site_admin":false},"created_at":"2016-10-21T16:28:36Z","updated_at":"2016-10-21T16:28:36Z","body":"rebased again."}}
{"actor":{"display_login":"hrnr"}}