{"action":"created","issue":{"url":"https://api.github.com/repos/RobotLocomotion/drake/issues/2619","repository_url":"https://api.github.com/repos/RobotLocomotion/drake","labels_url":"https://api.github.com/repos/RobotLocomotion/drake/issues/2619/labels{/name}","comments_url":"https://api.github.com/repos/RobotLocomotion/drake/issues/2619/comments","events_url":"https://api.github.com/repos/RobotLocomotion/drake/issues/2619/events","html_url":"https://github.com/RobotLocomotion/drake/issues/2619","id":161764927,"number":2619,"title":"Consider chunking for automatic differentiation","user":{"login":"tkoolen","id":2585880,"avatar_url":"https://avatars.githubusercontent.com/u/2585880?v=3","gravatar_id":"","url":"https://api.github.com/users/tkoolen","html_url":"https://github.com/tkoolen","followers_url":"https://api.github.com/users/tkoolen/followers","following_url":"https://api.github.com/users/tkoolen/following{/other_user}","gists_url":"https://api.github.com/users/tkoolen/gists{/gist_id}","starred_url":"https://api.github.com/users/tkoolen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tkoolen/subscriptions","organizations_url":"https://api.github.com/users/tkoolen/orgs","repos_url":"https://api.github.com/users/tkoolen/repos","events_url":"https://api.github.com/users/tkoolen/events{/privacy}","received_events_url":"https://api.github.com/users/tkoolen/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/RobotLocomotion/drake/labels/type:%20idea","name":"type: idea","color":"0e8a16"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":4,"created_at":"2016-06-22T19:23:38Z","updated_at":"2016-06-23T00:07:20Z","closed_at":null,"body":"I just saw a talk at JuliaCon (held in our building) about ForwardDiff.jl, an automatic differentiation package for Julia. It uses the same idea as Eigen's AutoDiffScalar.\r\n\r\nFirst off, his presentation was excellent and explained the concept very well. It also compares against other differentiation methods, including complex step. See here:\r\nhttps://github.com/jrevels/ForwardDiffPresentation/blob/master/slides.pdf\r\n\r\nMore importantly for this issue, the speaker mentioned the use of 'chunking'. If you're taking derivatives w.r.t. many variables, the 'AutoDiffScalar's will be large objects in memory. After a certain number of derivatives, this can cause what he called 'stack thrashing'. I'm not sure if that's the correct term, but I'm interpreting it as e.g. running out of L1 cache in the middle of your algorithm (although I'm not an expert on this subject).\r\n\r\nIn any case, he said that it can be beneficial to run your algorithm a bunch of times with smaller AutoDiffScalars (10 derivatives at a time or so), than to run your algorithm once with a huge derivative vector.\r\n\r\nUsing this chunking idea (if it can be made transparent to the user) can also make it easier to find an answer to the question of which finite set of AutoDiffScalar types to support."},"comment":{"url":"https://api.github.com/repos/RobotLocomotion/drake/issues/comments/227913661","html_url":"https://github.com/RobotLocomotion/drake/issues/2619#issuecomment-227913661","issue_url":"https://api.github.com/repos/RobotLocomotion/drake/issues/2619","id":227913661,"user":{"login":"liangfok","id":1388098,"avatar_url":"https://avatars.githubusercontent.com/u/1388098?v=3","gravatar_id":"","url":"https://api.github.com/users/liangfok","html_url":"https://github.com/liangfok","followers_url":"https://api.github.com/users/liangfok/followers","following_url":"https://api.github.com/users/liangfok/following{/other_user}","gists_url":"https://api.github.com/users/liangfok/gists{/gist_id}","starred_url":"https://api.github.com/users/liangfok/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/liangfok/subscriptions","organizations_url":"https://api.github.com/users/liangfok/orgs","repos_url":"https://api.github.com/users/liangfok/repos","events_url":"https://api.github.com/users/liangfok/events{/privacy}","received_events_url":"https://api.github.com/users/liangfok/received_events","type":"User","site_admin":false},"created_at":"2016-06-23T00:07:20Z","updated_at":"2016-06-23T00:07:20Z","body":"Does this approach impose any assumptions on the nature of the differential equations? Does it ultimately result in the same solution as the all-at-once approach?"}}
{"actor":{"display_login":"liangfok"}}