{"action":"created","issue":{"url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/5","repository_url":"https://api.github.com/repos/JuliaDiffEq/Roadmap","labels_url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/5/labels{/name}","comments_url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/5/comments","events_url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/5/events","html_url":"https://github.com/JuliaDiffEq/Roadmap/issues/5","id":186262672,"number":5,"title":"ODE/DAE solver interface","user":{"login":"mauro3","id":4098145,"avatar_url":"https://avatars.githubusercontent.com/u/4098145?v=3","gravatar_id":"","url":"https://api.github.com/users/mauro3","html_url":"https://github.com/mauro3","followers_url":"https://api.github.com/users/mauro3/followers","following_url":"https://api.github.com/users/mauro3/following{/other_user}","gists_url":"https://api.github.com/users/mauro3/gists{/gist_id}","starred_url":"https://api.github.com/users/mauro3/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mauro3/subscriptions","organizations_url":"https://api.github.com/users/mauro3/orgs","repos_url":"https://api.github.com/users/mauro3/repos","events_url":"https://api.github.com/users/mauro3/events{/privacy}","received_events_url":"https://api.github.com/users/mauro3/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2016-10-31T11:36:24Z","updated_at":"2016-11-01T10:55:26Z","closed_at":null,"body":"I think it would be great if all the ODE/DAE solvers could be called through the same interface.  This would essentially mirror the approach of https://github.com/JuliaOpt/MathProgBase.jl .  Would it be worth trying to hammer out this interface in this road-map issue?\r\n\r\nEdit: By \"same interface\" I don't mean that ODE and DAE solvers have the same interface.  But that all ODE solvers are wrapped to conform to the same interface (similarly all DAE, and maybe all IMEX, etc).  So, Sundials.jl, ODE.jl, etc. would then depend on `ODEProgBase` (or probably `DiffEqBase`) and extend some `solve` function there, taking as input some problem specification type.  Likewise all higher level Diff-Eq stuff (currently DifferentialEquations.jl and IVPTestSuite.jl) would program against this interface."},"comment":{"url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/comments/257538683","html_url":"https://github.com/JuliaDiffEq/Roadmap/issues/5#issuecomment-257538683","issue_url":"https://api.github.com/repos/JuliaDiffEq/Roadmap/issues/5","id":257538683,"user":{"login":"mauro3","id":4098145,"avatar_url":"https://avatars.githubusercontent.com/u/4098145?v=3","gravatar_id":"","url":"https://api.github.com/users/mauro3","html_url":"https://github.com/mauro3","followers_url":"https://api.github.com/users/mauro3/followers","following_url":"https://api.github.com/users/mauro3/following{/other_user}","gists_url":"https://api.github.com/users/mauro3/gists{/gist_id}","starred_url":"https://api.github.com/users/mauro3/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mauro3/subscriptions","organizations_url":"https://api.github.com/users/mauro3/orgs","repos_url":"https://api.github.com/users/mauro3/repos","events_url":"https://api.github.com/users/mauro3/events{/privacy}","received_events_url":"https://api.github.com/users/mauro3/received_events","type":"User","site_admin":false},"created_at":"2016-11-01T10:55:25Z","updated_at":"2016-11-01T10:55:25Z","body":">>Here I'm wondering whether it would make sense to declare it, or some variation thereof as the \"community standard\".\n>\n> The thing is, I don't know what right I, or anyone else, would have to declare a \"community standard\". What I can do is build everything I do off of this standard, document it, and personally recommend it. I am at least building it as a standard type hierarchy and interface for everything that I am doing. If there are ways to to make this a more useful interface in a broader sense, let's discuss it.\n\nI think that is fine as a community standard.  I guess one thing is that the interface ought to be stable-ish, which sounds like you're heading towards.\n\n> On that note Sundials.jl is undergoing an API change. I was trying to keep it \"as Sundials-y\" as possible (i.e. not introduce problem/solution types) but am realizing that to take full advantage of Sundials I am going to want at least a special solution type. To handle this I was going to write an alternative Sundials API using DiffEqBase.jl things, and use this through OrdinaryDiffEq.jl so as to not affect the current Sundials API too much.\n\nYes, it would be just fine for solvers to have their own interface too.  Although, one indication of a good the standard interface would be that most adapt it as the only interface eventually.\n\n> The thing I really need to think about is how to deal with existing implementations which I think are not optimized and may need to be re-written. For example, the GSoC project is written in a variable-coefficient format instead of a Nystrom format which is known to not be as well-optimized (and it uses a lot of mutable data structures in the inner loops). There's a double-edged sword here that I am going to be implementing a Nystrom version of ABM methods in order to have a well-optimized version available, but yes it would be in some sense stepping on another's toes (and the change would pretty much require a re-design of the current ABM anyways). I do see how this could build a culture against contributions and a \"ehh just let Chris do it attitude\" which is not good for the me or the community because, even though I have been pretty productive, this has a huge scope and I do need help.\n\nThat is fine, no worries.  So all the ODE solvers you're writing would go into DiffEqODE.jl (or DAE), right?\n\n>>Last, maybe it's still too early to do this. DifferentialEquations.jl is young, so is the iterator interface in ODE.jl. Thus it may not be clear yet what a useful interface would be.\n>\n> Indeed, that's why I'm taking a kind of brute-force tactic of \"build it all and see what sticks\". However, maybe we can hammer out a type-structure which works for all of us. Maybe this should then move over to DiffEqBase.jl, but here's some thoughts on that:\n>\n> - It may be better in many cases to define an interface over abstract types. For example, the plot recipes work on AbstractODESolution types, and `AbstractSDESolution <: AbstractODESolution` and contain fields `t` and `timeseries` so they plot the same. A clear definition of the interface would allow people to write tools over them, or get all of this stuff \"for free\".\n\nYes, that sounds right.  Maybe it would make sense to have a standard type which has one field which can be used freely:\n\n```\ntype ODESolution{T,Y,O}\n    t::T # times\n    y::Y # solution\n    ...\n    other::O # put whatever else is needed for a particular solver in-here\nend\n```\n\n> - The problem types I use are pretty simple, see [this](https://github.com/JuliaDiffEq/OrdinaryDiffEq.jl/blob/master/src/problems.jl). Maybe that could go over to DiffEqBase.jl with some modifications (add places for Jacobians and Hessians? I think that the Jacobian and Hessians are properties of the function, and should thus be interfaced as a dispatch on the function itself like in ParameterizedFunctions.jl. Because we are using such a powerful language, we should also allow for symbolically calculated inverse/pre-factorized Jacobians and Hessians which again ParameterizedFunctions.jl already does, and so I think it's another case where an interface through overloading the function makes it easier for people to use this without the extra machinery if it doesn't exist)\n\nI like simple.  Julia's powers make it easy to come up with too complicated solutions.  Having said this, I'll need to check ParameterizedFunctions.jl again as that did seem quite complicated to me.\n\n> - The Solution objects definitely have to be tailored to the solver output. Think about how for Sundials the `mem` type has to be stored and all interpolations need to be built for that, which is completely different than native algorithm dense output. Another case where this crops up is that with Hermite interpolations you want to store the derivative at each timepoint, whereas for higher order interpolations you want a vector of derivatives at evaluation points for each timepoint, so both what is stored and the algorithm on it are very different. What we could do is just define an abstract interface here, where for an `AbstractODESolution`, `dense==true` means that the object has its call overloaded with the interpolating function, that is `sol(t)` is the continuous solution at time `t`. This is already pretty much standardized, and other interfaces can be documented like that. Then, even if the implementation are very different for solutions of different algorithms, and algorithm written on an `AbstractODESolution` (like parameter estimation or sensitivity analysis) can know to just use `sol(t)`, no matter the algorithm.\n\nIn all types there needs to be room for custom fields.  Letting t use to get the solution at time t looks good.  Maybe use [] to get it though to mirror Interpolations.jl?\n\n> And the real biggie:\n>\n> - `ode_solve.jl` in OrdinaryDiffEq.jl is a mess. I am not sure using `alg=:DP5` and what not for choosing the algorithm was the right idea. If this was instead based off of types and dispatch (with a smart tolerance and specifier-based default dispatch using a `plan_ode` method, or just a dispatch from a DefaultODEAlgorithm), then conditional dependencies wouldn't be necessary, and would could just add new algorithms to OrdinaryDiffEq.jl in their own separate package by adding a new dispatch (you can already do this, but you would currently need to add a lot of stuff to internal dictionaries). If fixing up this algorithm selection mechanism would help bring in contributors, then this will get bumped up the priority list.\n\nWe were/are looking at this over in PR49 and we do dispatch on the solver-type because of the concern you expressed above.  Our strategy is the following: a problem is a pair: (IVP-specification, integrator), which corresponds to your (ODEProblem, ODEIntegrator).  In our case the types are called (IVP, AbstractSolver) and wrapped in a `Problem` type.  (Note, a bit confusing: what you call \"Problem\" is our IVP-specs only, whereas we call the pair IVP+integrator the \"Problem\").  See [here](https://github.com/JuliaODE/ODE.jl/blob/e47748ba852ebf4f9b1acf4ce0771e577bbe92e5/src/base.jl#L189).\n\nThe thing we noted then is that the integrator instance (<:AbstractSolver) can only be made once the IVP is specified as its type-parameters depend on the parameters of IVP (say time is in Float32).  Thus to create an instance of a Solver you need to pass it the IVP-instance: `Solver(ivp; opts...)` and thus `Problem(ivp, Solver(ivp; opts...))`.  (And the same goes for the structure holding the options as its type parameters also depend on the IVP-parameters and solver parameters.  This also goes in direction of addressing your `plan_ode` concern.)\n\nThus a integrator needs to provide the constructor `Solver(ivp; opts...)` and then it can be used in PR49.  A simple example is [here](https://github.com/JuliaODE/ODE.jl/blob/e47748ba852ebf4f9b1acf4ce0771e577bbe92e5/src/integrators/ode23s.jl#L6) and more complicated one is [here](https://github.com/JuliaODE/ODE.jl/blob/e47748ba852ebf4f9b1acf4ce0771e577bbe92e5/src/integrators/runge-kutta.jl#L49)\n\n**TLDR** Maybe a suitable interface would thus be:\n```\nabstract DEProblem\nabstract AbstractIntegrator\n\nsolve{I<:AbstractIntegrator}(de::DEProblem, ::Type{I}; options...) = solve(de, I(de, options))\n```\n\nIf for some reason a package cannot implement the `I(de, options)` constructor, then it can just overload `solve` directly `solve(de::DEProblem, ::Type{MyIntegrator}; options...) = ...`\n\nWe should provide standard subtypes of DEProblem for say ODE, DAE, etc.  But if a solver requires a special subtype of DEProblem then it can define one.\n\nNot sure whether we should try to standardize the options too.  In PR49 we try this: https://github.com/JuliaODE/ODE.jl/blob/master/src/options.jl.\n\nAlso, it maybe good to have several levels to the interface.  Above would be the top-level which would make most functionality of down-stream packages available.  But then if something extra is needed, say sensitivity analysis requires some extra bits, then a deeper layer of the interface also needs implementing to unlock that feature.\n\n> I never intended to work on ODEs, but I just needed something which was fast enough for the PDEs I was solving.\n\nSame here ;-)\n"}}
{"actor":{"display_login":"mauro3"}}