{"action":"opened","issue":{"url":"https://api.github.com/repos/JuliaML/Roadmap.jl/issues/8","repository_url":"https://api.github.com/repos/JuliaML/Roadmap.jl","labels_url":"https://api.github.com/repos/JuliaML/Roadmap.jl/issues/8/labels{/name}","comments_url":"https://api.github.com/repos/JuliaML/Roadmap.jl/issues/8/comments","events_url":"https://api.github.com/repos/JuliaML/Roadmap.jl/issues/8/events","html_url":"https://github.com/JuliaML/Roadmap.jl/issues/8","id":162541128,"number":8,"title":"Verbs, revisited... and much more","user":{"login":"tbreloff","id":933338,"avatar_url":"https://avatars.githubusercontent.com/u/933338?v=3","gravatar_id":"","url":"https://api.github.com/users/tbreloff","html_url":"https://github.com/tbreloff","followers_url":"https://api.github.com/users/tbreloff/followers","following_url":"https://api.github.com/users/tbreloff/following{/other_user}","gists_url":"https://api.github.com/users/tbreloff/gists{/gist_id}","starred_url":"https://api.github.com/users/tbreloff/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tbreloff/subscriptions","organizations_url":"https://api.github.com/users/tbreloff/orgs","repos_url":"https://api.github.com/users/tbreloff/repos","events_url":"https://api.github.com/users/tbreloff/events{/privacy}","received_events_url":"https://api.github.com/users/tbreloff/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-06-27T20:29:24Z","updated_at":"2016-06-27T20:29:24Z","closed_at":null,"body":"This has been discussed repeatedly, but it's important to get right if we want widespread adoption.  Some references:\r\n\r\nhttps://github.com/Evizero/MLModels.jl/issues/3\r\nhttps://github.com/JuliaOpt/Optim.jl/pull/87\r\nhttps://github.com/JuliaStats/Roadmap.jl/issues/15\r\nhttps://github.com/JuliaStats/Roadmap.jl/issues/4 \r\nhttps://github.com/JuliaStats/Roadmap.jl/issues/20\r\n\r\n(there are more linked in those issues, and I'm sure I missed a bunch of good conversations)\r\n\r\nI recommend a quick skim over those discussions before commenting, if you can find the time.\r\n\r\n\r\n## What are we supporting?\r\n\r\nIt's important to remember all the various things we'd like to support with the core abstractions, so we can evaluate when a concept applies and when it doesn't:\r\n\r\n- Static transformations: log, exp, logit, ...\r\n- Aggregations: mean, variance, extrema...\r\n- Learnable transformations: regressions, neural nets, decision trees, ...\r\n- Compression and dimensionality reduction: PCA, ...\r\n- Generative models: distributions, stochastic variables, ...\r\n\r\nAnd there are some opposing perspectives within these classes:\r\n\r\n- Bayesian vs Frequentist\r\n- Batch vs Online\r\n- Models producing distributions vs point estimates or classifications\r\n\r\nAll verbs need not be implemented by all transformations, but when there's potential for overlap, we should do our best to generalize.\r\n\r\n## Take in inputs, produce outputs\r\n\r\nThe generalization here is that the object knows how to produce `y` in `y = f(x)`.  This could be the logit function, or a previously fitted linear regression, or a decision tree.  Options:\r\n\r\n- transform\r\n- ~~predict~~ (taken by StatsBase)\r\n- ~~map~~ (taken by Base)\r\n- apply (deprecated in Base... similar to call)\r\n- evaluate\r\n- classify (too specific)\r\n\r\nI continue to be a fan of `transform`, with the caveat that we may wish to have the shorthand such that anything that can transform can be called as a functor.\r\n\r\n## Generate/draw from a generative model\r\n\r\n- rand\r\n- sample\r\n- simulate\r\n- draw\r\n- generate\r\n\r\nI think using Base.rand here is generally going to be fine, so I don't think we need this as one of our core verbs.\r\n\r\n## Use data to change the parameters of a model\r\n\r\n- learn\r\n- ~~fit~~ taken by StatsBase\r\n- train\r\n- update\r\n- solve\r\n- optimize\r\n\r\nI've started leaning towards `learn`, partially for the symmetry with `LearnBase`, but also because it is not so actively used in either stats (fit) or ML (train), and so could be argued it's more general.\r\n\r\nI think `solve`/`optimize` should be reserved for higher-level optimization algorithms, and `update` could be reserved for lower-level model updating.\r\n\r\n## Types\r\n\r\nI personally feel **everything** should be a `Transformation`, though I can see the argument that aggregations, distributions and others don't belong.  A `mean` is a function, but really it's a `CenterTransformation` that uses a \"mean function\" to transform data.\r\n\r\nCan a transformation take zero inputs?  If that's the case, then I could argue a generative model might take zero inputs and generate an output, transforming nothing into something.\r\n\r\nIf we think of \"directed graphs of transformations\", then I want to be able to connect a Normal distribution into that graph... we just have the flexibility that the Normal distribution can be a \"source\" in the same way the input data is a \"source\".\r\n\r\nWith this analysis, `AbstractTransformation` is the core type, and we should make every attempt to avoid new types until we require them to solve a conflict.\r\n\r\n## Introspection/Traits\r\n\r\nThere are many things that we could query regarding attributes of our transformations:\r\n\r\n- does it take input data, or is it a source (i.e. a generative process)?\r\n- is it invertible?\r\n- can we take a derivative/gradient?\r\n- is there a proximal operation? (this is not my strong suit!)\r\n- can it be learned?\r\n\r\nI would like to see these things eventually implemented as traits, but in the meantime we'll need methods to ask these questions.\r\n\r\n## Package Layout\r\n\r\nI think we agree that LearnBase will contain the core abstractions... enough that someone can create new models/transformations/solvers without importing lots of concrete implementations of things they don't need.\r\n\r\nWe need homes for concrete implementations of:\r\n\r\n- ModelLoss (MLModels.jl)\r\n- ParameterLoss (MLModels.jl)\r\n- StaticTransformation (MLModels.jl and others)\r\n- LearnableTransformation (MLModels.jl and others)\r\n- Solvers/updaters (StochasticOptimization and DeterministicOptimization?)\r\n\r\n## StatsBase and existing abstractions\r\n\r\nStatsBase contains a ton of assorted methods, types, and algorithms.  StatsBase is too big for it to be a dependency of LearnBase (IMO), and LearnBase is too new to expect that StatsBase would depend on it.  So I think we should have a package which depends on both LearnBase and StatsBase, and \"links\" the abstractions together when it's possible/feasible.  In some cases this might be as easy as defining things like:\r\n\r\n```julia\r\nStatsBase.fit!(t::AbstractTransformation, args...; kw...) = LearnBase.learn!(t, args...; kw...)\r\n```\r\n\r\nWhat are the other packages that we should consider linking with?\r\n\r\ncc: @Evizero @ahwillia @joshday @cstjean @andreasnoack @cmcbride @StefanKarpinski @ninjin @simonbyrne @pluskid \r\n\r\n(If I forgot to cc someone that you think should be involved, please cc them yourself)"}}
{"actor":{"display_login":"tbreloff"}}