{"action":"created","issue":{"url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/19","repository_url":"https://api.github.com/repos/dfdx/Boltzmann.jl","labels_url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/19/labels{/name}","comments_url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/19/comments","events_url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/19/events","html_url":"https://github.com/dfdx/Boltzmann.jl/issues/19","id":126058627,"number":19,"title":"Refactoring","user":{"login":"dfdx","id":428171,"avatar_url":"https://avatars.githubusercontent.com/u/428171?v=3","gravatar_id":"","url":"https://api.github.com/users/dfdx","html_url":"https://github.com/dfdx","followers_url":"https://api.github.com/users/dfdx/followers","following_url":"https://api.github.com/users/dfdx/following{/other_user}","gists_url":"https://api.github.com/users/dfdx/gists{/gist_id}","starred_url":"https://api.github.com/users/dfdx/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dfdx/subscriptions","organizations_url":"https://api.github.com/users/dfdx/orgs","repos_url":"https://api.github.com/users/dfdx/repos","events_url":"https://api.github.com/users/dfdx/events{/privacy}","received_events_url":"https://api.github.com/users/dfdx/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":19,"created_at":"2016-01-11T23:03:29Z","updated_at":"2016-02-29T23:56:10Z","closed_at":null,"body":"As recently discussed in #9, it will be nice to refactor the project to better reflect different kinds of RBMs and different ways to learn it. Currently I can think of the following variations: \r\n\r\n#### RBM kinds\r\n\r\n * ordinary RBMs (with Bernoulli and Gaussian variables, as well as using pure probabilities for visible layer)\r\n * Conditional RBMs\r\n * CLRBM - implementation of RBM for OpenCL (TBD)\r\n * I also have particular interest in implementing original Harmonium by Paul Smolensky as it may work better for very large RBMs with sparse weight matrix\r\n\r\n#### gradient calculation\r\n\r\n * contrastive divergence\r\n * persistent contrastive divergence\r\n * EMF approximation by @sphinxteam\r\n\r\n#### weight update options\r\n\r\n * learning rate\r\n * momentum \r\n * weight decay\r\n * sparsity\r\n\r\nGiven these requirements, I imagine minimal RBM setup to look something like this (for all but `CLRBM` types):\r\n\r\n```\r\ntype RBM\r\n    W::Matrix{Float64}\r\n    vbias::Vector{Float64}\r\n    hbias::Vector{FLoat64}\r\n    gradient::Function\r\n    update!::Function\r\nend\r\n```\r\nwhere `gradient()` and `update!()` are closures initialized with all needed options. Then a single method `fit()` can use these closures to learn RBM and provide monitoring. \r\n\r\nNote that this doesn't restrict implementations to the single type, but instead provides a contract, i.e. specifies fields required to provide compatibility between different methods for learning and inference. \r\n\r\n@Rory-Finnegan @eric-tramel Does it makes sense for you? Do your models fit into this approach?"},"comment":{"url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/comments/190455062","html_url":"https://github.com/dfdx/Boltzmann.jl/issues/19#issuecomment-190455062","issue_url":"https://api.github.com/repos/dfdx/Boltzmann.jl/issues/19","id":190455062,"user":{"login":"Rory-Finnegan","id":5276097,"avatar_url":"https://avatars.githubusercontent.com/u/5276097?v=3","gravatar_id":"","url":"https://api.github.com/users/Rory-Finnegan","html_url":"https://github.com/Rory-Finnegan","followers_url":"https://api.github.com/users/Rory-Finnegan/followers","following_url":"https://api.github.com/users/Rory-Finnegan/following{/other_user}","gists_url":"https://api.github.com/users/Rory-Finnegan/gists{/gist_id}","starred_url":"https://api.github.com/users/Rory-Finnegan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Rory-Finnegan/subscriptions","organizations_url":"https://api.github.com/users/Rory-Finnegan/orgs","repos_url":"https://api.github.com/users/Rory-Finnegan/repos","events_url":"https://api.github.com/users/Rory-Finnegan/events{/privacy}","received_events_url":"https://api.github.com/users/Rory-Finnegan/received_events","type":"User","site_admin":false},"created_at":"2016-02-29T23:56:09Z","updated_at":"2016-02-29T23:56:09Z","body":"@dfdx In theory, yes, you could use a conditional rbm for classification. You'd just need to softmax the label predictions. I'm not sure how common that is or if there are papers that compare it to other common methods, but we could always test it on the MNIST dataset and see how it does.\r\n\r\nThe PR looks good to me, so I'll just add my changes to it."}}