{"action":"opened","issue":{"url":"https://api.github.com/repos/jmschrei/pomegranate/issues/153","repository_url":"https://api.github.com/repos/jmschrei/pomegranate","labels_url":"https://api.github.com/repos/jmschrei/pomegranate/issues/153/labels{/name}","comments_url":"https://api.github.com/repos/jmschrei/pomegranate/issues/153/comments","events_url":"https://api.github.com/repos/jmschrei/pomegranate/issues/153/events","html_url":"https://github.com/jmschrei/pomegranate/issues/153","id":167814700,"number":153,"title":"Support pickling on hmm","user":{"login":"pixelou","id":3764009,"avatar_url":"https://avatars.githubusercontent.com/u/3764009?v=3","gravatar_id":"","url":"https://api.github.com/users/pixelou","html_url":"https://github.com/pixelou","followers_url":"https://api.github.com/users/pixelou/followers","following_url":"https://api.github.com/users/pixelou/following{/other_user}","gists_url":"https://api.github.com/users/pixelou/gists{/gist_id}","starred_url":"https://api.github.com/users/pixelou/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/pixelou/subscriptions","organizations_url":"https://api.github.com/users/pixelou/orgs","repos_url":"https://api.github.com/users/pixelou/repos","events_url":"https://api.github.com/users/pixelou/events{/privacy}","received_events_url":"https://api.github.com/users/pixelou/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-07-27T09:50:35Z","updated_at":"2016-07-27T09:50:35Z","closed_at":null,"body":"Hello,\r\n\r\nI would like to know if you could improve the support for serialization toward something more standard than the `clone()` method for hmms.\r\nCorrect me if I am wrong, but the clone() is specific to scikit learn, which falls back on `__deepcopy__` anyways, so I suggest adding a `__deepcopy__` method and optionally have `clone` point to it.\r\n\r\nAlso I would like to have support for pickling if possible. It is necessary when using python's concurrency as provided by [ProcessPoolExecutor](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor) for example. Although https://github.com/jmschrei/pomegranate/issues/108 adds support for multithreading inside the fitting procedure, I think having the ability to fit distinct models in parallel would be a nice addition.\r\n\r\nAnd while I am on the subject, do you support scikit-learn's [GridSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.grid_search.GridSearchCV.html#sklearn.grid_search.GridSearchCV) ?\r\n\r\nBelow is a sample code illustrating what I mean (slightly larger than necessary, sorry):\r\n\r\n```python\r\n#!/usr/bin/env python3\r\n\r\n\"\"\"This is the (possibly) usual workflow for fitting an HMM with GMM emissions\r\n\"\"\"\r\n\r\nfrom concurrent.futures import ProcessPoolExecutor\r\nfrom itertools import repeat\r\n\r\nimport numpy as np\r\n\r\nfrom pomegranate.base import State\r\nfrom pomegranate.gmm import GeneralMixtureModel\r\nfrom pomegranate.distributions import IndependentComponentsDistribution, \\\r\n    DiscreteDistribution, NormalDistribution\r\nfrom pomegranate.hmm import HiddenMarkovModel\r\n\r\n\r\n# GMM helper stuff ------------------------------------------------------------\r\n\r\ndef check_gmm(gmm):\r\n    \"\"\"Check wether the parameters of a GMM are not Inf or NaN.\r\n    \"\"\"\r\n    for component in gmm.distributions:\r\n        for gaussian in component.distributions:\r\n            if not np.all(np.invert(np.isnan(gaussian.parameters))):\r\n                return False\r\n    return True\r\n    \r\n\r\ndef make_train_gmm(n_components, x):\r\n    \"\"\"Build and fit a Gaussian mixture model.\r\n\r\n    .. note:: Gaussians are restricted to diagonal covariance matrix.\r\n    \"\"\"\r\n\r\n    gmm = GeneralMixtureModel([\r\n        IndependentComponentsDistribution([\r\n            NormalDistribution(1, 1) for _ in range(x.shape[1])])\r\n        for _ in range(n_components)])\r\n    gmm.fit(x)\r\n\r\n    # EM seldom fails to converge, just restart the process if it happens\r\n    return gmm if check_gmm(gmm) else make_train_gmm(n_components, x)\r\n    \r\n    \r\ndef secs2mat(sequences, n):\r\n    \"\"\"Take n roughly equally spaced observation of each sequence.\r\n\r\n    :sequences [np.ndarray]:\r\n        a list of observation sequences, sequences are stacked along the first \r\n        axis.\r\n    :n int:\r\n        number of samples to extract from each sequence\r\n    :return np.ndarray:\r\n        The extracted samples in an array of shape \r\n        (len(sequences), n) + shape(samples)\r\n    :note:\r\n        If the sequence is shorter than n, then some samples are duplicated.\r\n        If sthe sequence is longer than n, it is split into n equal sections, \r\n        and one sample is randomly taken from each.\r\n    \"\"\"\r\n    dispatch = np.stack([np.rint(\r\n        np.linspace(0, len(seq) - len(seq) / n - 1, n) +\r\n        np.random.rand(n) * len(seq) / n).astype(np.int)\r\n                         for seq in sequences])\r\n    return np.stack([seq[samples, ...]\r\n                     for seq, samples in zip(sequences, dispatch)])\r\n         \r\n         \r\n# HMM training routine --------------------------------------------------------\r\n         \r\ndef train_one_hmm(X, n_states=10, n_components=4):\r\n    n_sequences = len(X)\r\n    \r\n    # Build emission models with priors\r\n    pretrain_samples = secs2mat(X, n_states)\r\n    emission_priors = [make_train_gmm(n_components, pretrain_samples[:, s, :])\r\n                       for s in range(n_states)]\r\n\r\n    # Build HMM\r\n    states = [State(dist, name=\"S\" + str(i))\r\n              for i, dist in enumerate(emission_priors)]\r\n    model = HiddenMarkovModel()\r\n    model.add_states(*states)\r\n    model.add_transition(model.start, states[0], 1)\r\n    for i in range(len(states) - 1):  # Regular LR model\r\n        model.add_transition(states[i], states[i], .8)\r\n        model.add_transition(states[i], states[i+1], .1)\r\n    for i in range(len(states) - 2):  # Jumping connections\r\n        model.add_transition(states[i], states[i+2], .1)\r\n    model.add_transition(states[-1], states[-1], .9)\r\n    model.add_transition(states[-1], model.end, .1)\r\n    model.bake()\r\n\r\n    # Fit model\r\n    model.fit(X, stop_threshold=5e-2, max_iterations=5, verbose=True)\r\n    return model            \r\n\r\n\r\n# Demo script -----------------------------------------------------------------\r\n\r\ndef main():\r\n    n_sequences = 200  # the number of training samples\r\n    obs_dim = 24  # observation are 24 dimensional feature vectors\r\n\r\n    samples_lengths = np.random.randint(15, 60, size=(n_sequences))\r\n    X = [np.random.randn(l, obs_dim) for l in samples_lengths]\r\n    \r\n    # This works\r\n    hmm = train_one_hmm(X)\r\n    print(hmm.log_probability(X[0]))\r\n    \r\n    # This does not\r\n    with ProcessPoolExecutor(2) as executor:\r\n        hmms = list(executor.map(train_one_hmm, \r\n                                 repeat(X), [5, 10], [4, 6]))\r\n        for hmm in hmms:\r\n            print(hmm.log_probability(X[0]))\r\n                                 \r\n\r\nif __name__ == \"__main__\":\r\n    main()\r\n```\r\nAnd a proposition of a patch to fix the serialization part, the `__deepcopy__` addition is trivial and up to your personal taste :-) . Please, note that I haven't checked wether deserialization actually runs correctly:\r\n\r\n```patch\r\ndiff --git a/pomegranate/hmm.pyx b/pomegranate/hmm.pyx\r\nindex e16b3b4..f9281ea 100644\r\n--- a/pomegranate/hmm.pyx\r\n+++ b/pomegranate/hmm.pyx\r\n@@ -3079,3 +3079,62 @@ cdef class HiddenMarkovModel( Model ):\r\n \r\n \t\tmodel.bake( verbose=verbose, merge=merge )\r\n \t\treturn model\r\n+\r\n+\tdef __getstate__( self ):\r\n+\t\t\"\"\"Serialize object for pickling.\r\n+\r\n+\t\tReturns\r\n+\t\t-------\r\n+\t\tjson : str\r\n+\t\t\tA properly formatted JSON object.\r\n+\t\t\"\"\"\r\n+\t\treturn self.to_json()\r\n+\r\n+\tdef __setstate__( self, s ):\r\n+\t\t\"\"\"Deserialize object for unpickling.\r\n+\r\n+\t\tParameters\r\n+\t\t----------\r\n+\t\tjson : str\r\n+\t\t\tA properly formatted JSON object.\r\n+\r\n+\t\tReturns\r\n+\t\t-------\r\n+\t\tmodel : HiddenMarkovModel\r\n+\t\t\tThe baked model ready to go.\r\n+\t\t\"\"\"\r\n+\r\n+\t\t# Load a dictionary from a JSON formatted string\r\n+\t\ttry:\r\n+\t\t\td = json.loads( s )\r\n+\t\texcept:\r\n+\t\t\ttry:\r\n+\t\t\t\twith open( s, 'r' ) as infile:\r\n+\t\t\t\t\td = json.load( infile )\r\n+\t\t\texcept:\r\n+\t\t\t\traise IOError(\"String must be properly formatted JSON or filename of properly formatted JSON.\")\r\n+\r\n+\t\t# Make a new generic HMM\r\n+\t\tself.__init__( str(d['name']) )\r\n+\r\n+\t\t# Load all the states from JSON formatted strings\r\n+\t\tstates = [ State.from_json( json.dumps(j) ) for j in d['states'] ]\r\n+\t\tfor i, j in d['distribution ties']:\r\n+\t\t\t# Tie appropriate states together\r\n+\t\t\tstates[i].tie( states[j] )\r\n+\r\n+\t\t# Add all the states to the model\r\n+\t\tself.add_states( states )\r\n+\r\n+\t\t# Indicate appropriate start and end states\r\n+\t\tself.start = states[ d['start_index'] ]\r\n+\t\tself.end = states[ d['end_index'] ]\r\n+\r\n+\t\t# Add all the edges to the model\r\n+\t\tfor start, end, probability, pseudocount, group in d['edges']:\r\n+\t\t\tself.add_transition( states[start], states[end], probability,\r\n+\t\t\t\tpseudocount, group )\r\n+\r\n+\t\t# Bake the model\r\n+\t\tself.bake()\r\n+\r\n```"}}
{"actor":{"display_login":"pixelou"}}