{"action":"created","issue":{"url":"https://api.github.com/repos/BIDData/BIDMach/issues/115","repository_url":"https://api.github.com/repos/BIDData/BIDMach","labels_url":"https://api.github.com/repos/BIDData/BIDMach/issues/115/labels{/name}","comments_url":"https://api.github.com/repos/BIDData/BIDMach/issues/115/comments","events_url":"https://api.github.com/repos/BIDData/BIDMach/issues/115/events","html_url":"https://github.com/BIDData/BIDMach/issues/115","id":151459461,"number":115,"title":"Saving model matrices","user":{"login":"DanielTakeshi","id":5532096,"avatar_url":"https://avatars.githubusercontent.com/u/5532096?v=3","gravatar_id":"","url":"https://api.github.com/users/DanielTakeshi","html_url":"https://github.com/DanielTakeshi","followers_url":"https://api.github.com/users/DanielTakeshi/followers","following_url":"https://api.github.com/users/DanielTakeshi/following{/other_user}","gists_url":"https://api.github.com/users/DanielTakeshi/gists{/gist_id}","starred_url":"https://api.github.com/users/DanielTakeshi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/DanielTakeshi/subscriptions","organizations_url":"https://api.github.com/users/DanielTakeshi/orgs","repos_url":"https://api.github.com/users/DanielTakeshi/repos","events_url":"https://api.github.com/users/DanielTakeshi/events{/privacy}","received_events_url":"https://api.github.com/users/DanielTakeshi/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-27T18:37:24Z","updated_at":"2016-05-01T17:24:24Z","closed_at":null,"body":"BIDMach version: b4b21ec955851375f94d68ef5e33b3ba74bf5059\r\nBIDMat version: e69e2abbf8a59d30728a46b2062045c158a3ef2e\r\nOS: Linux\r\n\r\n(Note that the above versions of BIDMach/BIDMat are from March 15, so I'm using the older Net API since the newer one had [this issue I pointed out](https://github.com/BIDData/BIDMach/issues/110), though I'll update soon.)\r\n\r\nI am running `Net.scala` and am trying to save my model matrices, which are the weight matrices of the neural network. In my script to run the Net, I have something like the following, with some of the text missing for brevity:\r\n\r\n```\r\n// Input the training data and training targets\r\nval (nn0, opts0) = Net.learner( ... ); \r\n\r\n// establish several opts0 options here\r\n\r\n// Now create the Net. Note that this uses the OLD API we had.\r\nval net = Net.dnodes3(12, 600, 0.6f, 1, opts0, 2);\r\nopts0.nodeset = net;\r\nval model0 = nn0.model.asInstanceOf[Net];\r\nnn0.train\r\n\r\n// Then it trains. Then we can establish the predictions:\r\nval (mm0, mopts0) = Net.predictor(model0, ta);\r\n\r\n// And do more analysis here...\r\n```\r\n\r\nThere are four model matrices, which makes sense because I used three hidden layers. I can access them as I do in the following code.\r\n\r\nHowever, **I would like to save these matrices for later** so that I can load them in at a later date and retain the same neural network model. I noticed that `Model.scala` has a script that saves the model matrices, which seems to be what I need. However, I can't seem to get the method to work due to some null pointer exceptions. The Net code also has an intriguing `saveMetaData` script, but that doesn't save the model matrices; it just creates a JSON file.\r\n\r\n```\r\nscala> nn0.modelmats(0)\r\nres19: BIDMat.Mat =\r\n   -0.045297  -0.0092075     0.31523   -0.028672   -0.018180     0.22539     0.30375    -0.14349     0.29993     0.24410     0.40790    0.084087  2.9744e-05    -0.21426    0.051473    -0.21216...\r\n     0.42199    -0.17051    -0.14078     0.58514     0.21920   -0.017404     0.14943   -0.010119   -0.027717   -0.014781    0.091575     0.50787     0.31397     0.12593   -0.068055     0.18638...\r\n    0.041949    0.049116   -0.055485      1.9294     0.26858     0.48547     -2.0694     0.29517    -0.24338    -0.29202     0.33721    -0.17602     0.48330     0.36717    -0.23380    -0.21015...\r\n          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..          ..         ...\r\nscala> nn0.modelmats(3)\r\nres22: BIDMat.Mat = 0.12525,0.067624,0.35373,-0.45435,-0.25183,0.47297,-0.41093,0.027092,-0.40172,0.46529,0.23253,-0.42146,-0.23325,-0.18278,-0.031435,0.32417,0.30976,0.34873,0.33608,0.26926,0.057421,0.44147,0.015059,0.17950,0.28458,0.45523,0.11883,-0.071064,-0.098416,-0.34955,-0.23560,-0.29686,0.18701,0.14608,0.14355,-0.16220,-0.0030155,-0.12076,-0.16777,-0.23969,-0.25831,0.12304,0.44126,-0.11717,0.13880,0.15952,0.054054,0.35824,0.33546,0.43250,-0.29599,0.10132,0.65207,0.061772,0.19662,-0.17348,-0.35279,-0.11931,0.11853,-0.46544,-0.45618,-0.21153,0.43132,-0.50410,0.37630,-0.14499,0.037332,0.23761,-0.080131,0.087651,0.00063339,-0.16071,-0.36716,-0.12308,-0.064445,0.074264,-0.39024,0.10091,0.043229,0.033337,0.42847,-0.38802,0.26867,-0.30104,-0.0010750,-0.18589,-0.41896,-0.14014,0.46983,-...\r\nscala> nn0.modelmats(4)\r\njava.lang.ArrayIndexOutOfBoundsException: 4\r\n  ... 33 elided\r\n\r\nscala> nn0.save(\"nn_models_\")\r\n<console>:31: error: value save is not a member of BIDMach.Learner\r\n              nn0.save(\"nn_models_\")\r\n                  ^\r\n\r\nscala> nn0.model.save(\"nn_models_\")\r\njava.lang.NullPointerException\r\n  at BIDMach.models.Model$$anonfun$save$1.apply$mcVI$sp(Model.scala:111)\r\n  at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)\r\n  at BIDMach.models.Model.save(Model.scala:108)\r\n  ... 33 elided\r\n\r\nscala> model0.save(\"nn_models_\")\r\njava.lang.NullPointerException\r\n  at BIDMach.models.Model$$anonfun$save$1.apply$mcVI$sp(Model.scala:111)\r\n  at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)\r\n  at BIDMach.models.Model.save(Model.scala:108)\r\n  ... 33 elided\r\n\r\nscala> model0.saveMetaData(\"nn_models_\")\r\n```"},"comment":{"url":"https://api.github.com/repos/BIDData/BIDMach/issues/comments/216057146","html_url":"https://github.com/BIDData/BIDMach/issues/115#issuecomment-216057146","issue_url":"https://api.github.com/repos/BIDData/BIDMach/issues/115","id":216057146,"user":{"login":"jcanny","id":1195924,"avatar_url":"https://avatars.githubusercontent.com/u/1195924?v=3","gravatar_id":"","url":"https://api.github.com/users/jcanny","html_url":"https://github.com/jcanny","followers_url":"https://api.github.com/users/jcanny/followers","following_url":"https://api.github.com/users/jcanny/following{/other_user}","gists_url":"https://api.github.com/users/jcanny/gists{/gist_id}","starred_url":"https://api.github.com/users/jcanny/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jcanny/subscriptions","organizations_url":"https://api.github.com/users/jcanny/orgs","repos_url":"https://api.github.com/users/jcanny/repos","events_url":"https://api.github.com/users/jcanny/events{/privacy}","received_events_url":"https://api.github.com/users/jcanny/received_events","type":"User","site_admin":false},"created_at":"2016-05-01T17:24:24Z","updated_at":"2016-05-01T17:24:24Z","body":"Daniel, If you look at the source code near the exception you'll see:\r\n\r\n    for (i <- 0 until modelmats.length) {\r\n      val mat = modelmats(i);\r\n      val f = new File(fname+\"modelmat%02d.lz4\" format i);\r\n111      f.getParentFile().mkdirs();\r\n      saveMat(fname+\"modelmat%02d.lz4\" format i, cpu(mat));\r\n    }\r\n\r\nThe use of \"getParentFile\" tells you that this pathname should contain a parent directory. If not it returns null. The save function expects a directory name ending with a \"/\" and saves the model into that directory as separate files. I added some comments to that effect in the source code. "}}