{"action":"created","issue":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/1556","repository_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j","labels_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/1556/labels{/name}","comments_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/1556/comments","events_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/1556/events","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/1556","id":155328355,"number":1556,"title":"core dump in native code with LSTM and rc3.9","user":{"login":"fac2003","id":1728407,"avatar_url":"https://avatars.githubusercontent.com/u/1728407?v=3","gravatar_id":"","url":"https://api.github.com/users/fac2003","html_url":"https://github.com/fac2003","followers_url":"https://api.github.com/users/fac2003/followers","following_url":"https://api.github.com/users/fac2003/following{/other_user}","gists_url":"https://api.github.com/users/fac2003/gists{/gist_id}","starred_url":"https://api.github.com/users/fac2003/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fac2003/subscriptions","organizations_url":"https://api.github.com/users/fac2003/orgs","repos_url":"https://api.github.com/users/fac2003/repos","events_url":"https://api.github.com/users/fac2003/events{/privacy}","received_events_url":"https://api.github.com/users/fac2003/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/labels/bug","name":"bug","color":"d93f0b"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":19,"created_at":"2016-05-17T18:26:39Z","updated_at":"2016-05-22T13:23:45Z","closed_at":null,"body":"I am experiencing reproducible core dumps when running a LSTM training similar to the GravesLSTMCharModellingExample sample. I am happy to share the code and the data if you need it to track down the issue. \r\n```\r\n\r\n#\r\n# A fatal error has been detected by the Java Runtime Environment:\r\n#\r\n#  SIGSEGV (0xb) at pc=0x000000012cf8b6e9, pid=85398, tid=5891\r\n#\r\n# JRE version: Java(TM) SE Runtime Environment (8.0_40-b27) (build 1.8.0_40-b27)\r\n# Java VM: Java HotSpot(TM) 64-Bit Server VM (25.40-b25 mixed mode bsd-amd64 compressed oops)\r\n# Problematic frame:\r\n# C  [libnd4j.dylib+0x236e9]  _ZN5shape3TAD4initEPiS1_i+0x249\r\n#\r\n# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try \"ulimit -c unlimited\" before starting Java again\r\n#\r\n# An error report file with more information is saved as:\r\n# /Users/fac2003/IdeaProjects/git/dl4c/hs_err_pid85398.log\r\n#\r\n```\r\n\r\n\r\nHere's the relevant content of hs_err:\r\n\r\n\r\n```\r\n\r\nStack: [0x000070000011a000,0x000070000021a000],  sp=0x0000700000218400,  free space=1017k\r\nNative frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)\r\nC  [libnd4j.dylib+0x236e9]  shape::TAD::init(int*, int*, int)+0x249\r\nC  [libnd4j.dylib+0x279ef]  functions::reduce::ReduceFunction<float>::exec(float*, int*, float*, float*, int*, int*, int)+0xff\r\nC  [libnd4j.dylib+0x63aa]  NativeOps::execReduceFloat(long long*, int, long long, long long, long long, long long, long long, long long, int)+0x5a\r\nC  [libjnind4j.dylib+0xe6e9]  Java_org_nd4j_nativeblas_NativeOps_execReduceFloat___3JIJJJJJJI+0xb9\r\nJ 1461  org.nd4j.nativeblas.NativeOps.execReduceFloat([JIJJJJJJI)V (0 bytes) @ 0x000000010d7dfa31 [0x000000010d7df940+0xf1]\r\nJ 1986 C2 org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(Lorg/nd4j/linalg/api/ops/Accumulation;[I)Lorg/nd4j/linalg/api/ndarray/INDArray; (1441 bytes) @ 0x000000010d9f5938 [0x000000010d9f4ec0+0xa78]\r\n\r\nJava frames: (J=compiled Java code, j=interpreted, Vv=VM code)\r\nJ 1461  org.nd4j.nativeblas.NativeOps.execReduceFloat([JIJJJJJJI)V (0 bytes) @ 0x000000010d7df9b7 [0x000000010d7df940+0x77]\r\nJ 1986 C2 org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(Lorg/nd4j/linalg/api/ops/Accumulation;[I)Lorg/nd4j/linalg/api/ndarray/INDArray; (1441 bytes) @ 0x000000010d9f5938 [0x000000010d9f4ec0+0xa78]\r\nJ 1985 C2 org.nd4j.linalg.api.ndarray.BaseNDArray.sum([I)Lorg/nd4j/linalg/api/ndarray/INDArray; (18 bytes) @ 0x000000010d9d9dfc [0x000000010d9d9b20+0x2dc]\r\nj  org.deeplearning4j.nn.layers.recurrent.LSTMHelpers.backpropGradientHelper(Lorg/deeplearning4j/nn/conf/NeuralNetConfiguration;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;ZILorg/deeplearning4j/nn/layers/recurrent/FwdPassReturn;ZLjava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/deeplearning4j/berkeley/Pair;+1567\r\nj  org.deeplearning4j.nn.layers.recurrent.GravesLSTM.backpropGradientHelper(Lorg/nd4j/linalg/api/ndarray/INDArray;ZI)Lorg/deeplearning4j/berkeley/Pair;+127\r\nj  org.deeplearning4j.nn.layers.recurrent.GravesLSTM.backpropGradient(Lorg/nd4j/linalg/api/ndarray/INDArray;)Lorg/deeplearning4j/berkeley/Pair;+4\r\nj  org.deeplearning4j.nn.multilayer.MultiLayerNetwork.calcBackpropGradients(Lorg/nd4j/linalg/api/ndarray/INDArray;Z)Lorg/deeplearning4j/berkeley/Pair;+330\r\nj  org.deeplearning4j.nn.multilayer.MultiLayerNetwork.backprop()V+3\r\nj  org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore()V+38\r\nj  org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore()Lorg/deeplearning4j/berkeley/Pair;+12\r\nj  org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize()Z+14\r\nj  org.deeplearning4j.optimize.Solver.optimize()V+19\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/comments/220832226","html_url":"https://github.com/deeplearning4j/deeplearning4j/issues/1556#issuecomment-220832226","issue_url":"https://api.github.com/repos/deeplearning4j/deeplearning4j/issues/1556","id":220832226,"user":{"login":"fcecconi","id":1335186,"avatar_url":"https://avatars.githubusercontent.com/u/1335186?v=3","gravatar_id":"","url":"https://api.github.com/users/fcecconi","html_url":"https://github.com/fcecconi","followers_url":"https://api.github.com/users/fcecconi/followers","following_url":"https://api.github.com/users/fcecconi/following{/other_user}","gists_url":"https://api.github.com/users/fcecconi/gists{/gist_id}","starred_url":"https://api.github.com/users/fcecconi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fcecconi/subscriptions","organizations_url":"https://api.github.com/users/fcecconi/orgs","repos_url":"https://api.github.com/users/fcecconi/repos","events_url":"https://api.github.com/users/fcecconi/events{/privacy}","received_events_url":"https://api.github.com/users/fcecconi/received_events","type":"User","site_admin":false},"created_at":"2016-05-22T13:23:45Z","updated_at":"2016-05-22T13:23:45Z","body":"I have the same problem in linux:\r\n\r\nStack: [0x00007fe413be9000,0x00007fe413cea000],  sp=0x00007fe413ce7c90,  free space=1019k\r\nNative frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)\r\nC  [libnd4j.so+0x65353]  functions::reduce::ReduceFunction<float>::exec(float*, int*, float*, float*, int*, int*, int) [clone ._omp_fn.56]+0x233\r\n\r\nJava frames: (J=compiled Java code, j=interpreted, Vv=VM code)\r\nJ 3802  org.nd4j.nativeblas.NativeOps.execReduceFloat([JIJJJJJJI)V (0 bytes) @ 0x00007fe3fd9d9c37 [0x00007fe3fd9d9bc0+0x77]\r\nJ 4352 C2 org.nd4j.linalg.cpu.nativecpu.ops.NativeOpExecutioner.exec(Lorg/nd4j/linalg/api/ops/Accumulation;[I)Lorg/nd4j/linalg/api/ndarray/INDArray; (1441 bytes) @ 0x00007fe3fd9562c4 [0x00007fe3fd955740+0xb84]\r\nJ 4251 C2 org.nd4j.linalg.api.ndarray.BaseNDArray.sum([I)Lorg/nd4j/linalg/api/ndarray/INDArray; (18 bytes) @ 0x00007fe3fd89c6bc [0x00007fe3fd89c3e0+0x2dc]\r\nJ 6970 C2 org.deeplearning4j.nn.layers.recurrent.LSTMHelpers.backpropGradientHelper(Lorg/deeplearning4j/nn/conf/NeuralNetConfiguration;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;Lorg/nd4j/linalg/api/ndarray/INDArray;ZILorg/deeplearning4j/nn/layers/recurrent/FwdPassReturn;ZLjava/lang/String;Ljava/lang/String;Ljava/lang/String;)Lorg/deeplearning4j/berkeley/Pair; (1986 bytes) @ 0x00007fe3fe2a1f34 [0x00007fe3fe29e680+0x38b4]\r\nJ 6896 C1 org.deeplearning4j.nn.layers.recurrent.GravesLSTM.backpropGradientHelper(Lorg/nd4j/linalg/api/ndarray/INDArray;ZI)Lorg/deeplearning4j/berkeley/Pair; (131 bytes) @ 0x00007fe3fd383cc4 [0x00007fe3fd383060+0xc64]\r\nJ 6895 C1 org.deeplearning4j.nn.layers.recurrent.GravesLSTM.backpropGradient(Lorg/nd4j/linalg/api/ndarray/INDArray;)Lorg/deeplearning4j/berkeley/Pair; (8 bytes) @ 0x00007fe3fd481fcc [0x00007fe3fd481f40+0x8c]\r\nJ 6890 C1 org.deeplearning4j.nn.multilayer.MultiLayerNetwork.calcBackpropGradients(Lorg/nd4j/linalg/api/ndarray/INDArray;Z)Lorg/deeplearning4j/berkeley/Pair; (648 bytes) @ 0x00007fe3fe25b7bc [0x00007fe3fe258a20+0x2d9c]\r\nJ 7367 C1 org.deeplearning4j.nn.multilayer.MultiLayerNetwork.backprop()V (46 bytes) @ 0x00007fe3fdc1e064 [0x00007fe3fdc1dfc0+0xa4]\r\nj  org.deeplearning4j.nn.multilayer.MultiLayerNetwork.computeGradientAndScore()V+38\r\nJ 7693 C1 org.deeplearning4j.optimize.solvers.BaseOptimizer.gradientAndScore()Lorg/deeplearning4j/berkeley/Pair; (67 bytes) @ 0x00007fe3fd1d17bc [0x00007fe3fd1d16a0+0x11c]\r\nJ 7692 C1 org.deeplearning4j.optimize.solvers.StochasticGradientDescent.optimize()Z (147 bytes) @ 0x00007fe3fde1df54 [0x00007fe3fde1de40+0x114]\r\nJ 7691 C1 org.deeplearning4j.optimize.Solver.optimize()V (26 bytes) @ 0x00007fe3fdd12ccc [0x00007fe3fdd12b20+0x1ac]\r\nj  org.deeplearning4j.nn.multilayer.MultiLayerNetwork.fit(Lorg/nd4j/linalg/dataset/api/iterator/DataSetIterator;)V+272\u0000"}}
{"actor":{"display_login":"fcecconi"}}