{"action":"created","issue":{"url":"https://api.github.com/repos/cznic/kv/issues/32","repository_url":"https://api.github.com/repos/cznic/kv","labels_url":"https://api.github.com/repos/cznic/kv/issues/32/labels{/name}","comments_url":"https://api.github.com/repos/cznic/kv/issues/32/comments","events_url":"https://api.github.com/repos/cznic/kv/issues/32/events","html_url":"https://github.com/cznic/kv/issues/32","id":174950176,"number":32,"title":"Index out of range","user":{"login":"billyfbrain","id":4091464,"avatar_url":"https://avatars.githubusercontent.com/u/4091464?v=3","gravatar_id":"","url":"https://api.github.com/users/billyfbrain","html_url":"https://github.com/billyfbrain","followers_url":"https://api.github.com/users/billyfbrain/followers","following_url":"https://api.github.com/users/billyfbrain/following{/other_user}","gists_url":"https://api.github.com/users/billyfbrain/gists{/gist_id}","starred_url":"https://api.github.com/users/billyfbrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/billyfbrain/subscriptions","organizations_url":"https://api.github.com/users/billyfbrain/orgs","repos_url":"https://api.github.com/users/billyfbrain/repos","events_url":"https://api.github.com/users/billyfbrain/events{/privacy}","received_events_url":"https://api.github.com/users/billyfbrain/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/cznic/kv/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":{"login":"cznic","id":729580,"avatar_url":"https://avatars.githubusercontent.com/u/729580?v=3","gravatar_id":"","url":"https://api.github.com/users/cznic","html_url":"https://github.com/cznic","followers_url":"https://api.github.com/users/cznic/followers","following_url":"https://api.github.com/users/cznic/following{/other_user}","gists_url":"https://api.github.com/users/cznic/gists{/gist_id}","starred_url":"https://api.github.com/users/cznic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cznic/subscriptions","organizations_url":"https://api.github.com/users/cznic/orgs","repos_url":"https://api.github.com/users/cznic/repos","events_url":"https://api.github.com/users/cznic/events{/privacy}","received_events_url":"https://api.github.com/users/cznic/received_events","type":"User","site_admin":false},"assignees":[{"login":"cznic","id":729580,"avatar_url":"https://avatars.githubusercontent.com/u/729580?v=3","gravatar_id":"","url":"https://api.github.com/users/cznic","html_url":"https://github.com/cznic","followers_url":"https://api.github.com/users/cznic/followers","following_url":"https://api.github.com/users/cznic/following{/other_user}","gists_url":"https://api.github.com/users/cznic/gists{/gist_id}","starred_url":"https://api.github.com/users/cznic/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cznic/subscriptions","organizations_url":"https://api.github.com/users/cznic/orgs","repos_url":"https://api.github.com/users/cznic/repos","events_url":"https://api.github.com/users/cznic/events{/privacy}","received_events_url":"https://api.github.com/users/cznic/received_events","type":"User","site_admin":false}],"milestone":null,"comments":10,"created_at":"2016-09-04T13:57:19Z","updated_at":"2016-09-11T09:51:34Z","closed_at":null,"body":"Hi, after update, I catch panic.\r\npanic: runtime error: index out of range\r\ngoroutine 7957 [running]:\r\npanic(0x452e0e0, 0xc42000c0f0)\r\n       \t/Users/billyfbrain/coding/go/go/src/runtime/panic.go:500 +0x1a1\r\ncern/vendor/github.com/cznic/lldb.btreePage.find(0x0, 0x0, 0x0, 0x4836b20, 0xc4203eab00, 0x0, 0xc42016e190, 0xf, 0x10, 0x0, ...)\r\n       \t/Users/billyfbrain/coding/go/src/cern/vendor/github.com/cznic/lldb/btree.go:1599 +0x334\r\ncern/vendor/github.com/cznic/lldb.btree.get(0x1, 0x4836b20, 0xc4203eab00, 0xc420296000, 0x1010f, 0x20000, 0x0, 0xc42016e190, 0xf, 0x10, ...)\r\n       \t/Users/billyfbrain/coding/go/src/cern/vendor/github.com/cznic/lldb/btree.go:2018 +0x19c\r\ncern/vendor/github.com/cznic/lldb.(*BTree).Get(0xc4204e9770, 0x0, 0x0, 0x0, 0xc42016e190, 0xf, 0x10, 0x0, 0x0, 0x0, ...)\r\n       \t/Users/billyfbrain/coding/go/src/cern/vendor/github.com/cznic/lldb/btree.go:251 +0x157\r\ncern/vendor/github.com/cznic/kv.(*DB).Get(0xc4201c1b90, 0x0, 0x0, 0x0, 0xc42016e190, 0xf, 0x10, 0x0, 0x0, 0x0, ...)\r\n       \t/Users/billyfbrain/coding/go/src/cern/vendor/github.com/cznic/kv/kv.go:657 +0x127\r\ncern/storage/blob.Open(0xc4202eeba0, 0x51, 0xc42004d7e6, 0x45328e0, 0x37, 0xc42052e680)"},"comment":{"url":"https://api.github.com/repos/cznic/kv/issues/comments/246171698","html_url":"https://github.com/cznic/kv/issues/32#issuecomment-246171698","issue_url":"https://api.github.com/repos/cznic/kv/issues/32","id":246171698,"user":{"login":"billyfbrain","id":4091464,"avatar_url":"https://avatars.githubusercontent.com/u/4091464?v=3","gravatar_id":"","url":"https://api.github.com/users/billyfbrain","html_url":"https://github.com/billyfbrain","followers_url":"https://api.github.com/users/billyfbrain/followers","following_url":"https://api.github.com/users/billyfbrain/following{/other_user}","gists_url":"https://api.github.com/users/billyfbrain/gists{/gist_id}","starred_url":"https://api.github.com/users/billyfbrain/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/billyfbrain/subscriptions","organizations_url":"https://api.github.com/users/billyfbrain/orgs","repos_url":"https://api.github.com/users/billyfbrain/repos","events_url":"https://api.github.com/users/billyfbrain/events{/privacy}","received_events_url":"https://api.github.com/users/billyfbrain/received_events","type":"User","site_admin":false},"created_at":"2016-09-11T09:51:34Z","updated_at":"2016-09-11T09:51:34Z","body":"My friend @alexmurm have to reproduce the case\r\n``` go\r\npackage main\r\n\r\nimport (\r\n\t\"crypto/sha512\"\r\n\t\"encoding/hex\"\r\n\t\"log\"\r\n\t\"strconv\"\r\n\t\"sync\"\r\n\r\n\t\"github.com/cznic/kv\"\r\n)\r\n\r\nconst (\r\n\tdbName = \"test.db\"\r\n)\r\n\r\ntype DB struct {\r\n\tsync.Mutex\r\n\tdb       *kv.DB\r\n\twg       *sync.WaitGroup\r\n\tindexNum int      // indexes per db\r\n\tnumDB    int      // index number of db\r\n\tdbList   []string //list of all dbs\r\n}\r\n\r\ntype DBWithoutMutex struct {\r\n\tdb       *kv.DB\r\n\twg       *sync.WaitGroup\r\n\tindexNum int      // indexes per db\r\n\tnumDB    int      // index number of db\r\n\tdbList   []string //list of all dbs\r\n}\r\n\r\nfunc main() {\r\n\tdone := make(chan struct{})\r\n\tclosingDB := make(chan DBWithoutMutex)\r\n\r\n\tgo queue(closingDB, done)\r\n\r\n\tvar err error\r\n\tdb := DB{}\r\n\tdb.db, err = kv.Create(dbName+strconv.Itoa(db.numDB), &kv.Options{VerifyDbBeforeOpen: true})\r\n\tif err != nil {\r\n\t\tlog.Fatal(err)\r\n\t}\r\n\tdb.wg = &sync.WaitGroup{}\r\n\tdb.dbList = append(db.dbList, dbName+strconv.Itoa(db.numDB))\r\n\r\n\t// All keys\r\n\tkeys := struct {\r\n\t\tsync.Mutex\r\n\t\tkeys [][]byte\r\n\t}{\r\n\t\tkeys: [][]byte{},\r\n\t}\r\n\r\n\twg := sync.WaitGroup{}\r\n\tstartC := make(chan struct{})\r\n\tfor count := 0; count < 100; count++ {\r\n\t\twg.Add(1)\r\n\t\tgo func(count int) {\r\n\t\t\tdefer wg.Done()\r\n\t\t\t<-startC\r\n\t\t\tfor i := 0; i < 100; i++ {\r\n\r\n\t\t\t\th := sha512.New()\r\n\t\t\t\th.Write([]byte(strconv.Itoa(i) + \"_\" + strconv.Itoa(count)))\r\n\r\n\t\t\t\tkeys.Lock()\r\n\t\t\t\tkeys.keys = append(keys.keys, h.Sum(nil))\r\n\t\t\t\tkeys.Unlock()\r\n\r\n\t\t\t\tdb.Lock()\r\n\t\t\t\tdb.indexNum += 1\r\n\t\t\t\tif db.indexNum == 1000 {\r\n\t\t\t\t\tdb.indexNum = 0\r\n\t\t\t\t\tclosingDB <- DBWithoutMutex{\r\n\t\t\t\t\t\tdb:       db.db,\r\n\t\t\t\t\t\twg:       db.wg,\r\n\t\t\t\t\t\tindexNum: db.indexNum,\r\n\t\t\t\t\t\tnumDB:    db.numDB,\r\n\t\t\t\t\t\tdbList:   db.dbList,\r\n\t\t\t\t\t}\r\n\t\t\t\t\tdb.numDB += 1\r\n\t\t\t\t\tdb.db, err = kv.Create(dbName+strconv.Itoa(db.numDB), &kv.Options{VerifyDbBeforeOpen: true})\r\n\t\t\t\t\tif err != nil {\r\n\t\t\t\t\t\tlog.Fatalln(\"Error create\", dbName+strconv.Itoa(db.numDB))\r\n\t\t\t\t\t}\r\n\t\t\t\t\tdb.wg = &sync.WaitGroup{}\r\n\t\t\t\t\tdb.dbList = append(db.dbList, dbName+strconv.Itoa(db.numDB))\r\n\t\t\t\t}\r\n\t\t\t\tcurrentDB := DBWithoutMutex{\r\n\t\t\t\t\tdb:       db.db,\r\n\t\t\t\t\twg:       db.wg,\r\n\t\t\t\t\tindexNum: db.indexNum,\r\n\t\t\t\t\tnumDB:    db.numDB,\r\n\t\t\t\t\tdbList:   db.dbList,\r\n\t\t\t\t}\r\n\t\t\t\tcurrentDBname := dbName + strconv.Itoa(db.numDB)\r\n\t\t\t\tcurrentDB.wg.Add(1)\r\n\t\t\t\tdb.Unlock()\r\n\r\n\t\t\t\t// time.Sleep(time.Duration(rand.Intn(100)) * time.Microsecond)\r\n\t\t\t\tval := []byte{1, 2, 3}\r\n\r\n\t\t\t\t_, _, err = currentDB.db.Put(nil, h.Sum(nil), func(k, v []byte) ([]byte, bool, error) { return val, true, nil })\r\n\t\t\t\tif err != nil {\r\n\t\t\t\t\tlog.Fatalf(\"Error put key %d to %s: %s\\n\", currentDB.indexNum, currentDBname, err)\r\n\t\t\t\t}\r\n\r\n\t\t\t\t_, err = currentDB.db.Get(nil, h.Sum(nil))\r\n\t\t\t\tif err != nil {\r\n\t\t\t\t\tlog.Fatalf(\"Error1 get key %d from %s: %s\\n\", currentDB.indexNum, currentDBname, err)\r\n\t\t\t\t}\r\n\t\t\t\t_, err = currentDB.db.Get(nil, h.Sum(nil))\r\n\t\t\t\tif err != nil {\r\n\t\t\t\t\tlog.Fatalf(\"Error2 get key %d from %s: %s\\n\", currentDB.indexNum, currentDBname, err)\r\n\t\t\t\t}\r\n\t\t\t\tcurrentDB.wg.Done()\r\n\t\t\t}\r\n\t\t}(count)\r\n\t}\r\n\tclose(startC)\r\n\twg.Wait()\r\n\tlog.Println(len(keys.keys))\r\n\tdb.db.Close()\r\n\tclose(done)\r\n\r\n\tfor _, path := range db.dbList {\r\n\t\tcurDB, err := kv.Open(path, &kv.Options{VerifyDbBeforeOpen: true})\r\n\t\tif err != nil {\r\n\t\t\tlog.Fatalf(\"Error open %s: %s\\n\", path, err)\r\n\t\t}\r\n\t\tfor _, k := range keys.keys {\r\n\t\t\t_, err = curDB.Get(nil, k)\r\n\t\t\tif err != nil {\r\n\t\t\t\tlog.Fatalf(\"Error get key %s from %s: %s\\n\", hex.EncodeToString(k), path, err)\r\n\t\t\t}\r\n\t\t}\r\n\t\tcurDB.Close()\r\n\t}\r\n}\r\n\r\nfunc queue(closingDB <-chan DBWithoutMutex, done <-chan struct{}) {\r\n\tfor {\r\n\t\tselect {\r\n\t\tcase db := <-closingDB:\r\n\t\t\tlog.Println(\"Closing\")\r\n\t\t\tgo func() {\r\n\t\t\t\tdb.wg.Wait()\r\n\t\t\t\tdb.db.Close()\r\n\t\t\t}()\r\n\t\tcase <-done:\r\n\t\t\treturn\r\n\t\t}\r\n\t}\r\n}\r\n```"}}
{"actor":{"display_login":"billyfbrain"}}