{"action":"created","issue":{"url":"https://api.github.com/repos/mdeff/cnn_graph/issues/36","repository_url":"https://api.github.com/repos/mdeff/cnn_graph","labels_url":"https://api.github.com/repos/mdeff/cnn_graph/issues/36/labels{/name}","comments_url":"https://api.github.com/repos/mdeff/cnn_graph/issues/36/comments","events_url":"https://api.github.com/repos/mdeff/cnn_graph/issues/36/events","html_url":"https://github.com/mdeff/cnn_graph/issues/36","id":354737008,"node_id":"MDU6SXNzdWUzNTQ3MzcwMDg=","number":36,"title":"index X is out of bounds for axis 0 with size Y when pooling","user":{"login":"TMorville","id":4254061,"node_id":"MDQ6VXNlcjQyNTQwNjE=","avatar_url":"https://avatars0.githubusercontent.com/u/4254061?v=4","gravatar_id":"","url":"https://api.github.com/users/TMorville","html_url":"https://github.com/TMorville","followers_url":"https://api.github.com/users/TMorville/followers","following_url":"https://api.github.com/users/TMorville/following{/other_user}","gists_url":"https://api.github.com/users/TMorville/gists{/gist_id}","starred_url":"https://api.github.com/users/TMorville/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TMorville/subscriptions","organizations_url":"https://api.github.com/users/TMorville/orgs","repos_url":"https://api.github.com/users/TMorville/repos","events_url":"https://api.github.com/users/TMorville/events{/privacy}","received_events_url":"https://api.github.com/users/TMorville/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-28T13:41:20Z","updated_at":"2018-08-29T09:10:58Z","closed_at":null,"author_association":"NONE","body":"First off, thanks for developing and sharing this interesting package. I've forked the repo and all test cases work fine. This is probably related to #14, but I've made a new issue because I have data. \r\n\r\nI have an adjacency matrix from a large directed graph. The dimensions of the adjacency matrix are `(7919711, 7116242)` and the structure is extremely sparse, number of non-zero elements are `2732656`. \r\n\r\nWhen I try to run the pooling on a subset `(10000x10000)` of my own data that [you can find here](https://github.com/TMorville/shared_data/blob/master/sparse_matrix/sparse_adj_subset.npz) (5.07 KB file) I can produce errors with the flavour (ran on sparse_adj_subset) \r\n\r\n``` \r\n---------------------------------------------------------------------------\r\nIndexError                                Traceback (most recent call last)\r\n<ipython-input-91-7776269d0a82> in <module>()\r\n----> 1 graphs, perm = coarsening.coarsen(ajd_sparse_ss, levels=3, self_connections=False)\r\n\r\n~/projects/erst/graph-embedding/lib/coarsening.py in coarsen(A, levels, self_connections)\r\n      8     levels.\r\n      9     \"\"\"\r\n---> 10     graphs, parents = metis(A, levels)\r\n     11     perms = compute_perm(parents)\r\n     12 \r\n\r\n~/projects/erst/graph-embedding/lib/coarsening.py in metis(W, levels, rid)\r\n     80         cc = idx_col[perm]\r\n     81         vv = val[perm]\r\n---> 82         cluster_id = metis_one_level(rr,cc,vv,rid,weights)  # rr is ordered\r\n     83         parents.append(cluster_id)\r\n     84 \r\n\r\n~/projects/erst/graph-embedding/lib/coarsening.py in metis_one_level(rr, cc, vv, rid, weights)\r\n    140     for ii in range(N):\r\n    141         tid = rid[ii]\r\n--> 142         if not marked[tid]:\r\n    143             wmax = 0.0\r\n    144             rs = rowstart[tid]\r\n\r\nIndexError: index 9977 is out of bounds for axis 0 with size 9974\r\n```\r\nAnd if I rerun, I get \r\n\r\n```\r\nindex 9994 is out of bounds for axis 0 with size 9974\r\n```\r\nshowing that the index changes. \r\n\r\nI am running with `graphs, perm = coarsening.coarsen(ajd_sparse_ss, levels=3, self_connections=False)` but setting `self_connections=True` gives similar problems. "},"comment":{"url":"https://api.github.com/repos/mdeff/cnn_graph/issues/comments/416882594","html_url":"https://github.com/mdeff/cnn_graph/issues/36#issuecomment-416882594","issue_url":"https://api.github.com/repos/mdeff/cnn_graph/issues/36","id":416882594,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNjg4MjU5NA==","user":{"login":"TMorville","id":4254061,"node_id":"MDQ6VXNlcjQyNTQwNjE=","avatar_url":"https://avatars0.githubusercontent.com/u/4254061?v=4","gravatar_id":"","url":"https://api.github.com/users/TMorville","html_url":"https://github.com/TMorville","followers_url":"https://api.github.com/users/TMorville/followers","following_url":"https://api.github.com/users/TMorville/following{/other_user}","gists_url":"https://api.github.com/users/TMorville/gists{/gist_id}","starred_url":"https://api.github.com/users/TMorville/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/TMorville/subscriptions","organizations_url":"https://api.github.com/users/TMorville/orgs","repos_url":"https://api.github.com/users/TMorville/repos","events_url":"https://api.github.com/users/TMorville/events{/privacy}","received_events_url":"https://api.github.com/users/TMorville/received_events","type":"User","site_admin":false},"created_at":"2018-08-29T09:10:58Z","updated_at":"2018-08-29T09:10:58Z","author_association":"NONE","body":"OK. So the switching of indexes seems to stem from lines 55-56 \r\n```\r\n    if rid is None:\r\n        rid = np.random.permutation(range(N))\r\n```\r\nwhich is later used in `metis_one_level(rr,cc,vv,rid,weights)`\r\n\r\n```\r\n    for ii in range(N):\r\n        tid = rid[ii]\r\n        if not marked[tid]:\r\n            wmax = 0.0\r\n            rs = rowstart[tid]\r\n            marked[tid] = True\r\n            bestneighbor = -1\r\n```\r\nwhere the bug appears. Here `N = rr[nnz-1] + 1` which is 9974 in the test data. This conflicts with the maximum value of `rid`, 9999, which sets tid. So whenever the loop \r\n```\r\n    for ii in range(N):\r\n        tid = rid[ii]\r\n``` \r\ngoes over 9974, it gives `tid` a value > N. Here is the print of a subsample of `tid` before a crash\r\n\r\n```\r\nValue of tid: 322\r\nValue of tid: 2881\r\nValue of tid: 8202\r\nValue of tid: 9726\r\nValue of tid: 8039\r\nValue of tid: 126\r\nValue of tid: 276\r\nValue of tid: 9994\r\n```\r\n"}}
{"actor":{"display_login":"TMorville"}}