{"action":"created","issue":{"url":"https://api.github.com/repos/yt-project/yt/issues/1988","repository_url":"https://api.github.com/repos/yt-project/yt","labels_url":"https://api.github.com/repos/yt-project/yt/issues/1988/labels{/name}","comments_url":"https://api.github.com/repos/yt-project/yt/issues/1988/comments","events_url":"https://api.github.com/repos/yt-project/yt/issues/1988/events","html_url":"https://github.com/yt-project/yt/issues/1988","id":353592627,"node_id":"MDU6SXNzdWUzNTM1OTI2Mjc=","number":1988,"title":"parallel_analysis_interface.py issue","user":{"login":"sflarkin","id":41403179,"node_id":"MDQ6VXNlcjQxNDAzMTc5","avatar_url":"https://avatars3.githubusercontent.com/u/41403179?v=4","gravatar_id":"","url":"https://api.github.com/users/sflarkin","html_url":"https://github.com/sflarkin","followers_url":"https://api.github.com/users/sflarkin/followers","following_url":"https://api.github.com/users/sflarkin/following{/other_user}","gists_url":"https://api.github.com/users/sflarkin/gists{/gist_id}","starred_url":"https://api.github.com/users/sflarkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sflarkin/subscriptions","organizations_url":"https://api.github.com/users/sflarkin/orgs","repos_url":"https://api.github.com/users/sflarkin/repos","events_url":"https://api.github.com/users/sflarkin/events{/privacy}","received_events_url":"https://api.github.com/users/sflarkin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-08-23T23:40:55Z","updated_at":"2018-08-28T23:37:39Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"Hello All,\r\n\r\nI am having an error in the parallel_analysis_interface.py file. I have encountered this error in two places. Firstly on the main yt branch, and secondly on a branch of yt called yt-agora based in python 2.7 (you can find this repository here https://bitbucket.org/migroch/yt-agora) It loos like this function has not been changed since then. \r\n\r\nWhat I am doing is running rockstar, and it is deciding how many writers to assign to the job. Here is the error message I get in yt-agora, I would also post the error message in base yt, but I have misplaced it, and will keep looking for it.\r\n\r\nTraceback (most recent call last):\r\n  File \"./scripts/runagora.py\", line 325, in <module>\r\n    num_readers=args['num_readers'])\r\n  File \"./scripts/runagora.py\", line 223, in run_rockstar\r\n    total_particles=total_particles)\r\n  File \"/home1/sflarkin/build/yt-agora/yt/analysis_modules/halo_finding/rockstar/rockstar.py\", line 259, in __init__\r\n    self.pool, self.workgroup = self.runner.setup_pool()\r\n  File \"/home1/sflarkin/build/yt-agora/yt/analysis_modules/halo_finding/rockstar/rockstar.py\", line 109, in setup_pool\r\n    (self.num_writers, \"writers\") ]\r\n  File \"/home1/sflarkin/build/yt-agora/yt/utilities/parallel_tools/parallel_analysis_interface.py\", line 403, in from_sizes\r\n    pool.add_workgroup(size, name = name)\r\n  File \"/home1/sflarkin/build/yt-agora/yt/utilities/parallel_tools/parallel_analysis_interface.py\", line 371, in add_workgroup\r\n    group = self.comm.comm.Get_group().Incl(ranks)\r\nAttributeError: 'NoneType' object has no attribute 'Get_group'\r\n\r\nAs the error is caused by not having the attribute Get_group, I looked in the directories of each of the parts of self.comm.comm to see if there was anything like it. The results are below.\r\n\r\ndir(self) ['__class__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__getitem__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'add_workgroup', 'available_ranks', 'comm', 'free_all', 'free_workgroup', 'from_sizes', 'ranks', 'size', 'tasks', 'workgroups']\r\n\r\n\r\ndir(self.comm) ['_Communicator__tocast', '__class__', '__del__', '__delattr__', '__dict__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_distributed', '_grids', 'alltoallv_array', 'barrier', 'claim_object', 'comm', 'do_not_claim_object', 'get_filename', 'is_mine', 'merge_quadtree_buffers', 'mpi_Request_Testall', 'mpi_Request_Waitall', 'mpi_Request_Waititer', 'mpi_allreduce', 'mpi_bcast', 'mpi_exit_test', 'mpi_info_dict', 'mpi_nonblocking_recv', 'mpi_nonblocking_send', 'par_combine_object', 'preload', 'probe_loop', 'rank', 'recv_array', 'recv_quadtree', 'send_array', 'send_quadtree', 'size', 'write_on_root']\r\n\r\n\r\ndir(self.comm.comm) ['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']\r\n\r\nAs you can see, there is no Get_group() property in any of them. I am wondering if anyone knows the correct way to fix this, as I am not very familiar with parallel allocation."},"comment":{"url":"https://api.github.com/repos/yt-project/yt/issues/comments/416774784","html_url":"https://github.com/yt-project/yt/issues/1988#issuecomment-416774784","issue_url":"https://api.github.com/repos/yt-project/yt/issues/1988","id":416774784,"node_id":"MDEyOklzc3VlQ29tbWVudDQxNjc3NDc4NA==","user":{"login":"sflarkin","id":41403179,"node_id":"MDQ6VXNlcjQxNDAzMTc5","avatar_url":"https://avatars3.githubusercontent.com/u/41403179?v=4","gravatar_id":"","url":"https://api.github.com/users/sflarkin","html_url":"https://github.com/sflarkin","followers_url":"https://api.github.com/users/sflarkin/followers","following_url":"https://api.github.com/users/sflarkin/following{/other_user}","gists_url":"https://api.github.com/users/sflarkin/gists{/gist_id}","starred_url":"https://api.github.com/users/sflarkin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sflarkin/subscriptions","organizations_url":"https://api.github.com/users/sflarkin/orgs","repos_url":"https://api.github.com/users/sflarkin/repos","events_url":"https://api.github.com/users/sflarkin/events{/privacy}","received_events_url":"https://api.github.com/users/sflarkin/received_events","type":"User","site_admin":false},"created_at":"2018-08-28T23:37:39Z","updated_at":"2018-08-28T23:37:39Z","author_association":"CONTRIBUTOR","body":"I have been looking around the function add_workgroup as it is what fails. \r\n\r\ndef add_workgroup(self, size=None, ranks=None, name=None):\r\n        if size is None:\r\n            size = len(self.available_ranks)\r\n        if len(self.available_ranks) < size:\r\n            mylog.error('Not enough resources available, asked for %d have %d',\r\n                size, self.available_ranks)\r\n            raise RuntimeError\r\n        if ranks is None:\r\n            ranks = [self.available_ranks.pop(0) for i in range(size)]\r\n        # Default name to the workgroup number.\r\n        if name is None:\r\n            name = str(len(self.workgroups))\r\n        group = self.comm.comm.Get_group().Incl(ranks)\r\n        new_comm = self.comm.comm.Create(group)\r\n        if self.comm.rank in ranks:\r\n            communication_system.communicators.append(Communicator(new_comm))\r\n        self.workgroups.append(Workgroup(len(ranks), ranks, new_comm, name))\r\n\r\nI have looked through all the variables passed to it by the num_writers function, and everything appears to work exactly as intened, until the \r\n\r\ngroup = self.comm.comm.Get_group().Incl(ranks)\r\n\r\nline. I am wondering what this defination does, as I am unfamiliar with parallel communicators, to see if there is a way to add the correct when the pool is defined in the from_sizes function where it is created."}}
{"actor":{"display_login":"sflarkin"}}