{"action":"created","issue":{"url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/68","repository_url":"https://api.github.com/repos/pywren/pywren-ibm-cloud","labels_url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/68/labels{/name}","comments_url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/68/comments","events_url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/68/events","html_url":"https://github.com/pywren/pywren-ibm-cloud/issues/68","id":395232717,"node_id":"MDU6SXNzdWUzOTUyMzI3MTc=","number":68,"title":"concurrency/massive function spawning","user":{"login":"chen116","id":13974230,"node_id":"MDQ6VXNlcjEzOTc0MjMw","avatar_url":"https://avatars0.githubusercontent.com/u/13974230?v=4","gravatar_id":"","url":"https://api.github.com/users/chen116","html_url":"https://github.com/chen116","followers_url":"https://api.github.com/users/chen116/followers","following_url":"https://api.github.com/users/chen116/following{/other_user}","gists_url":"https://api.github.com/users/chen116/gists{/gist_id}","starred_url":"https://api.github.com/users/chen116/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chen116/subscriptions","organizations_url":"https://api.github.com/users/chen116/orgs","repos_url":"https://api.github.com/users/chen116/repos","events_url":"https://api.github.com/users/chen116/events{/privacy}","received_events_url":"https://api.github.com/users/chen116/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2019-01-02T12:25:16Z","updated_at":"2019-01-02T13:13:04Z","closed_at":null,"author_association":"NONE","body":"Hi, \r\n\r\nI have two 2 questions about concurrency\r\n\r\n1. \r\n\r\nIn the paper(https://dl.acm.org/citation.cfm?id=3284029), _massive function spawning_ is mentioned and  I am wondering how to enable/configure it as a user. I looked into the source code. and I assume the parameter` invoke_pool_threads` corresponds to the _massive function spawning_  mechanism? For example, quoting from the paper : \r\n\r\n> For example, pretend that it is necessary to execute 1, 000 functions. In this case, the massive function spawning mechanism would internally arrange 10 groups of 100 invocations, which would be processed by 10 remote invoker functions separately.\r\n\r\nso in that example, say the original script need to run 1000 functions, then to achieve this \"10 groups of 100 invocations\", user specify ` invoke_pool_threads=100` and `remote_invocation=True` ?\r\n\r\n\r\n2. \r\n\r\nGenerally, is concurrency enable/configure using the parameter `invoke_pool_threads` ? if not, which parameters are available? \r\n\r\nplease let me know, thanks!"},"comment":{"url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/comments/450860028","html_url":"https://github.com/pywren/pywren-ibm-cloud/issues/68#issuecomment-450860028","issue_url":"https://api.github.com/repos/pywren/pywren-ibm-cloud/issues/68","id":450860028,"node_id":"MDEyOklzc3VlQ29tbWVudDQ1MDg2MDAyOA==","user":{"login":"gilv","id":6321715,"node_id":"MDQ6VXNlcjYzMjE3MTU=","avatar_url":"https://avatars0.githubusercontent.com/u/6321715?v=4","gravatar_id":"","url":"https://api.github.com/users/gilv","html_url":"https://github.com/gilv","followers_url":"https://api.github.com/users/gilv/followers","following_url":"https://api.github.com/users/gilv/following{/other_user}","gists_url":"https://api.github.com/users/gilv/gists{/gist_id}","starred_url":"https://api.github.com/users/gilv/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gilv/subscriptions","organizations_url":"https://api.github.com/users/gilv/orgs","repos_url":"https://api.github.com/users/gilv/repos","events_url":"https://api.github.com/users/gilv/events{/privacy}","received_events_url":"https://api.github.com/users/gilv/received_events","type":"User","site_admin":false},"created_at":"2019-01-02T13:13:04Z","updated_at":"2019-01-02T13:13:04Z","author_association":"COLLABORATOR","body":"@chen116 This is how you can provide those parameters\r\n\r\n    pw.map_reduce(my_map_function, iterdata, my_reduce_function,\r\n    reducer_wait_local=False, remote_invocation=True, invoke_pool_threads=10)"}}
{"actor":{"display_login":"gilv"}}