{"action":"created","issue":{"url":"https://api.github.com/repos/rpm-software-management/rpm/issues/519","repository_url":"https://api.github.com/repos/rpm-software-management/rpm","labels_url":"https://api.github.com/repos/rpm-software-management/rpm/issues/519/labels{/name}","comments_url":"https://api.github.com/repos/rpm-software-management/rpm/issues/519/comments","events_url":"https://api.github.com/repos/rpm-software-management/rpm/issues/519/events","html_url":"https://github.com/rpm-software-management/rpm/issues/519","id":351663244,"node_id":"MDU6SXNzdWUzNTE2NjMyNDQ=","number":519,"title":"RFE: run rpm scripts on a different thread using MQTT pub/sub message queues","user":{"login":"n3npq","id":643926,"node_id":"MDQ6VXNlcjY0MzkyNg==","avatar_url":"https://avatars3.githubusercontent.com/u/643926?v=4","gravatar_id":"","url":"https://api.github.com/users/n3npq","html_url":"https://github.com/n3npq","followers_url":"https://api.github.com/users/n3npq/followers","following_url":"https://api.github.com/users/n3npq/following{/other_user}","gists_url":"https://api.github.com/users/n3npq/gists{/gist_id}","starred_url":"https://api.github.com/users/n3npq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/n3npq/subscriptions","organizations_url":"https://api.github.com/users/n3npq/orgs","repos_url":"https://api.github.com/users/n3npq/repos","events_url":"https://api.github.com/users/n3npq/events{/privacy}","received_events_url":"https://api.github.com/users/n3npq/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-17T16:42:44Z","updated_at":"2018-08-17T17:11:55Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"This RFE tries to provide a concrete example to known bottlenecks with rpm install/update mentioned in issue #517.\r\n\r\nMQTT is Yet Another Message Queue (YAMQ) like ZeroMQ, or AMQP, or M$ MQ developed by IBM with a client implementation maintained in the Apache Foundation.\r\n\r\nMessage queues are typically (ZeroMQ is an exception) implemented with a broker to ensure reliable delivery of messages in sequential order, using publish/subscribe methods for producers/consumers.\r\n\r\nWith MQTT the subscribe consumer is delivered a message through a callback on a different thread.\r\n\r\nOne approach to running scripts on a different thread would involve creating both a publish/subscribe mailbox within RPM, and sending the script to be executed through MQTT to be run asynchronously.\r\n\r\nThe subscribe code would take the message body (I.e. the rpm script to run) and either invoke /bin/sh or run embedded lua on a different thread. The return code would then be sent back to the original publisher to be collected.\r\n\r\nUsing MQTT in this fashion is just an obvious implementation of asynchronous RPC. The benefit comes from the simplicity by which the scriptlet runs on a different thread, and rpm execution can proceed without blocking on waitpid, nor implementing thread pools, or using locks, or worrying much about thread safety of rpmlib since MQTT messages hide all the gory details.\r\n\r\nI have working code for MQTT, the refactoring to achieve asynchronous execution is obvious, and I can provide an implementation, with measurements, if there is interest. Yadda yadda.\r\n\r\nThe real point of this RFE is to supply a concrete illustration of an alternative path to parallelism for issue #517 and to reask the question:\r\n\r\nWhat implementations are deemed acceptable to RPM?"},"comment":{"url":"https://api.github.com/repos/rpm-software-management/rpm/issues/comments/413930567","html_url":"https://github.com/rpm-software-management/rpm/issues/519#issuecomment-413930567","issue_url":"https://api.github.com/repos/rpm-software-management/rpm/issues/519","id":413930567,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMzkzMDU2Nw==","user":{"login":"n3npq","id":643926,"node_id":"MDQ6VXNlcjY0MzkyNg==","avatar_url":"https://avatars3.githubusercontent.com/u/643926?v=4","gravatar_id":"","url":"https://api.github.com/users/n3npq","html_url":"https://github.com/n3npq","followers_url":"https://api.github.com/users/n3npq/followers","following_url":"https://api.github.com/users/n3npq/following{/other_user}","gists_url":"https://api.github.com/users/n3npq/gists{/gist_id}","starred_url":"https://api.github.com/users/n3npq/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/n3npq/subscriptions","organizations_url":"https://api.github.com/users/n3npq/orgs","repos_url":"https://api.github.com/users/n3npq/repos","events_url":"https://api.github.com/users/n3npq/events{/privacy}","received_events_url":"https://api.github.com/users/n3npq/received_events","type":"User","site_admin":false},"created_at":"2018-08-17T17:11:55Z","updated_at":"2018-08-17T17:11:55Z","author_association":"CONTRIBUTOR","body":"To forestall the obvious objections to using MQTT, please note that I also have working code for SysV messages, POSIX message queues, AMQP, ZeroMQ, etc.\r\n\r\nI also am quite capable of an implementation of asynchronous RPC similar to above in XMLRPC, jabber, or UNIX domain sockets, or protocol du jour.\r\n\r\nThe type of transport used is not the issue.\r\n\r\nThe example provided provides an outline of a solution to a known problem in order to ask an important question.\r\n"}}
{"actor":{"display_login":"n3npq"}}