

Re: [Pulp-list] Messaging Questions

On 07/08/2010 09:51 AM, Jason Dobies wrote:
Hash: SHA1

Yes it does.  If we stick with only synchronous requests to that agent
and leave asynchronous stuff to the pulp Task engine, temporary
(non-durable) queues will be a good approach to pruning dead queues.

One more question on this and I'll shut up until the voice call. If all
requests are synchronous, is this gonna scale? What happens when we fire
off messages to 10K agents and/or if those requests take minutes (if not
more) to execute? Even if it's executing in the async task queue, won't
this be a fierce burden on the server (it may not be, I'm kinda thinking
out loud here).

If the thinking is that:

asynchronous = pub/sub = efficiency

Then, an important consideration is: How does qpid implements *durable* subscription to topics (pub/sub)? The durable nature of the subscription usually requires that brokers implement using queues where messages are routed to subscriber queues based on topic/subject and selectors. Asynchronous request/response assumes guaranteed delivery. This means that the message must be queued so it can be delivered to consumers that are not connected when the message is published. This is the definition of durable subscription. So, publishing a message to 10k agents probably still requires 10k queues.

But, if we anticipate this kind of mass operation, then asynchronous will be much more efficient and worth the extra complexity because we can have all the agents performing the operation in parallel. If we did this synchronously, we'd be limited to the Task thread limit.

- --
Jason Dobies
RHCE# 805008743336126
Freenode: jdob
Version: GnuPG v2.0.14 (GNU/Linux)


Pulp-list mailing list
Pulp-list redhat com

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature
