Re: [xmlblaster] Design Questioin

[Rob McDonald <rob.mcdonald at resurgent-tech.com>]

Marcel/Michele,

I really appreciate the info below!

I was digging around in the ConsumableQueuePlugin and *Worker source
today and noticed the "round robin" list removal and addition approach
mentioned below. 

I also ran the "javaclients.HelloWorldSubscribe" and
"javaclients.HelloWorldPublish -numPublish 50 -consumableQueue true"
demo tonight with 4 subscribers, one publisher and I did not witness the
behavior I expected.  It was my understanding that when a message is
published on a topic with a ConsumablePlugin distributor that it will
only be sent to one of the active/alive subscribers? (got demo
instructions from
http://www.xmlblaster.org/xmlBlaster/doc/requirements/msgDistributor.plugin.ConsumableQueue.html
)

However, when I send the 'hello' message with the demo, all 4 of the
subscribers received it.  I took a look at the HelloWorldPublish.java
source and did not notice any location where there was a call to
<topicProperty>.setMsgDistributor() to set the ConsumablePlugin nor do I
see any place where it is grabbing the init arg Property
"-consumableQueue true" from the Global.

I am very sorry if this is a ignorant question, I am still learning.

Thank You,
Rob



Marcel Ruff wrote:
> Rob McDonald wrote:
>> hmmm, the consumablequeue seems like the way to go as each audit server
>> will carry out the same set of tasks as the other audit servers.  They
>> are simply used for distributing the workload over multiple servers.
>>
>> However, there does not seem to be a very good way to allow even load
>> balancing among the servers as the message is just sent to the first
>> available subscriber.
>>   
> I think Michele has chosen the the round robin approach
> (ConsumableQueuePlugin.java:329)
> if a client fails, but using round robin for alive clients is not yet
> implemented.
>
> The nice thing with the plugin is that it detects unavailable audit
> services and
> is not using them anymore.
> If you code the logic in your service manager you
> need to detect yourself a broken audit service (for example by
> listening on logout events or by
> heart beats send by your audit servers).
>
> One option is that you extend our ConsumableQueuePlugin.java with some
> load balancing algorithm (like round robin)
> and we could take this into our distribution.
>
>> Would I have to modify this ConsumableQueue plugin to apply my message
>> delivery logic(even load balancing)?  or, should I simply maintain a
>> separate datasource on the management server that keeps track of what
>> jobs each audit server has, and then simply publish the job to a server
>> via a point to point message to provide the load balancing(obviously
>> making a load balanced decision from my separate datasource of currently
>> active jobs on the servers).
>>
>> As for your second option(Operation/Startup), I will have to study a bit
>> more, as I do not fully understand it.
>>   
> The 'Startup' is probably not important anymore, as all the audit
> servers do the same.
> It was just a suggestion how you could use xmlBlaster as a naming
> service for all your
> different audit servers.
>
> regards
> Marcel
>> I really appreciate your reply!
>>
>> Thanks,
>> Rob
>>
>> Marcel Ruff wrote:
>>  
>>> Rob McDonald wrote:
>>>    
>>>> I am still trying to pile through xmlblaster to determine all that it
>>>> can do, so I apologize if this is a stupid series of questions.
>>>>
>>>> I am working on an application that consists of three components: a
>>>> client, a management server, and a audit server.  The clients are used
>>>> to log in(to the management server) with various access levels to
>>>> schedule events, download completed audits, add users, etc.  The
>>>> Management server is used to process client requests, accept new
>>>> audits/commands from the clients, distribute commands/audits evenly to
>>>> the audit servers, and report responses.  The audit server, accepts
>>>> commands, performs actions, and reports results.
>>>>         
>>> Hi Rob,
>>>
>>> are the audit servers identical - they are just for load balancing /
>>> redundant fail over?
>>> In this case the consumable queue plugin is for you, see
>>> http://www.xmlblaster.org/xmlBlaster/doc/requirements/msgDistributor.plugin.ConsumableQueue.html
>>>
>>>
>>>
>>> If each audit server is specialized on different tasks the above is
>>> not useful.
>>>
>>> The general setup could be:
>>>
>>> A) Startup:
>>>
>>> 1. On startup each audit server subscribes persistently on a topic
>>>   "com.resurgent-tech.auditserver.request.<myAuditServerName>",
>>>  for example the 'accounting' audit server subscribes on:
>>>   "com.resurgent-tech.auditserver.request.accounting"
>>>   which will deliver the requests coming from the service manager.
>>>   For load balancing (if you have multiple 'accounting' audit servers)
>>>   the subscribe could configure the consumable queue plugin.
>>>
>>> 2. On startup all audit servers publish themselves persistently
>>> (prio=HIGH, no PtP) to separate topics
>>>   "com.resurgent-tech.registration.auditserver.<myAuditServerName>",
>>>   probably with information what they are capable to do.
>>>  Example: "com.resurgent-tech.registration.auditserver.accounting"
>>>
>>> 3. The service manager subscribes on
>>> 'startswith(com.resurgent-tech.registration.auditserver.)'
>>>  so he knows about the available audit servers (XPath subscription).
>>>
>>> 4. The service manager subscribes persistently on topic
>>> "com.resurgent-tech.manager.request"
>>>    which is used to receive requests from clients.
>>>
>>> Note: All audit services and the service manager should login fail
>>> save with a positive session id
>>>  like loginName="servicemanager/session/1" or "accounting/session/1"
>>>  and "-dispatch/connection/retries -1" and
>>>  a big enough callback queue. They should use persistent subscribes to
>>> never loose any message.
>>>
>>>
>>>
>>> B) Operation:
>>>
>>>
>>> 1. Clients publish transient requests (with a reasonable expiration
>>> time) to "com.resurgent-tech.manager.request"
>>>
>>> 2. The service manager processes the request and forwards the task to
>>> one of the audit servers using
>>>  a topic like "com.resurgent-tech.auditserver.request.accounting".
>>>   He adds a client property with the clients address (for example
>>> 'joe/session/-12')
>>>  which is bounced back by the audit servers when sending their
>>> responses to the service manager.
>>>  Like this the service manager knows for whom the response is, and
>>> remains stateless which reduces complexity.
>>>
>>> 3. The service manager receives the response, does some additional
>>> value enhancement (if needed)
>>>  and sends a transient PtP response message to the client (with a
>>> reasonable expiration time).
>>>  You could choose a response topic
>>> "com.resurgent-tech.client.response.joe"
>>>  to avoid creating/destroying many temporary topics (for the PtP
>>> messages).
>>>
>>> There can be many variations of the above scenario, depending on your
>>> setup and demands,
>>> for example the client could choose to not receive the responses
>>> asynchronously - by using the
>>> request/response pattern (thus blocking in the request until the
>>> response arrives).
>>>
>>> enjoy
>>> Marcel
>>>
>>>    
>>>> Now, my question is really with respect to the management server.
>>>>
>>>> I wish to "send" commands to the management server to be destined for
>>>> the audit servers, however, I want the management server to apply some
>>>> "logic" as to which audit server gets the action request.  So, finally
>>>> to my question:
>>>>
>>>> Should I publish a "management" channel via the management server from
>>>> which clients can issue commands, and then request point to point
>>>> channels to the servers so I can poll information and send specific
>>>> commands to each based on my "logic"(I need two way comm between my
>>>> audit servers and the management server, however, I don't necessarily
>>>> need broadcast capabilities, only point to point).
>>>>
>>>> I hope this email is not too "abstract" to get my general design
>>>> question across, if it is, let me know and I will apply more detail.
>>>>
>>>> Thank You,
>>>> Rob
>>>>
>>>>         
>>>
>>>     
>>
>>
>>   
>
>
>