[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster] SOCKET problem, why are messages delayed?

PBal wrote: 
Hi!

I've chosen XmlBlaster because of its wealth of features, simple tcp socket protocol support, compatibility and possible ease of use, and it looked good. However...
What i'm currently trying to write is a simple rpc framework in java for a quite complex commercial program being developed.
XmlBlaster is simply forwarding volatile PtP messages between two sessions. I've been trying to debug a problem for several days, after applying the ping patch for the SOCKET protocol.
The issue is that sometimes invocations of remote object's methods (on "host A") are not returning their return value, or rather, the messages don't arrive at "host B" immediately as "host A" sent published it. No other messages are being sent by any other host, traffic is zero.
This causes "host B" to get stuck in the invocation while waiting for the message containing the return value.
I noticed that when I wait 60 seconds, some warning messages appear on the xmlBlaster server:

[WARN XmlBlaster.DispatchWorkerPool.asus-1 requestBroker-/node/asus] Generating dead message 'callback:/node/asus/client/server/1/HIGH/1092849962617000000/rpc' from publisher=/node/asus/client/poka/1 because delivery with queue 'callback:/node/asus/client/server/1' failed: XmlBlasterException errorCode=[user.update.error] serverSideException=true location=[SOCKET-HandleClientRequest-server] message=[SOCKET callback of 1 messages failed : errorCode=resource.exhaust message=Timeout of 60000 milliseconds occured when waiting on update(server:6024) response. You can change it with -dispatch/callback/plugin/socket/responseTimeout <millis>] [See URL http://www.xmlblaster.org/xmlBlaster/doc/requirements/admin.errorcodes.listing.html#user.update.error]

[INFO XmlBlaster.DispatchWorkerPool.asus-1 XmlKey] DOM parsed the XmlKey __sys__deadMessage
[INFO XmlBlaster.DispatchWorkerPool.asus-1 TopicHandler-/node/asus/topic/__sys__deadMessage] New topic is ready, history/maxEntries=10, persistence/msgUnitStore/maxEntries=100000000
[WARN XmlBlaster.SOCKET.HandleClient-server SocketExecutor] Ignoring received 'update' message id=server:6024, nobody is interested in it 

It seems that the message with id "server:6024" is delivered to your
client (called 'server/1') and this client blocks in its update() method.
After the timeout (60 sec) suddenly the update() returns.

I believe it is a multi threading dead lock issue in your client code.

Could you please add logging output to your clients update() method
to see if the update arrives and to see if/when the update() returns?

Further you can add the option '-dump[socket] true' on xmlBlaster and
on your client side to see all messages sent and acknowledged..

During the blocking (0 ... 60 sec) you can make a thread dump
(on Linux killall -3 java, on Windows 'Ctrl-Break') to find
out where the dead lock happens.

Probably you can let return CallbackMultiplexer directly in update()
(comment out your business logic) and check if the problem disappears.


I don't understand the meaning of these messages, and assume that their appearance is not normal and has something to do with my problem.

My code to set up the connection to xmlBlaster on both clients is as follows:

String[] args = {
"-protocol", "SOCKET",
"-dispatch/connection/plugin/socket/hostname", host, "-dispatch/connection/plugin/socket/port", String.valueOf(port)};
Global global=new Global(args);
xmlBlasterConnection = global.getXmlBlasterAccess();
ConnectQos qos = new ConnectQos(global);
qos.setSessionTimeout(3600000L);
qos.setPtpAllowed(true);

Address address=new Address(global);
address.setDelay(10000L);
address.setPingInterval(10000L);
address.setRetries(-1);
qos.setAddress(address);

    CallbackAddress cba=new CallbackAddress(global, "SOCKET");
    cba.setDelay(10000L);
    cba.setPingInterval(5000L);
    cba.setRetries(-1);

    qos.addCallbackAddress(cba);

    if (session>=1)
            qos.setSessionName(new SessionName(global, user+"/"+session));
        else
            qos.setSessionName(new SessionName(global, user));
        qos.loadClientPlugin("htpasswd", "1.0", user, pass);

        CallbackMultiplexer cbmpx=new CallbackMultiplexer();
        ConnectReturnQos crq=xmlBlasterConnection.connect(qos, cbmpx);

CallbackMultiplexer implements I_Callback.
I'm not using the get, subscribe or unsubscribe methods, only publishOneway. Receiving messages with CallbackMultiplexer.
Hope this helps to track down my error, misunderstanding or anything. 

This code looks OK.

regards

Marcel 

Any help is greatly appreciated.

Regards,
Balázs Póka




--
http://www.xmlBlaster.org