[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster-devel] Clarification on socket protocol exception..

Brad Clements wrote:
I am working on a new pure Python implementation of the xmlBlaster socket protocol

I have a question about the encoding of exceptions



This example is given

Example of an XmlBlasterException as a response on a publish() (a content is not shown):

" 84**E**17711*publish*oxf6hZs**QueueOverflow*The destination queue is full*0*"

qos key len content +-----*---------*-----*----------+

I think the "0*" trail is the 'len' == 0 which ends with '\0',
the content is nothing here.

I am wondering about that last * the one after the 0.

My reading of:

errorCode{string}, message{string}, byteDump of exception

makes me think that byteDump = "0*"

that is, the trailing null is part  of the byteDump.

Otherwise it serves no purpose, since

a) the userData is entirely one single exception "block"

b) userData length is determined from message length.

So, is that last * in the example a mistake, or will there really be an extra \x00 at the end of the byteDump?

Also in flag, it says:

90 (typically 'Z') The userData is compressed with gzip (see jzlib and zlib). Currently there is support to compress the whole socket stream - in such case this flag makes no sense and is not set

How is it that the entire socket stream is compressed? Where do I find the specification for that?
For zlib:stream:

No compression with those flags is implemented, the compression as described in
is done above the low level compression you mention.
The drawback with the current approach is that the same connection can not mix compressed/uncompressed messages
and does not auto-recognize if the message is compressed or not, the nice thing is that it is high efficient
and we could reuse it for other IO stream based protocols - and we have C/C++ and Java client implementations.

For C (or C++) see
and for Java see
(four classes for in and out and for block and stream mode)
They use the (java.net) socket.getInputStream() and socket.getOutputStream()
and intercept the flow of data to be compressed (see SocketExecutor.java:76 ff)

The implementations are the spec in this case, anyhow, any more
textual spec is welcome.

Does this mean I should not compress userData using zlib because it's not supported?
Yes, or you need to add the support server side.
I am looking at the config docs on this same page and trying to figure out how
compress/type = "zlib" can possibly work. That is, some messages are compressed and some are not. How can I split messages out of a tcp stream without some demarcation?
I never use it (but it works fine), but it seems it works like this:
The <i>zlib</i> compresses block wise and violates the
described SOCKET protocol format, it adds a prefix to the message format.
A first byte at the beginning, the <code>compressionFlag</code>, where '1' is zlib compressed and '0' is uncompressed.
This is followed by 4 bytes which contain the length of the uncompressed message.
If <code>compressionFlag==1</code>, there are another 4 bytes which contain the compressed length.
(see ZBlockOutputStream.java:85)
Note that everything is compressed, not only the userData.
I guess only zlib:stream can really work on tcp, and I would have to know in advance if I was "sending" everything zlib compressed, and if I was receiving compressed data. I guess compression must be the same in both directions.
It should probably work with different settings in both directions, though i haven't tried it.
how does compress/type = "zlib" work with compress/minSize = 500.. ? How can I tell if I'm receiving a compressed message and how many bytes it's going to be so I know when to stop blocking on a read?
It works as described above.
If 'A' adler32 checksum is not supported, why is it in the spec at all?
Yes, this could/shall be removed, it was more some thought what could be done with this compression flag.