[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xmlblaster] ssl and compression with SOCKET

Marcel Ruff wrote:

There are many variants of compression.

2. Compress in the SOCKET protocol plugin

 This is your way. The SOCKET spec supports it with a compression flag
 and a 'lenUnzipped' field.
 + Simple
 ++ Compresses everything, Key+content+Qos and even MsgUnit[] in a bulk
 - The CPU overhead in the xmlBlaster server increases for each subscriber
   as the received message is uncompressed on arrival and needs to be
   compressed for each subscriber separately.

Right. What it currently does not to support is autodetection and runtime setting of compression type (per-connection).
My idea of how this compression negotiation should work is:
- the client can choose any compression type it wants when it connects, as the decompression is not as CPU intensive as compression.
- the server could offer the client a compression type (chosen somehow dynamically), which the client would acknowledge, or deny. In the latter case the server should fall back to other method(s).

3. Compress in MessageUnit.setContent()

 Here the Key and Qos is transferred uncompressed and only the
 message content is compressed. We could support this in our
 C/C++/Java MessageUnit struct. We could use a ClientProperty "__gzip"
 to mark it.
 + The xmlBlaster server would never uncompress the content
   (as it never looks into it) when receiving
   it and forwarding it to the subscribers.
 + This runs with all protocol plugins.
 ++ No CPU overhead
 - Key and Qos are not compressed

Also a good alternative to the CPU intensive compression outlined above, but we would also lose the ability to compress repeating patterns across messages, as every message needs to be one standalone block of data.


The solutions 2 and 3 are probably our ways to go.

Configuration: --------------

Typically only publish() and update() and get()-return (and their Array & Oneway variants)
need compression.
Other requests like connect(), disconnect(), subscribe(),
unSubscribe(), ping() and erase() don't need compression.

Your configuration switches on/off compression for a publisher
as a whole.
Updates for all subscribers are always delivered uncompressed OR compressed depending
on the SOCKET plugin configuration in the server.

In a future step we will add a <compress type="gzip"/> flag to PublishQos and
to SubscribeQos to have fine grained control.

OK, but this again is only easily implemented in the 3rd variant. Though it can be done with the 2nd variant, too.

I don't understand this. Do you say that the message stream
is compressed on the fly as bytes are pushed in?
I'll look into the code you send to understand it.

Exactly. Its just a filtered stream.