TIBCO Rendezvous RVD Daemon Remote Memory Leak DoS

The TIBCO Rendezvous RVD daemon is vulnerable to a memory leak, which when remotely triggered, prevents any further RV communication until the daemon is manually restarted.

Vulnerability Type / Importance: Remote DoS / High
Workaround: There are no known workarounds for this vulnerability

The RV daemon (RVD) within TIBCO’s Rendezvous messaging product is responsible for the communication of messages between RV-enabled applications. The vulnerability exists as the result of an error in the code that parses information within one of the headers in a TIBCO proprietary network protocol packet.
Technical Details:
Within a Rendezvous “wire format” TCP packet, the first four bytes represent the number of bytes of data to expect within the packet, for example:
“0000007c” //total length of data in packet
“9955eeaa” // “magic” number
“06″ // number of following bytes including null
“6d7479706500″ //the text “mtype”
…etc

In the above example the number of data bytes in the packet is “0x7c”, or 124 bytes. If this value is set to zero in a packet sent to the RVD daemon then it stops responding to all subsequent communication. This appears to result from a memory leak, which continues to attempt to allocate memory. Eventually, operating system alert messages start to appear, warning that the virtual memory in the underlying operating system is running low.