Multicast based Middleware for Parallel Stream Processing

The paper presents the design and evaluation of ultimate stream: middleware that allows real-time delivery of streaming video data to all computers in a local multicast group. No special hardware or software support other than standard TCP/IP multicast over a local subnet is required. Frame level data shredding and reconstruction are used to provide consistency and synchronization without retransmission. A prototype scientific application (MouseCam) was implemented to evaluate our design in a practical context. The middleware layer is flexible enough to work with a number of different types of data, including sequential JPEG images, as used in this prototype. The MouseCam prototype demonstrates that our software can be effectively used as a backbone inside a private subnet to transfer large amounts of streaming data to multiple nodes in real-time for subsequent processing in parallel.