Fault-tolerant middleware based on multistream pipeline for private storage services

This paper presents a fault-tolerant middleware for private storage services based on a client-server model. A client-side API split files into redundant chunks, which are encoded/decoded, anonymized, and distributed to different storage locations in parallel by using multistream pipeline. A chunk placement engine on the server-side API receives/delivers chunks from/to the client side API. In this model, the user preserves her files in-house and achieves data availability anytime from anywhere even in site failure scenarios while the organizations preserves the control of their storage resources. The experimental evaluation reveals the middleware can compensate the overhead produced on the client side by taking advantage of multiple cores commonly found in user´s devices. The users observe that the middleware produces similar performance to the private and public file hosting systems tested in the evaluation. The organizations observe a significant reduction of work in their servers. In site disaster scenarios, the middleware yields even better performance than fault-tolerant online distributed solutions tested in the evaluation.