Multi-Version Managers for Large Scalable Data-Management Systems

With the emergence of data-intensive computing, which is due to the growth of the data produced and generated each day, it became necessary to store and manage big data. Cloud data storage is actually the best choice for large distributed systems. Successful Cloud Computing cannot be achieved without a reliable data-management system to store and handle the enormous volume of data. Management of the available storage system at large scale becomes progressively more complicated, and we face many challenges, such as scalability, data availability, fault tolerance, etc. Also, data storage is faced with specific access patterns: highly concurrent reads of data from the same file, many overwrites, and very concurrent appends to the same file. Most of the existing storage systems use versioning to bring and enhance data access parallelism and this enables better performance levels under concurrency; but, generally, these systems use one component (version manager), which is responsible for generating new versions of each file stored. When we speak in the context of big data, the requests for read, write and append increase. If these requests are managed by a single component, then we have a performance bottleneck and an overloaded version manager. To avoid this drawback, we proposed and designed a new architecture of storage systems that uses versioning; the new architecture uses multi-version managers to support better the scalability and provide partial fault tolerance. To illustrate the practicability of our approach, we assessed it on the BlobSeer data-management system. The experimental results demonstrate that our architecture achieves near-linear scalability for CREATE operations (495 ops/s per additional version manager), reduces WRITE execution time by up to 66%, and maintains 67% availability under single-node failures, all while introducing minimal resource overhead (3% aggregate CPU increase). These results confirm that the proposed multi-version manager architecture offers a practical, scalable, and partially fault-tolerant solution for Cloud data-storage systems.