IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0185189.html
   My bibliography  Save this article

DOMe: A deduplication optimization method for the NewSQL database backups

Author

Listed:
  • Longxiang Wang
  • Zhengdong Zhu
  • Xingjun Zhang
  • Xiaoshe Dong
  • Yinfeng Wang

Abstract

Reducing duplicated data of database backups is an important application scenario for data deduplication technology. NewSQL is an emerging database system and is now being used more and more widely. NewSQL systems need to improve data reliability by periodically backing up in-memory data, resulting in a lot of duplicated data. The traditional deduplication method is not optimized for the NewSQL server system and cannot take full advantage of hardware resources to optimize deduplication performance. A recent research pointed out that the future NewSQL server will have thousands of CPU cores, large DRAM and huge NVRAM. Therefore, how to utilize these hardware resources to optimize the performance of data deduplication is an important issue. To solve this problem, we propose a deduplication optimization method (DOMe) for NewSQL system backup. To take advantage of the large number of CPU cores in the NewSQL server to optimize deduplication performance, DOMe parallelizes the deduplication method based on the fork-join framework. The fingerprint index, which is the key data structure in the deduplication process, is implemented as pure in-memory hash table, which makes full use of the large DRAM in NewSQL system, eliminating the performance bottleneck problem of fingerprint index existing in traditional deduplication method. The H-store is used as a typical NewSQL database system to implement DOMe method. DOMe is experimentally analyzed by two representative backup data. The experimental results show that: 1) DOMe can reduce the duplicated NewSQL backup data. 2) DOMe significantly improves deduplication performance by parallelizing CDC algorithms. In the case of the theoretical speedup ratio of the server is 20.8, the speedup ratio of DOMe can achieve up to 18; 3) DOMe improved the deduplication throughput by 1.5 times through the pure in-memory index optimization method.

Suggested Citation

  • Longxiang Wang & Zhengdong Zhu & Xingjun Zhang & Xiaoshe Dong & Yinfeng Wang, 2017. "DOMe: A deduplication optimization method for the NewSQL database backups," PLOS ONE, Public Library of Science, vol. 12(10), pages 1-17, October.
    • Handle: RePEc:plo:pone00:0185189
    DOI: 10.1371/journal.pone.0185189
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0185189
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0185189&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0185189?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Editors The, 2010. "Content," Basic Income Studies, De Gruyter, vol. 5(1), pages 1-1, September.
    2. Editors The, 2008. "From the Editors," Basic Income Studies, De Gruyter, vol. 3(1), pages 1-1, July.
    3. Editors The, 2008. "From the Editors," Basic Income Studies, De Gruyter, vol. 2(2), pages 1-3, January.
    4. Editors The, 2010. "Content," Basic Income Studies, De Gruyter, vol. 4(2), pages 1-1, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Laurent, Catherine E. & Berriet-Solliec, Marielle & Kirsch, Marc & Labarthe, Pierre & Trouve, Aurelie, 2010. "Multifunctionality Of Agriculture, Public Policies And Scientific Evidences: Some Critical Issues Of Contemporary Controversies," APSTRACT: Applied Studies in Agribusiness and Commerce, AGRIMBA, vol. 4(01-2), pages 1-6.
    2. Hsu, Dan K. & Burmeister-Lamp, Katrin & Simmons, Sharon A. & Foo, Maw-Der & Hong, Michelle C. & Pipes, Jesse D., 2019. "“I know I can, but I don't fit”: Perceived fit, self-efficacy, and entrepreneurial intention," Journal of Business Venturing, Elsevier, vol. 34(2), pages 311-326.
    3. Krueger, Norris & Bogers, Marcel L.A.M. & Labaki, Rania & Basco, Rodrigo, 2021. "Advancing family business science through context theorizing: The case of the Arab world," Journal of Family Business Strategy, Elsevier, vol. 12(1).
    4. Choi, James J. & Haisley, Emily & Kurkoski, Jennifer & Massey, Cade, 2017. "Small cues change savings choices," Journal of Economic Behavior & Organization, Elsevier, vol. 142(C), pages 378-395.
    5. Catherine Welch & Eriikka Paavilainen-Mäntymäki & Rebecca Piekkari & Emmanuella Plakoyiannaki, 2022. "Reconciling theory and context: How the case study can set a new agenda for international business research," Journal of International Business Studies, Palgrave Macmillan;Academy of International Business, vol. 53(1), pages 4-26, February.
    6. Weber, Ellen & Büttgen, Marion & Bartsch, Silke, 2022. "How to take employees on the digital transformation journey: An experimental study on complementary leadership behaviors in managing organizational change," Journal of Business Research, Elsevier, vol. 143(C), pages 225-238.
    7. Milazzo, M.F. & Spina, F. & Primerano, P. & Bart, J.C.J., 2013. "Soy biodiesel pathways: Global prospects," Renewable and Sustainable Energy Reviews, Elsevier, vol. 26(C), pages 579-624.
    8. Ravi KANBUR & Lucas RONCONI, 2018. "Enforcement matters: The effective regulation of labour," International Labour Review, International Labour Organization, vol. 157(3), pages 331-356, September.
    9. Ferreira, Manuel Portugal & Li, Dan & Guisinger, Stephen & Serra, Fernando A. Ribeiro, 2009. "Será o ambiente internacional de negócios o contexto efetivo para a pesquisa em negócios internacionais?," RAE - Revista de Administração de Empresas, FGV-EAESP Escola de Administração de Empresas de São Paulo (Brazil), vol. 49(3), July.
    10. Stephanie B Linek & Benedikt Fecher & Sascha Friesike & Marcel Hebing, 2017. "Data sharing as social dilemma: Influence of the researcher’s personality," PLOS ONE, Public Library of Science, vol. 12(8), pages 1-24, August.
    11. Robert E. Marks, 2010. "Welcome to SAGE Publications," Australian Journal of Management, Australian School of Business, vol. 35(1), pages 3-5, April.
    12. Petraud, Jean & Boucher, Stephen & Carter, Michael, 2015. "Competing theories of risk preferences and the demand for crop insurance: Experimental evidence from Peru," 2015 Conference, August 9-14, 2015, Milan, Italy 211383, International Association of Agricultural Economists.
    13. Jana Schmutzler & Edward Lorenz, 2018. "Tolerance, agglomeration, and enterprise innovation performance: a multilevel analysis of Latin American regions," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 27(2), pages 243-268.
    14. Baert, Stijn, 2015. "Hiring a Homosexual, Taking a Risk? A Lab Experiment on Employment Discrimination and Risk Aversion," IZA Discussion Papers 9536, Institute of Labor Economics (IZA).
    15. Dörrenbächer, Christoph & Gammelgaard, Jens & McDonald, Frank & Stephan, Andreas & Tüselmann, Heinz, 2013. "Staffing foreign subsidiaries with parent country nationals or host country nationals? Insights from European subsidiaries," Working Papers 74, Berlin School of Economics and Law, Institute of Management Berlin (IMB).
    16. Bonvecchi, Alejandro & Scartascini, Carlos, 2011. "The Presidency and the Executive Branch in Latin America: What We Know and What We Need to Know," IDB Publications (Working Papers) 3959, Inter-American Development Bank.
    17. Yip, Jeremy A. & Schweitzer, Maurice E., 2022. "Norms for Behavioral Change (NBC) model: How injunctive norms and enforcement shift descriptive norms in science," Organizational Behavior and Human Decision Processes, Elsevier, vol. 168(C).
    18. Richter, Barbara & Hanf, Jon H. & Bitsch, Linda & Yang, Yanlin & Li, Shuo, 2020. "Dragon Head Enterprises in China's Wine Production - The Example of Changyu," 60th Annual Conference, Halle/ Saale, Germany, September 23-25, 2020 305610, German Association of Agricultural Economists (GEWISOLA).
    19. Hemant Merchant & Ajai Gaur, 2008. "Opening the ‘Non-Manufacturing’ envelope: The next big enterprise for international business research," Management International Review, Springer, vol. 48(4), pages 379-396, April.
    20. Cyrine Ben-Hafaïedh & Mirela Xheneti & Pekka Stenholm & Robert Blackburn & Friederike Welter & David Urbano, 2024. "The interplay of context and entrepreneurship: the new frontier for contextualisation research," Small Business Economics, Springer, vol. 62(2), pages 571-582, February.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0185189. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.