Adaptive Ai-Driven Pluggable De-Duplication Algorithm for Optimized Data Management in Diverse Environments

Authors

  • E. Kanimozhi
  • T. Prabhu

Keywords:

AI-powered de-duplication, pluggable algorithms, machine learning, blockchain, cloud infrastructure, adaptive de-duplication, data efficiency, resource management, dynamic data analysis, scalable solutions

Abstract

In today’s rapidly evolving digital landscape, data duplication can significantly impede system efficiency and resource management. An adaptive AI-powered pluggable de-duplication algorithm proposed, designed to dynamically adjust to a wide range of computational environments, including blockchain networks and cloud infrastructures. The algorithm employs machine learning techniques to analyze various environmental factors, such as data size (from 1 GB to 100 GB), processing speed requirements and system architecture (e.g., sharing versus non-sharing environments). Based on this analysis, the algorithm selects the most suitable de-duplication method, ensuring it is both feasible and efficient for the given scenario. The proposed solution can evaluate multiple de-duplication techniques such as hash-based and chunk-based approaches by learning from historical datasets and real-time performance metrics. This allows it to achieve high accuracy and fast processing speeds. Its adaptive and pluggable nature makes it easily customizable for specific infrastructures, optimizing resource usage and ensuring seamless integration across various platforms. The algorithm continuously adjusts its strategy as system demands evolve, enhancing data management efficiency in a wide array of use cases in adaptive learning techniques.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

M. Ahmed, N. C. Debnath, and X. Zhou, “Adaptive Machine Learning-Based Data Deduplication for Cloud Storage Optimization,” IEEE Trans. Cloud Comput., vol. 11, no. 2, pp. 345–360, 2023, doi: 10.1109/TCC.2023.3048901.

P. Sharma and D. Singh, “Blockchain Data Bloat and Deduplication: A Comprehensive Review,” J. Distrib. Ledger Technol., vol. 4, no. 1, pp. 15–28, 2022, doi: 10.1007/s40860-022-00106-9.

T. D. Nguyen, T. N. Huynh, and Q. Nguyen, “A Hybrid Deduplication Strategy for Efficient Cloud Storage in Shared Environments,” J. Cloud Comput. Adv. Syst. Appl., vol. 12, no. 1, pp. 123–136, 2023, doi: 10.1186/s13677-023-00313-w.

J. Zhang, X. Li, and Y. Wang, “Scalable and Secure Deduplication in Blockchain for Distributed Ledgers,” IEEE Access, vol. 10, pp. 45910–45925, 2022, doi: 10.1109/ACCESS.2022.3175640.

H. Chen and Y. Wang, “Efficient Deduplication in Edge-Cloud Collaboration Using Machine Learning zechniques,” Future Gener. Comput. Syst., vol. 144, pp. 234–245, 2023, doi: 10.1016/j.future.2023.01.012.

Z. Li, K. Xu, and C. Luo, “Secure Deduplication for Blockchain: A Cross-Chain Perspective,” Blockchain: Res. Appl., vol. 4, no. 3, p. 102027, 2023, doi: 10.1016/j.bcra.2023.102027.

N. Kumar and D. Prakash, “Adaptive Deduplication Methods for Scalable Cloud Data Management,” J. Syst. Softw., vol. 197, p. 111377, 2023, doi: 10.1016/j.jss.2022.111377.

M. Sun, X. Zhao, and S. Zhang, “Cloud Data De-Duplication for Large-Scale Storage Systems: A Machine Learning Approach,” J. Cloud Comput. Adv. Syst. Appl., vol. 12, no. 1, pp. 93–106, 2023, doi: 10.1186/s13677-023-00308-x.

M. Ramesh and B. Srinivasan, “Survey on Deduplication Techniques in Cloud Computing for Storage Efficiency,” J. Cloud Comput. Adv. Syst. Appl., vol. 11, no. 2, pp. 45–56, 2022, doi: 10.1186/s13677-022-00229-y.

L. Qiu, P. Wang, and H. Dai, “On Deduplication in Non-Sharing Cloud Environments: Design and Optimization Approaches,” IEEE Trans. Cloud Comput., vol. 11, no. 1, pp. 127–140, 2023, doi: 10.1109/TCC.2023.3049319.

K. Park, H. Lee, and W. Jung, “Blockchain-Based Secure Deduplication Mechanism for Cross-Cloud Data Sharing,” J. Inf. Secur. Appl., vol. 72, p. 102958, 2022, doi: 10.1016/j.jisa.2022.102958.

R. P. Rao and P. Mahajan, “Machine Learning in Cloud Deduplication for Efficient Storage Optimization: A Review,” ACM Comput. Surv., vol. 55, no. 3, pp. 41–58, 2022, doi: 10.1145/3503884.

Y. Wu, F. Chen, and H. Luo, “Energy-Efficient Deduplication in Blockchain Systems Using Adaptive Data Chunking,” J. Parallel Distrib. Comput., vol. 175, pp. 59–70, 2023, doi: 10.1016/j.jpdc.2023.03.003.

Z. Tang, J. Liu, and X. He, “Deduplication in Blockchain-Based Cloud Storage: A Comprehensive Survey,” Inf. Syst. Front., vol. 25, no. 1, pp. 111–126, 2023, doi: 10.1007/s10796-023-10429-1.

Z. Wang, Y. Gu, and Q. Zhang, “Deduplication Methods for Cloud Storage: A Comparative Review,” IEEE Trans. Cloud Comput., vol. 10, no. 4, pp. 976–989, 2022, doi: 10.1109/TCC.2022.3054217.

H. Yu, Y. Xu, and J. Li, “Blockchain-Aided Deduplication for Secure Cloud Data Storage,” Future Gener. Comput. Syst., vol. 130, pp. 66–78, 2022, doi: 10.1016/j.future.2022.08.001.

G. Mahesh and M. Rao, “Deep Learning-Based Deduplication Techniques for Optimizing Cloud Data Storage,” J. Big Data, vol. 10, no. 1, pp. 45–56, 2023, doi: 10.1186/s40537-023-00688-7.

H. Ju and Z. Yang, “Blockchain Data Deduplication Using Off-Chain Storage for Secure and Scalable Ledger Management,” J. Distrib. Ledger Technol., vol. 6, no. 2, pp. 202–216, 2022, doi: 10.1007/s40860-022-00119-2.

F. Alomari and I. Khalil, “AI-Driven Data Deduplication Techniques for Enhancing Performance in Cloud Environments,” ACM Trans. Cloud Comput., vol. 6, no. 3, pp. 41–52, 2023, doi: 10.1145/3583467.

P. Patel and K. Singh, “Blockchain Storage Optimization through Deduplication: Challenges and Opportunities,” J. Blockchain Res., vol. 7, no. 3, pp. 95–110, 2023, doi: 10.1016/j.bcra.2023.102012.

M. Hassan and A. Ali, “Secure Deduplication in Hybrid Cloud Storage Using Blockchain,” Future Internet, vol. 15, no. 1, pp. 28–39, 2023, doi: 10.3390/fi15010012.

L. Tran and H. Phan, “Deduplication Techniques for Edge-Cloud Collaboration in IoT Systems,” J. Cloud Comput., vol. 11, no. 4, pp. 45–61, 2022, doi: 10.1186/s13677-022-00321-1.

P. Suri and P. Malhotra, “Machine Learning-Assisted Cloud Storage Deduplication for High-Performance Data Management,” Int. J. Cloud Comput., vol. 12, no. 2, pp. 179–195, 2023, doi: 10.1504/IJCC.2023.123456.

R. Brown and T. Singh, “Deduplication Approaches for Efficient Blockchain Storage: A Survey,” J. Inf. Commun. Technol., vol. 7, no. 2, pp. 201–220, 2022, doi: 10.1007/s40998-022-00388-6.

Downloads

Published

2025-05-10

How to Cite

1.
Kanimozhi E, Prabhu T. Adaptive Ai-Driven Pluggable De-Duplication Algorithm for Optimized Data Management in Diverse Environments. J Neonatal Surg [Internet]. 2025May10 [cited 2025Oct.19];14(18S):973-80. Available from: https://www.jneonatalsurg.com/index.php/jns/article/view/5512

Issue

Vol. 14 No. 18S (2025): Journal of Neonatal Surgery

Section

Original Article

License

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

You are free to:

  • Adapt — remix, transform, and build upon the material for any purpose, even commercially.

Terms:

  • Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • No additional restrictions — You may not apply legal terms or technological measures that legally restrict others from doing anything the license permits.