Enhancing Large-Scale Medical Image Processing in HPC Using Distributed Convolutional Neural Networks

Abstract

The demand for efficient and scalable processing techniques in the era of high-resolution medical imaging has grown significantly. This research introduces a distributed Convolutional type Neural-based Network (CNN)-based framework optimized for Higher-Performance Computation (HPC) environments to perform large-scale medical image processing better. The proposed system utilizes parallel computing architectures and data partitioning strategies to accelerate deep learning-based feature extraction and classification and executes multiple training tasks simultaneously on the different nodes within a cluster. A two-step approach of data parallelism with model parallelism is adopted for the deep neural network training, which in turn parallel manner in  the layers of a neural network and breaks down each layer of a network into a sub-network, with all nodes working together to perform the computation. Furthermore, the adaptive learning mechanism is also embedded to improve convergence and generalization across the different medical imaging modalities. Further agility is ensured by the dynamic workload scheduling strategy which guarantees the effective distribution of computational schools. A real-time processing data analysis system will help diagnose patients quickly and accurately, resulting in better and faster clinical decision making. By describing the capabilities of HPC, this method presents a scalable and efficient solution for medical image analysis, which in terms of speed and computational efficiency provides significant improvements as it is offered via the traditional centralized deep learning methods.