Skip to main content

Pruning Deep Learning Architectures for Medical Applications

Supervisor

Suitable for

Computer Science and Philosophy, Part C
Mathematics and Computer Science, Part C
Computer Science, Part C

Abstract

Much of the unprecedented success of deep learning models can be attributed to the construction of increasingly large convolutional neural networks (CNNs), containing several parameters that can capture the complexity of growing datasets and diverse tasks. The trend has been that the larger the network, the better the performance. However, large CNNs are computationally demanding as they require more expensive and specialised GPU hardware, take longer to run, and require vast memory stores. These limitations make them difficult to redistribute, and thus impractical for many real-world applications. One such scenario is in transferring validated algorithms to operate on standard medical workstations or to mobile devices that interface with portable, point-of-care ultrasound probes.

The goal of this project is to devise strategies to compress the models in such a way that model size (parameter count) is reduced, while minimising the loss in accuracy. As part of this project, the student will explore network pruning strategies for intelligent model compression, thus enabling their application across medical imaging tasks. We will investigate the success of channel pruning, which involves removing entire filters from the network, thus explicitly making the model smaller during the training process. The student will investigate: (1) pruning criteria, to determine the functions that will best reduce tensor redundancy; (2) extent of pruning; and (3) identifying effective pruning locations.

This project is suitable for a student interested in deep learning, algorithm development, and image analysis. Programming experience is desirable (ideally Python).

References: To date, the OMNI group has explored model compression using separable convolution kernels (to reduce filter sizes) and knowledge distillation: a “teacher-student” network setup in which only the task-relevant weights learned by the larger teacher network are transferred to a much smaller student network. This preliminary work led to a reduction in network size, with negligible compromise in accuracy for processing ultrasound images on Android-based mobile devices.

• Low-Memory CNNs enabling real-time ultrasound segmentation towards mobile deployment Vaze, S, Xie, W, Namburete, AIL. IEEE JBHI 2020