Deep learning-based neural network for regions of interest retrieval in t2* – weighted brain perfusion MRI

Размещено на http://www.allbest.ru/

Размещено на http://www.allbest.ru/

National technical university of Ukraine «Igor Sikorsky kyiv polytechnic institute»

Deep learning-based neural network for regions of interest retrieval in t2* - weighted brain perfusion MRI

Alkhimova Svitlana Mykolaivna,

PhD, associate professor at the department of biomedical cybernetics

Diumin Oleksii Dmytrovych,

student of the 6^th course at the department of biomedical cybernetics

Kyiv

Abstract

Brain region segmentation is usually the first step for dynamic susceptibility contrast perfusion analysis. Although manual segmentation is more accurate, it is a time-consuming and not sufficiently reproducible process. Clinicians still rely on manual segmentation especially for cases with abnormal brain anatomy, as removing brain parts or inclusion of non-brain tissues can be a potential source of falsely high or falsely low values of perfusion parameters. This study proposes an effective deep learning-based neural network for fully automatic segmentation of brain from non-brain tissues in T2* - weighted magnetic resonance images with abnormal brain anatomy. Our neural network architecture combines U-Net and ResNet with plugged spatial and channel squeeze and excitation attention modules into the ResNet backbone. The train, validation, and test processes are conducted on 32 three-dimensional volumes of different subjects from the TCGA glioblastoma multiforme collection. Four performance metrics are used in our experiments: Dice coefficient, sensitivity, specificity, and accuracy. Quantitative results (i.e., Dice coefficient of 0.9726 ± 0.004, sensitivity of 0.9514±0.007, specificity of 0.9983±0.001, and accuracy of 0.9864±0.003) reveal that the proposed neural network architecture is efficient and accurate for brain segmentation. The obtained results also demonstrate that the training model using the proposed U-Net+ResNet architecture of the neural network provides the best Dice coefficient, specificity, and accuracy metric values compared to current methods under the same hardware conditions and using the same test dataset of magnetic resonance images of a human head with abnormal brain anatomy. Moreover, obtained results also indicate that the proposed U-Net+ResNet architecture of deep learning-based neural network could be good enough in a clinical setup to reduce the need for time-consuming and non - reproducible manual segmentation.

Keywords: brain, segmentation, region of interest, deep neural network, dynamic susceptibility contrast perfusion, magnetic resonance imaging

Анотація

Алхімова Світлана Миколаївна кандидат технічних наук, доцент кафедри біомедичної кібернетики, Національний технічний університет України «Київський політехнічний інститут ім. Ігоря Сікорського», м. Київ

Дюмін Олексій Дмитрович студент 6 го курсу кафедри біомедичної кібернетики, Національний технічний університет України «Київський політехнічний інститут ім. Ігоря Сікорського», м. Київ,

Нейронна мережа на основі глибокого навчання для отримання зон інтересу на т2* - зважених перфузійних зображеннях МРТ мозку

Сегментація ділянки мозку зазвичай є першим кроком в аналізі даних динамічно-сприйнятливої контрастної перфузії. Хоча ручна сегментація є більш точною, це трудомісткий і не відтворюваний процес. Клініцисти все ще застосовують ручну сегментацію, особливо у випадках обробки зображень з аномальною анатомією мозку. Це обумовлено тим, що вилучення з аналізу частин зображень мозку або урахування частин зображень, які відповідають не мозковим тканинам, може бути потенційним джерелом хибно високих або хибно низьких значень параметрів перфузії. У цьому дослідженні пропонується ефективна нейронна мережа на основі глибокого навчання для виконання повністю автоматичної сегментації ділянки мозку на Т2* - зважених магнітно-резонансних зображеннях з аномальною анатомією мозку. Запропонована архітектура нейронної мережі поєднує U-Net і ResNet із вбудованими модулями просторового та канального нагнітання-пригнічення механізму уваги у базові ResNet частині мережі. Навчання, валідація та тестування ґрунтується на використанні 32 тривимірних об'ємів даних від різних суб'єктів із колекції TCGA даних мультиформної гліобластоми. У наших експериментах використовуються чотири метрики ефективності сегментації: коефіцієнт Дайса, чутливість, специфічність та точність. Кількісні результати (коефіцієнт Дайса 0,9726 ± 0,004, чутливість 0,9514 ± 0,007, специфічність 0,9983 ± 0,001, точність 0,9864 ± 0,003) показують, що запропонована архітектура нейронної мережі є ефективною та точною для сегментації ділянки мозку. Отримані результати також демонструють, що навчена модель запропонованої U-Net+ResNet архітектури нейронної мережі забезпечує найкращі значення коефіцієнта Дайса, специфічності та точності порівняно до інших методів за тих самих умов апаратного забезпечення і використання того самого тестового набору магнітно-резонансних зображень людської голови з аномальною анатомією мозку. Крім того, отримані результати також вказують на те, що запропонована U-Net+ResNet архітектура нейронної мережі на основі глибокого навчання може бути достатньо хорошою, щоб використовуватися в клінічній практиці з метою зменшення потреб у трудомісткій і невідтворюваній ручній сегментації.

Ключові слова: мозок, сегментація, зона інтересу, нейронна мережа глибинного навчання, динамічно-сприйнятлива контрастна перфузія, магнітно-резонансна томографія.

Main part

Introduction. Segmentation of brain from non-brain tissues, more commonly known as skull-striping, plays an important role in the analysis of dynamic susceptibility contrast (DSC) perfusion magnetic resonance (MR) data [1].

The accuracy of brain segmentation directly affects the results of perfusion parameters assessment, since the presence of wrongly segmented pixels is a potential source of falsely high or falsely low values of perfusion parameters and visual artifacts on perfusion maps [2, 3]. Moreover, brain segmentation facilitates image registration between different MR imaging modalities and between computed tomography and MR scans [4], it is considered as a crucial step to differentiate brain tissues and detect brain tumors [5, 6].

Analysis of recent research and publications. To determine the brain perfusion region in MR images of a human head obtained with DSC perfusion examination, modern medical analysis software usually provides some tools. Among the possible options for conducting the segmentation procedure, there may be tools for manual, semi-automated, and fully automated approaches.

Although manual segmentation gives more accurate results, it is a timeconsuming and not sufficiently reproducible procedure. To obtain accurate results of brain segmentation, the operator must have sufficient knowledge of the anatomical structures of the human head visualized in MR images. However, the results, even those provided by an experienced operator, can suffer from subjective biases. Also, it is almost impossible to accurately reproduce the results of manual segmentation due to the subjectivity of human visual perception of MR data.

Such problems can be overcome by applying an automated approach to the segmentation procedure. It is also not error-free but makes it possible to reproduce the results.

Most of the algorithms that allow providing an automated approach to brain segmentation can be divided into two groups: those that are based on the analysis of pixel intensity, and those that use some templates [7, 8].

The algorithms that deal with pixel intensity analysis are different approaches to provide thresholding or clustering of image data. The main drawback of this group of algorithms is that most images with abnormal brain anatomy do not satisfy the proposed statistical assumptions.

The algorithms that deal with some templates are based on atlases or use neural network models. The issue with this group of algorithms is the lack of templates, i.e., marked atlases or data sets with predefined segments, the so-called ground truth images, for different age-sex-race-specific patients and different sizes, densities, and volumes of brain lesions.

To get a brain segment in T2 - or T2* - weighted MR images, the result of the exclusion of non-brain tissues should be accurate. The successful processing of such pixels is complicated as they are bright in T2 - or T2* - weighted compared to T1-weighted MR images. In some places, it is difficult to clearly define the boundaries between the brain and the fatty tissue presented between the brain and the skull. Indeed, it is the main reason for brain segmentation failures in T2 - or T2* - weighted images while applying algorithms developed primarily to process T1-weighted images [9].

Several algorithms of automated brain segmentation that can process T1-weighted as well as T2 - or T2* - weighted MR images have been proposed [10, 11]. The disadvantage of such algorithms is that they require proper estimation of input parameters to perform image processing. Also, it should be mentioned that the quality of segmentation of such algorithms degrades when they are applied to images with low resolution [5]. Therefore, their usage gives questionable quality as DSC MR imaging usually provides a low-resolution outcome.

The usage of transformation from T1-weighted to T2-weighted MR images [12] provides a possibility to apply segmentation algorithms initially developed to process T1-weighted images. However, this approach can lead to only partial extraction of studied objects data during the image transformation.

The need to change examination protocols that leads to exam time increasement prevents the usage of algorithms that utilize information from a specific imaging technique [9] or pairs of high-resolution T1 - and T2-weighted images [13]).

Applying the strategy to reduce the segmentation area to the approximate anatomical brain location (AABL) [14] partially improves segmentation by removing noisy pixels and parts of extracranial soft tissues. However, this strategy cannot effectively process cases with lesion locations close to the brain boundaries [3].

Cases with lesion locations close to the brain boundaries can be processed by using the CUSUM filter for boundary pixels [15]. Despite that, false activations of the algorithm iterative movements are observed when pixel intensities on the lesion edges are similar to pixel intensities of extracranial soft tissues.

BEaST [16] and MONSTR [17] are atlas-based algorithms that use template information about anatomical structures and can be applied to segment the brain in T2-/T2* - weighted MR images. As referred algorithms largely depend on the quality of the registration results with the template, it is problematic for their application to images with abnormal brain anatomy.

The availability of access to resources for powerful calculations has led to the development of deep learning algorithms. However, the proposed neural network architectures for brain segmentation are applicable for processing the MR images of healthy subjects [18] or with specific abnormal brain anatomy [19].

The aim of the article - is to design and implement the architecture of a deep neural network, which will be able to segment brain from non-brain tissues in T2* - weighted MR images of a human head obtained with DSC perfusion examination. To be applicable for clinical use, automatic segmentation must provide correct results for images with abnormal brain anatomy. So, the proposed deep neural network should be efficient for such image processing.

To achieve the aim, in the current study the following tasks have been set:

- to prepare a dataset of T2* - weighted perfusion MR images of a human head with abnormal anatomy to be used for deep learning purposes,

- to design deep learning-based neural network architecture and implement it using a deep learning framework,

- to determine the most appropriate hyperparameters for the final model,

- to assess the overall performance of brain segmentation in T2* - weighted perfusion MR images of a human head with abnormal anatomy using the proposed deep neural network.

Presenting main material. Deep neural networks have achieved success in a variety of machine learning application domains. In recent years, experimental confirmation of deep learning algorithms usage shows its state-of-the-art performance compared to traditional approaches. In particular, they are extremely successful in the fields of image processing including medical image segmentation. Thus, in this study, we use a deep neural network to segment brain from non-brain tissues because of its advantages.

The proposed approach combines U-Net [20] and ResNet [21] to provide brain segmentation on T2* - weighted MR images. To get a performance improvement, we plugged spatial and channel squeeze and excitation attention modules [22] into the ResNet backbone.

Dataset and preprocessing.

We considered only T2* - weighted MR images for training and prediction.

The results shown here are whole based upon data generated by the TCGA Research Network: http://cancergenome.nih.gov/. For our experiments, we use a subset of raw T2* - weighted DSC perfusion MR images of the TCGA glioblastoma multiforme collection.

The image processing software program was in-house developed to get ground truth images and provide some pre-processing steps, including motion correction, image resizing, and normalization. It is written in C# and uses an open - source EvilDICOM (http://rexcardan.github.io/Evil-DICOM/) to load medical images.

Ground truth images were manually marked brain regions by one experienced radiologist, and confirmed by another radiologist. Ground truth image creation was performed using the 4^th time-point image, on which signal intensity reached a steady state.

The 4^th time-point image was also used as a reference image against which all other images were aligned during applying motion correction.

To enable better convergence of the neural network solution, pixel data were normalized into the range [0-1] for each pixel.

In the experiment, we only used 32 three-dimensional volumes of different subjects. We applied a train-validation-test split of the dataset with the ratio of 68-12-20 commonly used in other works.

Implementation Details.

The proposed neural network was implemented using Tensorflow 2.9.2 library. We run the algorithm on an NVIDIA Tesla K80 with 12 GB of RAM running Linux.

To determine the most appropriate hyperparameters, we conducted a grid search. If we do not provide values of some hyperparameters below, it means that default values reported in Tensorflow 2.9.2 documentation were used.

The final U-Net + ResNet architecture is depicted in Fig. 1.

Fig. 1. The overall architecture of the proposed U-Net+ResNet neural network for brain segmentation on T2* - weighted MR images. The channel number is shown at the top of the box. The x-y size is shown at the lower left edge of the box. White boxes stand for copied feature maps. The arrows stand for different operations

The model training was done from scratch by Adam optimizer [23] with the initial learning rate of 0.00005, 01 = 0.9, 02 = 0.999. If the validation loss value had not been improved for 10 epochs, the learning rate was divided by 10. As a loss function for our binary classification task, we selected sparse categorical crossentropy.

The epoch number was 100 for the training process. The models were trained using mini-batches that had a size of 16 images. The training data were shuffled randomly at the beginning of every epoch.

We used GLOROT uniform initialization [24] to set the initial random values for all the weights of trained models.

To prevent overestimation and improve the generalizing ability of the model, data augmentation was applied during the training process. Data augmentation included random horizontal flipping and random image rotating in the range of [-15°, +15°] using the scipy.ndimage.rotate function with an order of 3 for spline interpolation settings [26].

Our network takes 2D images of 128 x 128 as inputs. Each input is a transversal view. The output of our network is an array with the probabilities of each category separately specified for each pixel of the input image. Finally, we provide a 2D array of category indexes of the most likely matching category.

Evaluation measurements.

To overall performance of brain segmentation, we calculated several evaluation metrics such as Dice coefficient, sensitivity, specificity, and accuracy. All of them are based on the fact that each image pixel can be assigned to one of the following categories: false positive (FP), false negative (FN), true positive (TP), and true negative (TN). FP pixels are defined in the resulting image as brain pixels, but they are non-brain pixels in the ground truth image. FN pixels are defined in the resulting image as non-brain pixels, but they are brain pixels in the ground truth image. TP pixels are defined in the resulting image as brain pixels, and they are brain pixels in the ground truth image. TN pixels are defined in the resulting image as nonbrain pixels, and they are non-brain pixels in the ground truth image.

The Dice coefficient provides information about the spatial overlap between segmented and ground truth regions of the brain. This metric value can vary from 0 (no overlap) to 1 (perfect agreement). The value of the Dice coefficient is calculated as follows:

DC =2 * TP

2 * TP + FP + FN.

The sensitivity provides information about the ability to correctly detect pixels, which belong to the brain segment. This metric value can vary from 0 to 1, where the higher value indicates a lower level of missed true pixels of the brain region. It is calculated as follows:

TP

SENS -----------=.

TP + FN

The specificity provides information about the ability to correctly detect pixels, which belong to the non-brain tissues segment (i.e., skull, extracranial soft tissues, and a large empty space surrounding the head). This metric value can vary from 0 to 1, where the higher value indicates a lower level of missed true pixels of the non-brain region. It is calculated as follows:

SPEC = TN

TN + FP.

The accuracy provides information about the rate of the pixels that are correctly identified with a segmentation algorithm. This metric value can vary from 0 (in case of non-agreement) to 1 (in case of complete agreement of the obtained result and ground truth regions). It is calculated as follows:

ACC = TP + TN

TP + FP + TN + FN

To assess the overall performance of brain segmentation using our proposed neural network architecture, we employed the best model for the test dataset.

We evaluated the performance of our proposed U-Net+ResNet neural network architecture in comparison to several algorithms for brain segmentation in T2 - or T2* - weighted MR images, including bi-level Otsu-based thresholding [25], low-intensity pixels extraction in the AABL region [14], CUSUM filter for boundary pixels [15]. All implementations were optimized to work with T2* - weighted MR images from the test dataset.

The complete evaluation results for the test dataset are shown in Table 1.

Brain segmentation performance comparison, mean ± standard

Table 1. Deviation (the best result in a given category is highlighted in bold)