A classification of MRI brain tumor based on two stage feature level ensemble of deep CNN models

https://doi.org/10.1016/j.compbiomed.2022.105539Get rights and content

Highlights

  • Present a new two-stage ensemble approach for multi-classification of brain tumors.

  • CNN models and classifiers are selected through experiments and trials to build the two-stage ensemble model.

  • Model's robustness is assured by three different experiments on the datasets.

  • Evaluation and validation are done through comparative analysis and real-time performance.

Abstract

The brain tumor is one of the deadliest cancerous diseases and its severity has turned it to the leading cause of cancer related mortality. The treatment procedure of the brain tumor depends on the type, location and size of the tumor. Relying solely on human inspection for precise categorization can lead to inevitably dangerous situation. This manual diagnosis process can be improved and accelerated through an automated Computer Aided Diagnosis (CADx) system. In this article, a novel approach using two-stage feature ensemble of deep Convolutional Neural Networks (CNN) is proposed for precise and automatic classification of brain tumors. Three unique Magnetic Resonance Imaging (MRI) datasets and a dataset merging all the unique datasets are considered. The datasets contain three types of brain tumor (meningioma, glioma, pituitary) and normal brain images. From five pre-trained models and a proposed CNN model, the best models are chosen and concatenated in two stages for feature extraction. The best classifier is also chosen among five different classifiers based on accuracy. From the extracted features, most substantial features are selected using Principal Component Analysis (PCA) and fed into the classifier. The robustness of the proposed two stage ensemble model is analyzed using several performance metrics and three different experiments. Through the prominent performance, the proposed model is able to outperform other existing models attaining an average accuracy of 99.13% by optimization of the developed algorithms. Here, the individual accuracy for Dataset 1, Dataset 2, Dataset 3, and Merged Dataset is 99.67%, 98.16%, 99.76%, and 98.96% respectively. Finally a User Interface (UI) is created using the proposed model for real time validation.

Introduction

Brain is the significant part of our central nervous system that controls all our functionalities through a huge number of connected neurons [1]. Any malfunction or abnormality in the brain cells affects the organs connected to the corresponding part of the brain, consequently damaging the functionalities of that organ. Cancer originating in the brain and other nervous system is considered to be the 10th leading cause of death. The 5-year survival rate of the patients having cancerous brain is only 36% [2]. As brain tumor is caused by the unnatural and uncontrolled growth of brain cells, its severe consequences can be life-threatening. Around 400,000 people are affected by brain tumor and 120,000 people have died in the past years all over the world, as reported by World Health organization (WHO) [3]. Early and proper detection can play an indispensable role in increasing the survival rate by accelerating the treatment process [4]. Manual detection of brain tumors can be tedious, time consuming and erroneous due to the variations on types and sizes. Proper and precise detection needs expertise and it's even harder for complicated cases. Hence, besides human inspection, we can't avoid the necessity of an automated process for precise detection and classification of brain tumors. Deep learning and convolutional neural network can significantly accelerate the whole diagnosis process making the classification task automated and conscientious.

The emerging technologies of machine learning and deep learning have profoundly developed different fields of applications [[5], [6], [7]]. Particularly, a huge scope has been created in medical image processing, and numerous researches are ongoing for enhancing this research area. Automating the process of brain tumor segmentation and classification is a significant part of this research field.

Recently, deep learning technology has become a very popular choice in brain tumor classification from brain MRI images. For example, Gumaei et al. [1] introduced a hybrid method for feature extraction called PCA-NGIST. Then, from the extracted features, brain images are classified into three categories (meningioma, glioma, pituitary) using Regularized Extreme Learning Machine classifier and achieved an accuracy of 94.233%.They worked with only one dataset and didn't included any tumor-free or normal brain images. Tandel et al. [8] proposed a deep learning model based on CNN for classification of brain tumor from five individual multi-class datasets. The highest achieved accuracy was 96.65% but they didn't added any analysis on model's robustness.

Sajjad et al. [9] proposed an architecture, where they used InputCascadeCNN for tumor segmentation and fine-tuned VGG-19 for three types of tumor classification, where the achieved accuracy was 94.58% for brain tumor dataset. Alqudah et al. [10] proposed a deep learning technique for classification of three types of tumor, where the carried out experiments on cropped, uncropped and segmented images. They achieved the average accuracy of 98.93%, 99% and 97.62% for cropped, uncropped and segmented images, respectively. Their dataset doesn't include any normal brain images and more analysis could have been done on the proposed architecture. Deepak et al. [11] used transfer learning approach, where fine-tuned GoogleNet was used for classification of three types of brain tumor. The overall accuracy was 98%. In Refs. [10,11], a particular dataset has been considered using which it is very much possible to achieve a good accuracy by optimizing a model but this certainly does not guarantee the model's robustness for other random data. Moreover, in Ref. [11] there was a considerable misclassification of “meningioma” class and had an overfitting tendency.

Pashaei et al. [12] used combination of CNN and Extreme Learning Machine for classifying tumors into three classes achieving an accuracy of 93.68% but Some other recent approaches attained better accuracy. Irmak et al. [13] proposed three different CNN models for three datasets containing different types of brain tumors. The models individually attained accuracy of 99.33%, 92.66%, and 98.14% for the three datasets, respectively. The accuracy is very high for binary classification but in multi-classification they couldn't achieve that much accuracy.

Balasooriya et al. [14] proposed a less complex CNN model for classifying five types of brain tumors, which resulted in an accuracy of 99.69%. They didn't provide much analysis on the performance of their model and didn't include any comparative study with other models. Das et al. [15] presented a comparatively shallow CNN based model for the classification of three types of brain tumors, which could attain an accuracy of only 94.39%. For increasing generalization capability this could be implemented on other datasets including normal brain images. Afshar et al. [16] proposed Capsule Network architecture for three tumor types classification namely glioma, meningioma and pituitary, where they got an accuracy of 90.89%.Compared to other existing models, their attained accuracy is not much satisfactory.

Hemanth et al. [17] introduced a modified deep CNN architecture to address the computational complexity of deep CNN model, and classified brain tumors into four classes. In this way, they achieved an average accuracy of 96.4%. Any comparative analysis with state-of-the-art models have not been showed in this paper. Badža et al. [18] presented a new CNN architecture for classifying three types of brain tumors. After 10-fold cross validation, they achieved an accuracy of 96.56%. They used only one dataset but generalization capability could have been guaranteed using multiple diverse datasets. Ayadi et al. [19] suggested a new CNN model for multi-classification of brain tumors and attained average accuracy of 94.74%. They did a detail analysis and discussion on their classification model but the attained accuracy was comparatively lower. Sultan et al. [20] proposed a deep CNN based model for classifying tumors into three labels, and also different grades of glioma were differentiated. For two studies, they achieved best accuracy of 96.13% and 98.7%, respectively. Though the model achieved a quite good accuracy but using large scale and diverse dataset, this could have attained more generalization capability. Deepak et al. [21] presented a CNN-SVM based classification model for three types of brain tumors and attained 95.82% accuracy. Better result could have been gained using data augmentation techniques.

Besides multi-classification, deep learning approach is also used for binary classification of tumors. Kumar et al. [22] introduced an optimized deep learning technique for classifying tumor and non-tumor cells and attained accuracy of 95.3% and 96.3%, for two different datasets, respectively. The extra preprocessing, segmentation and feature extraction processed made this model a bit complex. More optimization could be done to avoid these overheads. Toǧaçar et al. [23] introduced a new deep learning model called BrainMRNet to differentiate between normal and tumorous brain images, achieving 96.05% accuracy. Their used dataset contain only 253 images which is not enough to train and build a robust model. Hossain et al. [24] proposed a CNN based architecture for classifying tumor and non-tumor images from brain MRI that achieved 97.87% accuracy. They used only 217 images which doesn't ensure the generalization capability of their proposed model. Siar et al. [25] presented a model, where CNN feature extractor and softmax classifier were used for classifying tumorous and normal brain images attaining 98.67% accuracy. They didn't include much analysis and comparison with other methods.

Srinivas et al. [26] proposed a hybrid method based on CNN and K-Nearest Neighbor to classify benign and malignant tumors from MRI of brain, where 96.25% accuracy was achieved. The dataset was very small for a CNN model to train which included only 400 images for training. Besides, no analysis on model's robustness and comparison with other models were shown. Kader et al. [27] introduced differential deep CNN model to classify normal and abnormal (tumorous) images from brain MRI and attained an accuracy of 99.25% but their model was only limited in binary classification.

Along with advanced deep learning techniques, several researches have also been done using different classical machine learning approaches for automatic classification of brain tumors. Rajagopal et al. [28] proposed a method, where the derived features of brain MRI are optimized using the ant colony optimization technique, and then classified into Glioma or non-Glioma images using the random forest classifier. This approach achieved 98.01% accuracy. Their work was confined in detecting only glioma and non-glioma tumor detection. Further analysis could be done by detecting other types of tumors and also comparing with other state-of-the-art methods. Arasi et al. [29] proposed a method for classifying benign and malignant tumor from brain MRI, where they could attain 97.69% accuracy. The whole process includes tumor segmentation using fuzzy clustering algorithm, feature extraction using GLCM, and finally classification was conducted using the Boosting Support Vector Machine. They didn't provide any comparison or analysis mentioning the performance of other classifiers on their dataset.

Jayaprada et al. [30] proposed a fast classifier, which is based on hybrid binary Adaboost algorithm to classify normal and tumorous images of brain MRI which resulted in 90.4% accuracy. Using this approach they needed to do a lot of preprocessing work which could be an overhead. Besides, the dataset contained only 253 images which is insufficient to build a robust model. Padlia et al. [31] proposed a fractional Sobel filter and SVM based binary classification of normal and tumorous images and the best accuracy was 99.19%. They focused on the preprocessing part but further analysis could be done in the classification approach using other classifiers.

Feature level ensemble or fusion of different deep learning and machine learning models are also used for increasing model's robustness. Iqbal et al. [32] proposed a model based on the fusion of Long Short Term Memory (LSTM) model and CNN model for segmentation of tumor area which achieved 82.29% accuracy. Further research could be done by classifying those segmented tumors using the same approach. Khan et al. [33] proposed an architecture for multi-modal brain tumor classification, where robust features from VGG-16 and VGG-19 were extracted and fused before they were fed into Extreme Learning machine classifier. The highest accuracy of this approach was achieved as 97.8%. In this work, the researcher only focused on classifying modality of the images but they didn't dealt with tumor type classification. Noreen et al. [34] introduced a concatenation approach using Inception-V3 and DenseNet201. Features from different inception modules and dense blocks are extracted and further concatenated for classification of three types of brain tumor. In this way, DensNet201 performed better by achieving an accuracy of 99.51%. Sachdeva et al. [35] presented a dual level ensemble neural network for multi-classification of brain tumors. This ensemble approach achieved better performance compared to single neural network. Kang et al. [36] proposed an approach of ensemble deep features extracted from different pre-trained models. The best models are chosen based on different classifiers and the extracted features are further fed into the classifiers for final classification. This approach significantly improved the classification performance but no implementation was done to use the model in real-time environment. Amin et al. [37] proposed a score level fusion approach for binary classification of brain tumor. They used AlexNet and GoogleNet for extracting feature vector and the individual classification score are fused before final classification. In this way, they achieved the highest accuracy of 99.44%. This approach can be experimented with different tumor types besides only binary classification. Amin et al. [38] proposed a system, where tumor region is enhanced and segmented, and then the features are extracted through Local Binary Pattern and Gabor Wavelet Transform. These features are further fused for a better classification of tumor and non-tumor images. Based on the fused features, KNN performed better than other classifiers. Shankar K et al. [39] presented a process of binary classification (benign, malignant) of brain tumors, where features are extracted based on Gray Level Co-occurrence Matrix and Maximum Intensity. Those features are fused and classified using Adaptic Neuro Fuzzy Interface System (ANFIS) classifier and the obtained accuracy was 96.23%. This approach can be experimented for multi-classification of tumors. Kaur et al. [40] proposed a voting ensemble technique for detecting benign and malignant brain tumors from brain MRI. They used three classifiers, i.e., Support Vector Machine, K-Nearest Neighbor and Decision Tree for classification, and then the final outcome was calculated using majority voting. Their acquired accuracy was 97.91%. Though they achieved a good accuracy using a very small datasets, further researches could be done for diverse and large dataset.

There are some works based on non-iterative approaches like General Regression Neural Network (GRNN). For example, Izonin et al. [41]. Presented a GRNN based prediction model for small medical dataset. They prepared and applied the input doubling method using small datasets in medical application. Though this has been used for urine analysis, image classification can also be done using this approach. Sinha et al. [42] used a non-iterative convolutional feature learning based approach for brain tumor classification. For faster and accurate classification they used convolutional feature based Euclidean and achieved 97.02% accuracy. However, they have considered a specific dataset and this work should be extended by applying it on more diverse datasets.

Recent important existing works regarding brain tumor detection are summarized in Table 1 considering deep learning, classical ML and ensemble approach. This literature survey suggested that there is a great scope of experimenting with multi stage ensemble technique. One of the common limitations found from the analysis of the existing models is the lack of generalization capability and robustness. To the best of our knowledge, there is lack of studies in conducting a multi or two-stage ensemble approach for such classification of brain tumors from brain MRI, though researchers in Ref. [43] have used this approach for localization of hippocampus. This motivated us to devise a new two-stage ensemble approach of deep CNN models for classification of brain tumors. Besides, a detailed analysis of model selection, building and discussion on model validation is added in our study. The main contributions of the article are outlined below-

  • Best CNN feature extractors and classifier are selected through several trials and experiments on three different datasets. Initially total 6 features extractors were considered including 5 pre-trained models and one proposed CNN model.

  • A new two stage ensemble model is built by scrutinizing the models twice, i.e., first time when building one-stage ensemble model, second time when building the final two-stage ensemble model.

  • A real-time verification is proved through a brand-new User Interface built with the proposed model.

The remaining sections of the paper is structured as follows: A detailed description of all the materials and methodology including feature extraction, selection and classification process is presented in Section 2. The result and discussion including all other validation, verification procedures are discussed in Section 3. Finally, the conclusion is added in Section 4.

Section snippets

Materials and methodology

Fig. 1(a) and (b) depict the summary of the proposed methodology and the schematic diagram of the proposed approach respectively. The input MRI data are first preprocessed and then fed into the feature extractor. The feature extraction process is done in two stages. For the first stage, the best individual models are selected and several ensemble models are built. In the second stage further ensemble is done with best models found on first ensemble. The final ensemble model is used for feature

Result & discussion

The proposed second stage ensemble model has been verified in different ways to accurately observe the performance. Firstly, the performance is observed based on different performance metrics. The impact of applying two stage ensemble technique is also discussed. How the performance upgraded after data augmentation and Principal Component Analysis is also analyzed. To check the generalization capability of the model, it is validated through three experiments. The proposed model is also compared

Conclusion

The consequences of brain tumors can be very dreadful and life-threatening as they can cause cancer in the long term. To avoid the heinous effect, this paper proposes an automated classification system for early and precise diagnosis. To the best of our knowledge, it is the first time a two-stage ensemble of deep CNN models is used for categorizing three different types of tumors and normal brain cells. The whole process of two-stage ensemble and classification has been implemented by analyzing

Declaration of competing interest

We do not have any conflict of interest.

Acknowledgements

This research was supported by the Information and Communication Technology (ICT) division of the Government of the People's Republic of Bangladesh in 2021–2022.

References (71)

  • Kavitha Angamuthu Rajasekaran et al.

    Advanced brain tumour segmentation from mri images

  • Tonmoy Hossain et al.

    Brain tumor detection using convolutional neural network

  • Nahid Ferdous Aurna, Tanjil Mostafa Rubel, Tanveer Ahmed Siddiqui, Tajbia Karim, Sabrina Saika, Md Murshedul Arifeen,...
  • Nahid Ferdous Aurna et al.

    Predicting periodic energy saving pattern of continuous iot based transmission data using machine learning model

  • Md Badrul Alam Miah et al.

    Detection of lung cancer from ct image using image processing and neural network

  • Ali Mohammad Alqudah et al.

    Brain Tumor Classification Using Deep Learning Technique–A Comparison between Cropped, Uncropped, and Segmented Lesion Images with Different Sizes

    (2020)
  • Pashaei Ali et al.

    Brain tumor classification via convolutional neural network and extreme learning machines

  • Emrah Irmak

    Multi-classification of brain tumor mri images using deep convolutional neural network with fully optimized framework

    Iran. J. Sci. Technol., Transac. Electr. Eng.

    (2021)
  • Narmada M Balasooriya et al.

    A sophisticated convolutional neural network model for brain tumor classification

  • Sunanda Das et al.

    Brain tumor classification using convolutional neural network

  • Parnian Afshar et al.

    Capsule networks for brain tumor classification based on mri images and coarse tumor boundaries

  • D Jude Hemanth et al.

    A modified deep convolutional neural network for abnormal brain image classification

    IEEE Access

    (2018)
  • Milica M. Badža et al.

    Classification of brain tumors from mri images using a convolutional neural network

    Appl. Sci.

    (2020)
  • Wadhah Ayadi et al.

    Deep cnn for brain tumor classification

    Neural Process. Lett.

    (2021)
  • Hossam H. Sultan et al.

    Multi-classification of brain tumor images using deep neural network

    IEEE Access

    (2019)
  • S. Deepak et al.

    Automated categorization of brain tumor from mri using cnn features and svm

    J. Ambient Intell. Hum. Comput.

    (2021)
  • Tonmoy Hossain et al.

    Brain tumor detection using convolutional neural network

  • Masoumeh Siar et al.

    Brain tumor detection using deep neural network and machine learning algorithm

  • B. Srinivas et al.

    A hybrid cnn-knn model for mri brain tumor classification

    Int. J. Recent Technol. Eng. (IJRTE)

    (2019)
  • Isselmou Abd El Kader et al.

    Differential deep convolutional neural network model for brain tumor classification

    Brain Sci.

    (2021)
  • R. Rajagopal

    Glioma brain tumor detection and segmentation using weighting random forest classifier with optimized ant colony features

    Int. J. Imag. Syst. Technol.

    (2019)
  • P. Rupa Ezhil Arasi et al.

    A clinical support system for brain tumor classification using soft computing techniques

    J. Med. Syst.

    (2019)
  • S. Jayaprada et al.

    Fast hybrid adaboost binary classifier for brain tumor classification

  • Minal Padlia et al.

    Fractional sobel filter based brain tumor detection and segmentation using statistical features and svm

  • Sajid Iqbal et al.

    Deep learning model integrating features and novel classifiers fusion for brain tumor segmentation

    Microsc. Res. Tech.

    (2019)
  • Cited by (65)

    View all citing articles on Scopus
    View full text