“Artificial intelligence,” “machine learning,” and “deep learning” are terms frequently used by the media as new technologies that will disrupt a variety of fields. These have included self-driving cars, mobile devices, and even the ancient Chinese game of Go, which has more than 2 x 10¹⁷⁰ possible legal moves (more than there are atoms in the universe). Health care is no exception, and it is important for practicing clinicians to be aware of the ever-changing landscape that will soon inevitably use many of these new applications.

Machine learning is a subfield of artificial intelligence in which machines are “trained” to perform tasks such as pattern recognition without explicit programming.¹ These methodologies have evolved from the earliest approaches toward image analysis and computer vision using basic logistic or linear regressions of semiquantitative metrics acquired from limited ROIs.² While these techniques can be useful in certain situations, they inherently distill a complex dataset (eg, over a million voxels of information from a brain MRI) into a handful of numeric descriptors and assume a simple relationship among chosen features, which may not exist for a complex biologic system.

By contrast, newer machine-learning methods are continually being developed to leverage this information and model nonindependent and nonlinear relationships among the various chosen features. While many such machine-learning classifiers exist, the most popular include random forests, support vector machines (SVMs), k-nearest neighbor clustering, and neural networks.³  In general, these techniques are modeled by an underlying, finite number of adjustable parameters. As a given set of features is passed through the model, these adjustable parameters act to convert the input descriptors into a predicted output class. Starting with randomly initialized parameters, a series of iterative updates is performed until an accurate mapping between numeric features and the correct class is achieved, thus “training” the machine-learning model.⁴

Presently, machine-learning techniques are shifting toward end-to-end machine learning with convolutional neural networks (CNNs), which can combine feature selection and classification into 1 algorithm.^5,6 Thus, deep-learning approaches are able to learn the image features that are critical for solving a classification problem without human supervision. In fact, given sufficient training data, the machine will determine the optimal feature set and the relative importance of each feature, allowing it to use combinations of features to classify images. In recent years, CNN approaches have been used to successfully tackle a variety of problems in engineering that have included computer vision,^7-9 as well as in the natural sciences, which have included physics,¹⁰ chemistry,¹¹ and biology.^12-14

CNN approaches model the animal visual cortex by applying a feed-forward artificial neural network to simulate multiple layers of neurons organized in overlapping regions within a visual field; each layer acts to transform the raw input image into more complex, hierarchic, and abstract representations.⁵ Thus, it is natural to also consider applying deep-learning methods to biomedical images. For example, Shen et al¹⁵ developed multiscale CNNs for lung nodule detection with CT images, while Wang et al¹⁶ devised a 12-layer CNN for cardiovascular disease detection in mammograms, as well as CNNs for metastatic spinal cancer detection.¹⁷

This edition of the AJNR News Digest highlights several machine-learning applications in neuroimaging. For neurodegenerative conditions, Haller et al¹⁸ were able to accurately classify patients with Parkinson disease at the individual level using SVMs. Chen et al¹⁹ leveraged a Bayesian machine-learning approach to identify hepatic encephalopathy among patients with cirrhosis. The remaining 4 articles have focused on using a variety of tools for neuro-oncology imaging applications, including the discernment of lymphoma,²⁰ nonenhancing disease,²¹ areas of radionecrosis,²² and tumor cellularity.²³

References

1. Goodfellow I, Bengio Y, Courville A. Deep Learning. Cambridge, MA: The MIT Press; 2016
2. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002;35:352–59, 10.1016/S1532–0464(03)00034-0.
3. Wang S, Summers RM. Machine learning and radiology. Med Image Anal 2012;16:933–51, 10.1016/j.media.2012.02.005.
4. Jordan MI, Mitchell TM. Machine learning: trends, perspectives, and prospects. Science 2015;349:255–60, 10.1126/science.aaa8415.
5. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436–44, 10.1038/nature14539.
6. Simonyan K, Vedaldi A, Zisserman A. Deep inside convolutional networks: visualising image classification models and saliency maps. CoRR 2013;arXiv:1312.6034v2.

7. He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. CoRR 2015;arXiv:1512.03385v1
8. Krizhevsky A, Sutskever I, Hinton G. Imagenet classification with deep convolutional neural networks. Proc Advances in Neural Information Processing Systems 2012:1090–98.
9. Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. CoRR 2014;arXiv:1409.4842.
10. Baldi P, Sadowski P, Whiteson D. Searching for exotic particles in high-energy physics with deep learning. Nat Commun 2014;5:4308, 10.1038/ncomms5308.
11. Kayala MA, Baldi P. ReactionPredictor: prediction of complex chemical reactions at the mechanistic level using machine learning. J Chem Inf Model 2012;52:2526–40, 10.1021/ci3003039.
12. Lena PD, Nagata K, Baldi P. Deep architectures for protein contact map prediction. Bioinformatics 2012;28:2449–57, 10.1093/bioinformatics/bts475.
13. Lusci A, Pollastri G, Baldi P. Deep architectures and deep learning in chemoinformatics: the prediction of aqueous solubility for drug-like molecules. J Chem Inf Model 2013;53:1563–75, 10.1021/ci400187y.
14. Agostinelli F, Ceglia N, Shahbaba B, et al. What time is it? Deep learning approaches for circadian rhythms. Bioinformatics 2016;32:i8–17, 10.1093/bioinformatics/btw243.
15. Shen W, Zhou M, Yang F, et al. Multi-scale convolutional neural networks for lung nodule classification. In: Information Processing in Medical Imaging, IPMI 2015, Lecture Notes in Computer Science, vol 9123.
16. Wang J, Ding H, Bidgoli FA, et al. Detecting cardiovascular disease from mammograms with deep learning. IEEE Trans Med Imaging 2017;36:1172–81, 10.1109/TMI.2017.2655486.
17. Wang J, Fang Z, Lang N, et al. A multi-resolution approach for spinal metastasis detection using deep siamese neural networks. Comput Biol Med 2017;84:137–46, 10.1016/j.compbiomed.2017.03.024.
18. Haller S, Badoud S, Nguyen D, et al. Individual detection of patients with parkinson disease using support vector machine analysis of diffusion tensor imaging data: initial results. AJNR Am J Neuroradiol 2012;33:2123–28, 10.3174/ajnr.A3126.
19. Chen HJ, Chen R, Yang M, et al. Identification of minimal hepatic encephalopathy in patients with cirrhosis based on white matter imaging and Bayesian data mining. AJNR Am J Neuroradiol 2015;36:481–87, 10.3174/ajnr.A4146.
20. Alcaide-Leon P, Dufort P, Geraldo AF, et al. Differentiation of enhancing glioma and primary central nervous system lymphoma by texture-based machine learning. AJNR Am J Neuroradiol 2017;38:1145–50, 10.3174/ajnr.A5173.
21. Blumenthal DT, Artzi M, Liberman G, et al. Classification of high-grade glioma into tumor and nontumor components using support vector machine. AJNR Am J Neuroradiol 2017;38:908–14, 10.3174/ajnr.A5127.
22. Tiwari P, Prasanna P, Wolansky L, et al. Computer-extracted texture features to distinguish cerebral radionecrosis from recurrent brain tumors on multiparametric MRI: a feasibility study. AJNR Am J Neuroradiol 2016;37:2231–36, 10.3174/ajnr.A4931.
23. Chang PD, Malone HR, Bowden SG, et al. A multiparametric model for mapping cellularity in glioblastoma using radiographically localized biopsies. AJNR Am J Neuroradiol 2017;38:890–98, 10.3174/ajnr.A5112.

T1 postcontrast features identified by a convolutional neural network associated with IDH wildtypes (A, C) and mutation (B, D). Specifically, IDH wildtypes demonstrated thick and irregular enhancement (A) or thin, irregular, poorly marinated peripheral enhancement (B). In contrast, patients with IDH mutations demonstrated absent or minimal enhancement (C) with well-defined tumor margins (D). All features were identified automatically using deep learning. The histologic grade of each tumor is denoted in the top left corner.

Photo Acknowledgment: Figure 2 from Chang P, Grinband J, Weinberg BD, et al. Deep learning convolutional neural networks accurately classify genetic mutations in gliomas. AJNR Am J Neuroradiol. Accepted for publication.

Image from: Blumenthal DT, Artzi M, Liberman G, et al. Classification of high-grade glioma into tumor and nontumor components using support vector machine. AJNR Am J Neuroradiol 2017;38:908–14, 10.3174/ajnr.A5127.