The risk factors for knee OA can be categorised into: demographic data, anthropometric features, medical history, soluble biomarkers, genes, imaging features and clinical outcomes.1 In DL, a sample (e.g. a patient) is represented by a number of features, including patient's characteristics, risk factors, shape/texture characteristics in medical images (X-ray and quantitative MRI structure assessment) or clinical history data. To facilitate the learning process, these elements are typically linked, forming a multidimensional feature vector. We review the common computational research methodologies that aim to address this issue and provide examples of supervised machine learning algorithms for disease prediction and or classification models.
Deep Learning and Recent Developments in Artificial Intelligence
In its essence, DL is the study of how computer algorithms (specifically Convolutional Neural Networks) can “learn” complex relationships or patterns from empirical data and hence, produce (mathematical) models linking an even larger number of covariates to some target variable of interest. The medical imaging A.I. community has seen a gradual shift from a ‘model-centric’ towards a ‘data-centric’ approach to training algorithms, I.e. The quality of the data greatly determines its performance. For example, consider the performance of a model that employs unsupervised learning on unlabelled data (raw images of knee X-rays or knee MRI’s) versus a model that employs supervised/semi-supervised learning on annotated data generated by domain-experts reviewing images and labelling them for ground truth(e.g. Knee X-rays labelled with clinical endpoints such as the Kellgren and Lawrence classification) for which the algorithm employs to make predictions when it encounters ‘unseen’ cases. Conversely, unsupervised algorithms may yield useful insights as they are not biased by misattribution i.e. Domain experts potential misconceptions, particularly in the interpretation of ambiguous imaging findings.
Literature Review
A literature search was performed using Medline, Embase, Web of Science and Google Scholar to identify peer-reviewed literature that used DL methodologies for the study of KOA between 2006 and 2021. Search terms included ‘machine learning,’ ‘deep learning’ and ‘knee osteoarthritis’ and article inclusion was based upon the presence of these terms within either the title or abstract. Articles were extracted and screened by two authors. Exclusion criteria were: non-English articles; dissertations; population with OA of joints other than the knee; study employed traditional statistics or methodology which utilised traditional machine learning techniques alone. Studies were sub-grouped according to their data source (i.e. knee radiograph, knee MRI, clinical/demographic data or biomechanical data). In addition, they were classified according to their purpose with regards to diagnosis, segmentation, predicting the risk of developing KOA and those which aimed to predict the risk of KOA progression to knee replacement. We critically appraised the methodology of each study as per current evidence-based guidelines below.
Checklist for Artificial Intelligence in Medical Imaging (CLAIM):
To address applications of AI in medical imaging that include classification, image reconstruction, text analysis, the CLAIM8 guideline provides a standardised framework for authors and reviewers. Its purpose is to promote transparent, and reproducible scientific communication about the application of AI to medical imaging and is the current standard of appraisal adopted by the Radiological Society of North America.
Deep Learning Techniques:
Broadly, machine learning can be categorised as supervised, unsupervised or reinforcement learning. In supervised learning, each data sample is represented by a pair consisting of an input (typically a multi-dimensional feature vector) and a desired output value (e.g. classifying an image as Kellgren Lawrence grades in case of KOA). The training phase involves the task of learning a function that maps every input to its associated output. The inferences generated are then used as functions to map ‘unseen’ inputs during the testing phase.
Unsupervised learning is a method by which unlabeled data is analysed with the goal of discovering structures or patterns in the dataset and involves clustering of similar data points to generate inferences. In reinforcement learning, a model learns through trial and error interactions with its environment and is programmed to iteratively maximise reward and minimise the penalty which is assigned to each of its predictions.
A subset of machine learning is deep learning, whereby a convolutional neural network is employed with differing architectures common to which is a ‘hidden layer’ of nodes which uses multiple layers to progressively extract higher-level features from the raw input data (see Figure 1). Datasets are partitioned into training and test sets for validation whereby the performance of the model can be evaluated. Models can be trained iteratively where iterative adjustment of weights between each node can influence subsequent performance and feedback loops are employed to refine the algorithm (see Figure 2).
Medical Image Segmentation:
Imaging data form a crucial source of information in KOA research. While the majority of the published literature employs knee radiograph data, there has been growing interest in knee MRI data analysis using feature engineering. Image segmentation is the procedure of transforming and delineating an image into meaningful components, for example, outlining cartilage, menisci and bone as well as any pathological features (e.g. bone oedema, subchondral bone lesions) which may be present. The Gray Level Co-occurrence matrix (GLCM) and Principal Component Analysis (PCA) are examples of common feature selection algorithms used in automated segmentation of knee MRI.
Findings:
Figure 3 illustrates the appraisal of studies according to the CLAIM criteria. From our search, the majority of study methodology employed supervised learning, in addition, the use of deep learning techniques was limited in comparison to those which employed traditional machine learning techniques, which was likely due to the attributes of the available training data (heterogeneous features and small sample sizes). With regards to the particular application of AI models using knee MRI data, we identified 3 studies that evaluated KOA progression9-11, 2 studies that aimed to diagnose/grade the severity of KOA12, 13, 1 study which investigated the risk of developing KOA14 and 1 study which sought to segment and quantify cartilage morphology.15
Among the studies we appraised, image pre-processing with landmark localisation using manual cropping and selection of seed points was commonly employed to improve the performance of segmentation techniques prior to the initialisation of parameters for training. Data from the Osteoarthritis Initiative (OAI) cohort were the most frequently used to validate the performance of image segmentation models.
Discussion
The findings of our appraisal suggest that heterogeneity of study methodology is a challenge when comparing the performance and outcomes between ensemble deep learning algorithms. KOA is a big-data problem and DL could play a key role in developing data-driven insights to enable the early diagnosis and prevention of KOA, as well as prediction of which patients are likely to progress and require knee replacement, such that risk stratification may enable optimised distribution of resources towards this sub-population. Multiparametric models which employ demographic information, clinical history, imaging data and biomechanical parameters demonstrate a potential to be useful input data in KOA diagnosis and prognostication. In addition, subjective measures of pain outcome scores and objective data in the form of soluble biomarkers and genetics have been proposed.
Open data interoperability and multidisciplinary research collaboration are encouraged to enable the external validation and subsequent clinical point of care adoption of ensemble deep learning algorithms in the management of KOA.