A CNN-based Scoliosis Screening and Monitoring Algorithm Using Bare Back Images

Author(s): Bryan Chiu

Originally published on Towards AI.

Bryan Chiu

I. Adolescent Idiopathic Scoliosis

Affecting approximately 2–5% of the global population, adolescent idiopathic scoliosis (AIS) is a condition marked by an abnormal lateral curvature of the spine, measuring 10 degrees or more, that typically occurs in individuals aged 10 to 18 years [1]. AIS often presents as a “C” or “S” shaped curve due to the three-dimensional twisting of the vertebrae, commonly involving the thoracic and lumbar regions. Although the precise causes of AIS remain unclear, it is thought to result from a mix of genetic, hormonal, and environmental factors, usually emerging during the rapid growth period preceding puberty [2]. Early identification is essential because, if left untreated, AIS can lead to serious complications, including altered body appearance, impaired cardiopulmonary function, and, in severe cases, paralysis. Furthermore, advanced scoliosis can restrict chest cavity space, compromising lung capacity and function [4]. Given these potentially severe outcomes and the condition’s progressive nature, early detection and intervention are imperative.

There are several methods for assessing AIS, including forward bending tests, appearance inspection, and scoliometer measurements [1]. Unfortunately, most of these require medical expertise and specialization. Using X-rays is an alternative with high accuracy, but it subjects teenagers to unnecessary radiography exposure. Due to the limitations and inconveniences of these traditional methods, many cases go undetected early, contributing to the high global prevalence of scoliosis. This article investigates the absence of a fast, convenient and non-harmful way to perform scoliosis early screening regularly with teenagers aged 10 to 18 years, especially in developing economies and remote communities.

My interest in adolescent idiopathic scoliosis (AIS) is deeply personal, as I was unexpectedly diagnosed with low-to-moderate scoliosis (Cobb angle of 17%, see Fig. 1) during a hospitalization for pneumothorax in 2023. This experience has fuelled my commitment to advancing early detection and innovative solutions for scoliosis screening and management.

II. Problem with the Current Methods of Screening and Detection

Recent advancements have seen researchers developing various image-based detection methods, utilizing bare back photographs for scoliosis screening. For example, Yang Tang’s paper discussing the analysis of the ROI (Region-of-Interest) based on the extracted features [5]. One limitation of this solution is its ineffectiveness for obese patients due to their less distinct back contours and anatomical landmarks. A clinical study in Japan by Dr. Kokabu et al. used deep learning model on 3D depth sensor imaging to detect symmetry in the users’ forward bend posture in the Adam’s forward bend test [6] [7]. Another study by Junlin Yang proposed a deep learning algorithm model using Faster-RCNN to detect scoliosis using extensive internal validation and algorithm training [1]. Despite these innovations, the current solutions have notable drawbacks, such as the inability to monitor disease progression over time and the requirement for significant computational resources.

III. A CNN-based Scoliosis Screening and Monitoring Algorithm Using Bare Back Images

The aim of this AI algorithm is to enable quick and accessible scoliosis screening for adolescents anytime and anywhere. It provides a binary classification output indicating either a positive or negative screening result. A positive outcome necessitates further confirmatory testing by healthcare professionals.

As illustrated in Fig. 2, the solution is an AI algorithm embedded in the frontend user interfaces of web or mobile apps. A simple user experience is key to mass adoption, especially in communities that are remote or have relatively low economic and education levels. The next step describes the design to enable such a simple user experience principle.

The core part of the solution is the AI algorithm backend. AI is extensively used in medical image understanding in recent years. Of all the machine learning and deep learning techniques, the Convolutional Neural Network (CNN) is a deep learning model designed for image and video data, leveraging convolutional layers to extract spatial features, pooling layers to reduce dimensionality, and fully connected layers for classification. CNN models excel at high-dimensional tasks such as image classification, object detection, and segmentation, and their training involves learning optimal weights through backpropagation and model adjustments.

IV. Solution Design

In this study, datasets were created for the CNN model with common pre-processing in terms of image data resizing and normalization. Each step of the method is described below:

Algorithm Training, Internal Validation and Testing (External Validation) Dataset

The training, internal, and testing (external validation) datasets are based on 142 labeled bareback images from Kaggle [8] and Scoliosis.co.uk [9]. The images were resized, normalized, and split into a training dataset, internal validation dataset and external validation dataset as shown in Table I below. The training dataset was used to train the model, while the validation dataset was used to evaluate the model's performance.

Data Augmentation

The data augmentation in this study includes zooming in the image and horizontal flipping of the image to create a new data point. No rotation, color or resolution adjustments were made to the images.

Architecture of CNN

Based on the comprehensive surveys by Sarvamangala et al 2022 [10], we have selected a 50-layer Residual Network (ResNet50) model for screening bareback images.

Optimization of Learning Weights and Model Parameters

For the CNN model, the algorithm first analyses the image in different sections and extracts its most prominent feature. It then downsamples the image and creates the output. After calculating the loss function, it adjusts the weights of the neurons using backward propagation and attempts to improve accuracy. A summary of the parameters for CNN model is provided in Table II.

Result

This CNN model leads to a 77% accuracy in validation and 72% accuracy in testing. The confusion matrix with low false negative results is provided in Fig. 3.

I have successfully developed an AI-powered engine for scoliosis screening, with the CNN model showing superior performance on our current dataset. However, to enhance the model’s accuracy, currently ranging from 72–77%, we need a larger and more diverse dataset, as well as additional pre-processing and optimization. We are partnering with a spine care clinic in Hong Kong to conduct a clinical trial for large-scale dataset validation, ensuring patient consent and ethical standards.

Integrating this AI-based scoliosis screening into a website or mobile app could revolutionize early detection by providing a faster, more convenient, and non-invasive alternative to traditional methods. Individuals flagged with a positive screening result could be promptly referred to medical professionals for comprehensive diagnostic evaluation. This approach would significantly improve screening efficiency and frequency, particularly in underserved areas with limited access to medical resources. Additionally, the AI algorithm can be used to monitor the progress of correctional treatments, reducing the need for frequent radiographic exams and thereby minimizing radiation exposure.

V. Challenges and Future Plans

The limited size of the current dataset restricts the model’s generalizability, highlighting the need for large, high-quality datasets — a crucial element for the success of any AI project. Our upcoming clinical trial partnership aims to significantly expand the dataset, enhancing the model’s robustness. However, as we scale, we anticipate challenges related to data storage, management, cross-regional sharing, and adherence to regulatory compliance.

The next steps in advancing this research will require further refinement and the integration of advanced modeling techniques, such as attention mechanisms, hyperparameter tuning, and pre-processing methods like contour mapping of the back and feature engineering. These improvements are expected to enhance both the model’s accuracy and its performance on the confusion matrix. Ideally, I aim to reach an accuracy of over 90. Equally important are efforts to raise community awareness and education, along with the development of practical tools like an e-scoliometer that utilizes a smartphone’s gyroscope sensor for easy, at-home assessments.

Additionally, this AI platform could be augmented with “ScoliosisGPT,” a next-generation AI companion powered by Gen AI and LLMs. ScoliosisGPT would provide patients with comprehensive information on scoliosis, track their progress, and offer data-driven summaries of their treatment journey for both patients and healthcare providers, facilitating better understanding and management of the condition.

VI. Conclusions

This study successfully demonstrated the integration of mobile technology and AI for convenient, non-invasive scoliosis screening. Our CNN deep learning model could achieve an initial acceptable accuracy of 72–77% and the product prototype illustrates the potential of facilitating pervasive fast scoliosis screening anywhere, anytime as well as ongoing progress monitoring of diagnosed patients. Significant room for improvement in terms of accuracy, performance, and usability.

Acknowledgments

I would like to thank Dr. Saeed Rahman, Ms. Vivian Fung, Mr. Bosco Yiu, Mr. Tony Chow, Mr. Alex Tong, Mr. Alex Wu and Ms. Agnes Chiu from the Canadian International School of Hong Kong; Mr. Paul Hodgson from IEEE; Prof. Ting Su, Prof. Ding Zhao and Prof. Gang Liu from the Carnegie Mellon University (CMU); Prof. M.S. Wong of the Department of Biomedical Engineering of the Hong Kong Polytechnic University; Prof. Grace Zhang of the AI Med Group, Department of Orthopaedics and Traumatology, the University of Hong Kong; Ms. Alyssa Gong and Mr. Keynes Wu from “AI for Humanities” at the CMU summer session 2024; the Jockey Club Children’s Spine Care Community Project; and all volunteers who generously contributed data and advice.

Data and Code Availability Statement

The data and code that support the findings of this study are available from the corresponding author upon reasonable request.

References

[1] Yang et al., “Development and validation of deep learning algorithms for scoliosis screening using back images,” Communications Biology, Nature, 2019

[2] Zhang et al., “A New Method for Scoliosis Screening Incorporating Deep Learning With Back Images,” Global Spine Journal, 2024

[3] [Online]. Available: https://scoliosis.hku.hk/en/what-is-scoliosis/ . Accessed 28 July 2024.

[4] S.L. Weinstein, L.A. Dolan, J.C. Cheng, A. Danielsson, and J.A. Morcuende, “Adolescent Idiopathic Scolosis,” The Lancet, Vol. 371, №9623, pp. 1527–1537, 2008

[5] Tang, Yang. “Scoliosis Detection Based on Feature Extraction from Region-of-Interest.” ResearchGate, June 2022, www.researchgate.net/publication/362313605_Scoliosis_Detection_Based_on_Feature_Extraction_from_Region-of-Interest . Accessed 28 July 2024.

[6] Terufumi Kokabu, MD, Satoshi Kanai, PhD, Noriaki Kawakami, MD, Koki Uno, MD, Toshiaki Kotani, MD, Teppei Suzuki, MD, Hiroyuki Tachi, MD, Yuichiro Abe, MD, Norimasa Iwasaki, MD, Hideki Sudo, MD, “An algorithm for using deep learning convolutional neural networks with three-dimensional depth sensor imaging in scoliosis detection,” Vol. 21, pp. 980–987, The Spine Journal 2021

[7] Grivas TB, Vasiliadis ES, Mitas C, Triantafyllopoulos G, Kaspiris A., “Trunk Asymmetry In Juveniles”, 3:13, Scoliosis, 2008

[8] [Online]. Kaggle Database for scoliosis bare back images. Available: https://www.kaggle.com/datasets/akbotayelemessova/scoliosis-back-images-yes Accessed 28 July 2024.

[9] [Online]. Available: https://scoliosis.co.uk/ . Accessed 28 July 2024.

[10] Sarvamangala, D.R., Kulkarni, R.V., “Convolutional neural networks in medical image understanding: a survey,” Evol. Intel. 15, 1–22, 2022.

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI