Automatic 3-D spine curve measurement in freehand ultrasound via structure-aware reinforcement learning spinous process localization

Freehand 3-D ultrasound systems have been advanced in scoliosis assessment to avoid radiation hazards, especially for teenagers. This novel 3-D imaging method also makes it possible to evaluate the spine curvature automatically from the corresponding 3-D projection images. However, most approaches neglect the three-dimensional spine deformity by only using the rendering images, thus limiting their usage in clinical applications. In this study, we proposed a structure-aware localization model to directly identify the spinous processes for automatic 3-D spine curve measurement using the images acquired with freehand 3-D ultrasound imaging. The pivot is to leverage a novel reinforcement learning (RL) framework to localize the landmarks, which adopts a multi-scale agent to boost structure representation with positional information. We also introduced a structure similarity prediction mechanism to perceive the targets with apparent spinous process structures. Finally, a two-fold filtering strategy was proposed to screen the detected spinous processes landmarks iteratively, followed by a three-dimensional spine curve fitting for the spine curvature assessments. We evaluated the proposed model on 3-D ultrasound images among subjects with different scoliotic angles. The results showed that the mean localization accuracy of the proposed landmark localization algorithm was 5.95 pixels. Also, the curvature angles on the coronal plane obtained by the new method had a high linear correlation with those by manual measurement (R = 0.86, p < 0.001). These results demonstrated the potential of our proposed method for facilitating the 3-D assessment of scoliosis, especially for 3-D spine deformity assessment.