A 2D/3D correspondence building method for reconstruction of a patient-specific 3D bone surface model using point distribution models and calibrated X-ray images
Introduction
The applications of X-ray imaging in orthopaedic surgery are pervasive, both pre-operatively and intra-operatively. Pre-operative planning based on information about the patient anatomy provided by conventional X-ray imaging has been established since several years (Eggli et al., 1998). Due to the projective character of X-ray imaging, accuracy of pre-operative X-ray radiograph based planning is restricted. Intra-operatively, fluoroscopy is a valuable imaging tool for visualizing underlying bone and surgical tools in various orthopaedic procedures. However, disadvantages of intra-operative fluoroscopy are also apparent, including two-dimensional (2D) projection image from single view, limited field of view, distorted image, and high radiation to both the patient and the surgical team. Various papers (Joskowicz et al., 1998, Hofstetter et al., 1999, Yao et al., 2001, Tate et al., 2001, Livyatan et al., 2002, Laurence et al., 2005) have described methods of calibration and registration of X-ray images using a positional localizer, thus allowing to compute the position of the surgical tools relative to the patient anatomy with respect to acquired images during intervention. However, the surgeon still needs to mentally fuse projection images taken from different view points. No real three-dimensional (3D) information is available. One way to address these problems is to build a statistical shape model (SSM) and to adapt the SSM to the patient’s individual anatomy based on a limited number (e.g. 2) of calibrated X-ray images. The reconstructed shape model can then provide detailed 3D information for the considered anatomical structure.
However, constructing a 3D bone surface model from a limited number of calibrated 2D X-ray images and a 3D SSM is a challenging task, especially when we would like to construct a patient-specific surface model of a bone with pathology. Moreover, inherent to the surgical navigation application is the high accuracy requirement. When surface reconstruction is used for the purpose of surgical guidance, the error of the reconstructed model should be in the range of surgical usability, which is typically in the area of 1.5 mm average error (2–3 mm, worst case) as suggested by Livyatan et al. (2003) under the context of 2D/3D image registration. In this paper, we present a 2D/3D reconstruction scheme that can seamlessly handle both non-pathologic and pathologic cases, even when our SSM is constructed from training instances without pathology. At the heart of our approach lies the combination of sophisticated surface reconstruction techniques and a novel algorithm for establishing correspondences between the 2D X-ray images and the 3D SSM.
The first part of this paper deals with the correspondence establishment. Our 2D/3D correspondence building method is based on a non-rigid 2D point matching process, which iteratively uses a symmetric injective nearest-neighbor mapping operator and 2D thin-plate splines (TPS) based deformations to find a fraction of best matched 2D point pairs between features extracted from the 2D images and those extracted from the 3D models (Zheng, 2006). The estimated point pairs are then used to set up a set of 3D point pairs such that we turn a 2D/3D reconstruction problem to a 3D/3D one, whose solutions are well studied (Zheng et al., 2007b).
Incorporating the correspondence building method, the second part of this paper deals with accurately fitting the SSM to the input images. The fitting problem is formulated as a three-stage optimal estimation process (Zheng and Nolte, 2006). Fig. 1 shows a schematic diagram of the 2D/3D fitting scheme. The first stage, scaled rigid registration, is to iteratively estimate a scale and a rigid transformation between the mean surface model of the SSM and the input images using an adapted iterative closest point (ICP) algorithm (Guéziec et al., 1998). The second stage, statistical instantiation, stably instantiates a surface model from the SSM using a Mahalanobis prior based statistical approach (Rajamani et al., 2007). This surface model is then fed to the third stage, regularized shape deformation. In this stage, we further refine the statistically instantiated surface model using an alternative derivation of the familiar interpolating thin-plate splines (TPS) (Bookstein, 1989) that enables weighting between the SSM instantiated surface model and the TPS interpolation (Zheng et al., 2007b).
Section snippets
Related Work
Statistical shape analysis (Kendall, 1989, Small, 1996, Dryden and Mardia, 1998) is an important tool for understanding anatomical structures from medical images. Statistical models give efficient parameterizations of the shape variations found in a collection of sample models of a given population. Model based approaches are popular (Turk and Pentland, 1991, Cootes et al., 1994, Corouge et al., 2003) due to their ability to robustly represent objects. In the last few years, constructing a
Statistical model construction
The first step is to build a statistical shape model from a training database. Several different geometric representations have been used to model anatomy. Bookstein (1986) uses landmarks to capture the important geometric features. The active shape model (ASM) of Cootes et al. (1995) represents an object’s geometry as a dense collection of boundary points. Cootes et al. (1998) have augmented their statistical models to include the variability of the image information as well as shape. Kelemen
A 2D/3D correspondence building method
Given a few fluoroscopic images, our task is to establish correspondences between the input images and a model estimated from the PDM. Here, we assume that the input images are calibrated and registered to a common coordinate system. And for a pixel in an input image we can always find a projection ray emitting from the focal point of the associated image through the pixel.
3D/3D Reconstruction
Using the developed algorithm for establishing correspondences, we can always find a set of 3D point pairs given an initial model state, i.e., the initial scale and the initial pose parameters of our point distribution model. The problem of surface reconstruction is then solved optimally in three stages (Zheng and Nolte, 2006): scaled rigid registration, statistical instantiation, and regularized shape deformation.
Experiments and results
The primary applications that we focus on are hip surgeries such as total hip replacement (THR) or hip resurfacing. Hence we began by concentrating on the proximal femur. To evaluate the performance of the present 2D/3D correspondence building method and that of the 2D/3D reconstruction scheme, comprehensive experiments were conducted in two co-institutions using two different point distribution models.
The PDM used in the first co-institution (MEM Research Center), which we named as MEM-PDM,
Discussions and conclusions
In this paper, we presented a 2D/3D correspondence building method. Incorporating this 2D/3D correspondence building method, we developed a 2D/3D reconstruction scheme combining a statistical instantiation with a regularized shape deformation. We designed and conducted comprehensive experiments on different types of data to evaluate the performance of the 2D/3D correspondence building method as well as that of the 2D/3D reconstruction scheme.
Our 2D/3D correspondence building method belongs to
Acknowledgements
The authors gratefully acknowledge the contributions of Dr. Moritz Tannast and Dr. Xuan Zhang for their help in experimental setup. They would also like to thank the anonymous reviewers whose comments and suggestions helped improve the original manuscript. This work was partially supported by Swiss National Science Foundation through the project NCCR CO-ME.
References (56)
- et al.
Evaluation of image features and search strategies for segmentation of bone structures in radiographs using active shape models
Medical Image Analysis
(2002) - et al.
3D/2D registration and segmentation of scoliotic vertebrae using statistical models
Computerized Medical Imaging and Graphics
(2003) - et al.
A new point matching algorithm for non-rigid registration
Computer Vision and Image Understanding
(2003) - et al.
Active shape models – their training and application
Computer Vision and Image Understanding
(1995) - et al.
Interindividual functional mapping: a nonlinear local approach
NeuroImage
(2003) - et al.
Image analysis for computer-assisted internal fixation of hip fractures
Medical Image Analysis
(2000) - et al.
Statistical deformable bone models for robust 3d surface extrapolation from sparse data
Medical Image Analysis
(2007) - Aspert, N., Santa-Crutz, D., Ebrahimi, T., 2002. Mesh: measuring error between surfaces using the hausdorff distance....
- et al.
A method for registration of 3D shapes
IEEE Transactions on Pattern Analysis Machine Intelligence
(1992) Size and shape spaces for landmark data in two dimensions (with discussion)
Statistical Science
(1986)
Principal warps: thin-plate splines and the decomposition of deformations
IEEE Transactions on Pattern Analysis Machine Intelligence
A computational approach to edge detection
IEEE Transactions on Pattern Analysis and Machine Intelligence
Automatic extraction of femur contours from hip X-ray images
The use of active shape models for locating structures in medical images
Image and Vision Computing
A minimum description length approach to statistical shape modeling
IEEE Transactions on Medical Imaging
Statistical Shape Analysis
The value of preoperative planning for total hip arthroplasty
The Journal of Bone and Joint Surgery
Regularization networks and support vector machine
Advances in Computational Mathematics
Nonrigid 3D/2D registration of images using statistical models
Anatomy-based registration of ct-scan and intraoperative x-ray images for guiding a surgical robot
IEEE Transactions on Medical Imaging
Fluoroscopy as an image means for computer-assisted surgical navigation
Computer Aided Surgery
Computer-assisted fluoroscopy-based reduction of femoral fractures and antetorsion correction
Computer Aided Surgery
Cited by (134)
Benchmarking off-the-shelf statistical shape modeling tools in clinical applications
2022, Medical Image AnalysisComputer-assisted orthopedic surgery
2022, Human Orthopaedic Biomechanics: Fundamentals, Devices and ApplicationsFemoral strength can be predicted from 2D projections using a 3D statistical deformation and texture model with finite element analysis
2021, Medical Engineering and PhysicsFeasibility study for the automatic surgical planning method based on statistical model
2023, Journal of Orthopaedic Surgery and Research