Elsevier

Medical Image Analysis

Volume 13, Issue 6, December 2009, Pages 883-899
Medical Image Analysis

A 2D/3D correspondence building method for reconstruction of a patient-specific 3D bone surface model using point distribution models and calibrated X-ray images

https://doi.org/10.1016/j.media.2008.12.003Get rights and content

Abstract

Constructing a 3D bone surface model from a limited number of calibrated 2D X-ray images (e.g. 2) and a 3D point distribution model is a challenging task, especially, when we would like to construct a patient-specific surface model of a bone with pathology. One of the key steps for such a 2D/3D reconstruction is to establish correspondences between the 2D images and the 3D model. This paper presents a 2D/3D correspondence building method based on a non-rigid 2D point matching process, which iteratively uses a symmetric injective nearest-neighbor mapping operator and 2D thin-plate splines based deformations to find a fraction of best matched 2D point pairs between features extracted from the X-ray images and those extracted from the 3D model. The estimated point pairs are then used to set up a set of 3D point pairs such that we turn a 2D/3D reconstruction problem to a 3D/3D one, whose solutions are well studied. Incorporating this 2D/3D correspondence building method, a 2D/3D reconstruction scheme combining a statistical instantiation with a regularized shape deformation has been developed. Comprehensive experiments on clinical datasets and on images of cadaveric femurs with both non-pathologic and pathologic cases are designed and conducted to evaluate the performance of the 2D/3D correspondence building method as well as that of the 2D/3D reconstruction scheme. Quantitative and qualitative evaluation results are given, which demonstrate the validity of the present method and scheme.

Introduction

The applications of X-ray imaging in orthopaedic surgery are pervasive, both pre-operatively and intra-operatively. Pre-operative planning based on information about the patient anatomy provided by conventional X-ray imaging has been established since several years (Eggli et al., 1998). Due to the projective character of X-ray imaging, accuracy of pre-operative X-ray radiograph based planning is restricted. Intra-operatively, fluoroscopy is a valuable imaging tool for visualizing underlying bone and surgical tools in various orthopaedic procedures. However, disadvantages of intra-operative fluoroscopy are also apparent, including two-dimensional (2D) projection image from single view, limited field of view, distorted image, and high radiation to both the patient and the surgical team. Various papers (Joskowicz et al., 1998, Hofstetter et al., 1999, Yao et al., 2001, Tate et al., 2001, Livyatan et al., 2002, Laurence et al., 2005) have described methods of calibration and registration of X-ray images using a positional localizer, thus allowing to compute the position of the surgical tools relative to the patient anatomy with respect to acquired images during intervention. However, the surgeon still needs to mentally fuse projection images taken from different view points. No real three-dimensional (3D) information is available. One way to address these problems is to build a statistical shape model (SSM) and to adapt the SSM to the patient’s individual anatomy based on a limited number (e.g. 2) of calibrated X-ray images. The reconstructed shape model can then provide detailed 3D information for the considered anatomical structure.

However, constructing a 3D bone surface model from a limited number of calibrated 2D X-ray images and a 3D SSM is a challenging task, especially when we would like to construct a patient-specific surface model of a bone with pathology. Moreover, inherent to the surgical navigation application is the high accuracy requirement. When surface reconstruction is used for the purpose of surgical guidance, the error of the reconstructed model should be in the range of surgical usability, which is typically in the area of 1.5 mm average error (2–3 mm, worst case) as suggested by Livyatan et al. (2003) under the context of 2D/3D image registration. In this paper, we present a 2D/3D reconstruction scheme that can seamlessly handle both non-pathologic and pathologic cases, even when our SSM is constructed from training instances without pathology. At the heart of our approach lies the combination of sophisticated surface reconstruction techniques and a novel algorithm for establishing correspondences between the 2D X-ray images and the 3D SSM.

The first part of this paper deals with the correspondence establishment. Our 2D/3D correspondence building method is based on a non-rigid 2D point matching process, which iteratively uses a symmetric injective nearest-neighbor mapping operator and 2D thin-plate splines (TPS) based deformations to find a fraction of best matched 2D point pairs between features extracted from the 2D images and those extracted from the 3D models (Zheng, 2006). The estimated point pairs are then used to set up a set of 3D point pairs such that we turn a 2D/3D reconstruction problem to a 3D/3D one, whose solutions are well studied (Zheng et al., 2007b).

Incorporating the correspondence building method, the second part of this paper deals with accurately fitting the SSM to the input images. The fitting problem is formulated as a three-stage optimal estimation process (Zheng and Nolte, 2006). Fig. 1 shows a schematic diagram of the 2D/3D fitting scheme. The first stage, scaled rigid registration, is to iteratively estimate a scale and a rigid transformation between the mean surface model of the SSM and the input images using an adapted iterative closest point (ICP) algorithm (Guéziec et al., 1998). The second stage, statistical instantiation, stably instantiates a surface model from the SSM using a Mahalanobis prior based statistical approach (Rajamani et al., 2007). This surface model is then fed to the third stage, regularized shape deformation. In this stage, we further refine the statistically instantiated surface model using an alternative derivation of the familiar interpolating thin-plate splines (TPS) (Bookstein, 1989) that enables weighting between the SSM instantiated surface model and the TPS interpolation (Zheng et al., 2007b).

Section snippets

Related Work

Statistical shape analysis (Kendall, 1989, Small, 1996, Dryden and Mardia, 1998) is an important tool for understanding anatomical structures from medical images. Statistical models give efficient parameterizations of the shape variations found in a collection of sample models of a given population. Model based approaches are popular (Turk and Pentland, 1991, Cootes et al., 1994, Corouge et al., 2003) due to their ability to robustly represent objects. In the last few years, constructing a

Statistical model construction

The first step is to build a statistical shape model from a training database. Several different geometric representations have been used to model anatomy. Bookstein (1986) uses landmarks to capture the important geometric features. The active shape model (ASM) of Cootes et al. (1995) represents an object’s geometry as a dense collection of boundary points. Cootes et al. (1998) have augmented their statistical models to include the variability of the image information as well as shape. Kelemen

A 2D/3D correspondence building method

Given a few fluoroscopic images, our task is to establish correspondences between the input images and a model estimated from the PDM. Here, we assume that the input images are calibrated and registered to a common coordinate system. And for a pixel in an input image we can always find a projection ray emitting from the focal point of the associated image through the pixel.

3D/3D Reconstruction

Using the developed algorithm for establishing correspondences, we can always find a set of 3D point pairs given an initial model state, i.e., the initial scale and the initial pose parameters of our point distribution model. The problem of surface reconstruction is then solved optimally in three stages (Zheng and Nolte, 2006): scaled rigid registration, statistical instantiation, and regularized shape deformation.

Experiments and results

The primary applications that we focus on are hip surgeries such as total hip replacement (THR) or hip resurfacing. Hence we began by concentrating on the proximal femur. To evaluate the performance of the present 2D/3D correspondence building method and that of the 2D/3D reconstruction scheme, comprehensive experiments were conducted in two co-institutions using two different point distribution models.

The PDM used in the first co-institution (MEM Research Center), which we named as MEM-PDM,

Discussions and conclusions

In this paper, we presented a 2D/3D correspondence building method. Incorporating this 2D/3D correspondence building method, we developed a 2D/3D reconstruction scheme combining a statistical instantiation with a regularized shape deformation. We designed and conducted comprehensive experiments on different types of data to evaluate the performance of the 2D/3D correspondence building method as well as that of the 2D/3D reconstruction scheme.

Our 2D/3D correspondence building method belongs to

Acknowledgements

The authors gratefully acknowledge the contributions of Dr. Moritz Tannast and Dr. Xuan Zhang for their help in experimental setup. They would also like to thank the anonymous reviewers whose comments and suggestions helped improve the original manuscript. This work was partially supported by Swiss National Science Foundation through the project NCCR CO-ME.

References (56)

  • F. Bookstein

    Principal warps: thin-plate splines and the decomposition of deformations

    IEEE Transactions on Pattern Analysis Machine Intelligence

    (1989)
  • J. Canny

    A computational approach to edge detection

    IEEE Transactions on Pattern Analysis and Machine Intelligence

    (1986)
  • Y. Chen et al.

    Automatic extraction of femur contours from hip X-ray images

  • T.F. Cootes et al.

    The use of active shape models for locating structures in medical images

    Image and Vision Computing

    (1994)
  • Cootes, T.F., Edwards, G.J., Taylor, C.J., 1998. Active appearance models. In: ECCV 1998. pp....
  • R.H. Davies et al.

    A minimum description length approach to statistical shape modeling

    IEEE Transactions on Medical Imaging

    (2002)
  • Donato, G., Belongie, S., 2002. Approximate thin plate spline mappings. In: ECCV 2002, Part III. pp....
  • Dong, X., González Ballester, M.A., Zheng, G., 2007. Automatic extraction of femur contours from calibrated...
  • I. Dryden et al.

    Statistical Shape Analysis

    (1998)
  • S. Eggli et al.

    The value of preoperative planning for total hip arthroplasty

    The Journal of Bone and Joint Surgery

    (1998)
  • T. Evgeniou et al.

    Regularization networks and support vector machine

    Advances in Computational Mathematics

    (2000)
  • M. Fleute et al.

    Nonrigid 3D/2D registration of images using statistical models

  • Gollmer, S., 2006. Evaluation and enhancement of a procedure for generating a 3D bone model using radiographs. Master...
  • Gremban, K., Thorpe, C., Kanade, T., 1988. Geometric camera calibration using systems of linear equations. In: IEEE...
  • A. Guéziec et al.

    Anatomy-based registration of ct-scan and intraoperative x-ray images for guiding a surgical robot

    IEEE Transactions on Medical Imaging

    (1998)
  • Hertzmann, A., Zorin, D., 2000. Illustrating smooth surfaces. In: SIGGRAPH 2000. pp....
  • R. Hofstetter et al.

    Fluoroscopy as an image means for computer-assisted surgical navigation

    Computer Aided Surgery

    (1999)
  • R. Hofstetter et al.

    Computer-assisted fluoroscopy-based reduction of femoral fractures and antetorsion correction

    Computer Aided Surgery

    (2000)
  • Cited by (134)

    • Computer-assisted orthopedic surgery

      2022, Human Orthopaedic Biomechanics: Fundamentals, Devices and Applications
    View all citing articles on Scopus
    View full text