A comparative review of camera calibrating methods with accuracy evaluation

Abstract

Camera calibrating is a crucial problem for further metric scene measurement. Many techniques and some studies concerning calibration have been presented in the last few years. However, it is still difficult to go into details of a determined calibrating technique and compare its accuracy with respect to other methods. Principally, this problem emerges from the lack of a standardized notation and the existence of various methods of accuracy evaluation to choose from. This article presents a detailed review of some of the most used calibrating techniques in which the principal idea has been to present them all with the same notation. Furthermore, the techniques surveyed have been tested and their accuracy evaluated. Comparative results are shown and discussed in the article. Moreover, code and results are available in internet.

Introduction

Camera calibration is the first step towards computational computer vision. Although some information concerning the measuring of scenes can be obtained by using uncalibrated cameras [1], calibration is essential when metric information is required. The use of precisely calibrated cameras makes the measurement of distances in a real world from their projections on the image plane possible [2], [3]. Some applications of this capability include:

• Dense reconstruction: Each image point determines an optical ray passing through the focal point of the camera towards the scene. The use of more than a single view of a motionless scene (taken from a stereoscopic system, a single moving camera, or even a structured light emitter) permits crossing both optical rays to get the metric position of the 3D points [4], [5], [6]. Obviously, the correspondence problem has to be previously solved [7].

• Visual inspection: Once a dense reconstruction of a measuring object is obtained, the reconstructed object can be compared with a stored model in order to detect any manufacturing imperfections such as bumps, dents or cracks. One potential application is visual inspection for quality control. Computerized visual inspection allows automatic and exhaustive examination of products, as opposed to the slow human inspection which usually implies a statistical approach [8].

• Object localization: When considering various image points from different objects, the relative position among these objects can be easily determined. This has many possible applications such as in industrial part assembly [9] and obstacle avoidance in robot navigation [10], [11], among others.

• Camera localization: When a camera is placed in the hand of a robot arm or on a mobile robot, the position and orientation of the camera can be computed by locating some known landmarks in the scene. If these measurements are stored, a temporal analysis allows the handler to determine the trajectory of the robot. This information can be used in robot control and path planning [12], [13], [14].

Camera calibration is divided into two phases. First, camera modelling deals with the mathematical approximation of the physical and optical behavior of the sensor by using a set of parameters. The second phase of camera calibration deals with the use of direct or iterative methods to estimate the values of these parameters. There are two kinds of parameters in the model which have to be considered. On the one hand, the intrinsic parameter set, which models the internal geometry and optical characteristics of the image sensor. Basically, intrinsic parameters determine how light is projected through the lens onto the image plane of the sensor. The other set of parameters are the extrinsic ones. The extrinsic parameters measure the position and orientation of the camera with respect to a world coordinate system, which, in turn, provides metric information with respect to a user-fixed coordinate system instead of the camera coordinate system.

Camera calibration can be classified according to several different criteria. For instance, (1) Linear versus non-linear camera calibration (usually differentiated depending on the modelling of lens distortion) [15]. (2) Intrinsic versus extrinsic camera calibration. Intrinsic calibration is concerned only with obtaining the physical and optical parameters of the camera [16], [17]. Besides, extrinsic calibration concerns the measurement of the position and orientation of the camera in the scene [18], [19]. (3) Implicit [20] versus explicit [21] calibration. Implicit calibration is the process of calibrating a camera without explicitly computing its physical parameters. Although, the results can be used for 3D measurement and the generation of image coordinates, they are useless for camera modelling as the obtained parameters do not correspond to the physical ones [22]. Finally, (4) the methods which use known 3D points as a calibrating pattern [23], [24] or even a reduced set of 3D points [25], [26], with respect to others which use geometrical properties in the scene such as vanishing lines [27] or other line features [28], [29].

These different approaches can also be classified regarding the calibration method used to estimate the parameters of the camera model:

• Non-linear optimization techniques. A calibrating technique becomes non-linear when any kind of lens imperfection is included in the camera model. In that case, the camera parameters are usually obtained through iteration with the constraint of minimizing a determined function. The minimizing function is usually the distance between the imaged points and the modelled projections obtained by iterating. The advantage of these iterating techniques is that almost any model can be calibrated and accuracy usually increases by increasing the number of iterations up to convergence. However, these techniques require a good initial guess in order to guarantee convergence. Some examples are described in classic photogrammetry [30] and Salvi [31].

• Linear techniques which compute the transformation matrix. These techniques use the least squares method to obtain a transformation matrix which relates 3D points with their 2D projections. The advantage here is the simplicity of the model which consists in a simple and rapid calibration. One drawback is that linear techniques are useless for lens distortion modelling, entailing a rough accuracy of the system. Moreover, it is sometimes difficult to extract the parameters from the matrix due to the implicit calibration used. Some references related to linear calibration can be found in Hall [20], Toscani-Faugeras [23], [32] and Ito [15].

• Two-step techniques. These techniques use a linear optimization to compute some of the parameters and, as a second step, the rest of the parameters are computed iteratively. These techniques permit a rapid calibration considerably reducing the number of iterations. Moreover, the convergence is nearly guaranteed due to the linear guess obtained in the first step. Two-step techniques make use of the advantages of the previously described methods. Some references are Tsai [24], Weng [33] and Wei [22].

This article is a detailed survey of some of the most frequently used calibrating techniques. The first technique was proposed by Hall in 1982 and is based on an implicit linear camera calibration by computing the 3×4 transformation matrix which relates 3D object points with their 2D image projections [20]. The latter work of Faugeras, proposed in 1986, was based on extracting the physical parameters of the camera from such a transformation technique, thus it is explained as the second technique [23], [32]. The following methods are based on non-linear explicit camera calibration, including the modelling of lens distortion. Hence, the first one is a simple adaptation of the Faugeras linear method with the aim of including radial lens distortion [31], [34]. The widely used method proposed by Tsai, which is based on a two-step technique modelling only radial lens distortion, is also detailed [24]. Finally, the complete model of Weng, which was proposed in 1992, including three different types of lens distortion, is explained as the last technique [33]. Note that one of the principal problems to understand a calibrating technique in detail is the lack of notation standardization in mathematical equations and the use of different sets of coordinate systems. Both limitations complicate the comparing of techniques, thus a great deal of effort has been made to present the survey using the same notation. All five techniques are explained herein and their 2D and 3D accuracy shown and discussed. A brief overview of camera accuracy evaluation [35] is included with the aim of using the same tools to compare different calibrating techniques implemented.

This article is structured as follows. Section 2 deals with camera modelling and how the camera model is gradually obtained by a sequence of geometrical transformations is explained. Section 3 describes the five different techniques of camera calibration, which estimate the parameters of the camera model. Then, a few methods for accuracy evaluation of camera calibrating techniques are explained in Section 4. Finally, both 2D and 3D accuracy of each calibration technique have been measured and their results are shown and compared. The paper ends with conclusions.