1. Introduction

Omnidirectional Vision systems that enable 360° imaging have been widely used in several research fields such as robot navigation, telepresence, close-range Photogrammetry and virtual reality (^{Kang and Szeliski, 1997}; ^{Yagi, 1999}; ^{Spacek, 2005}; ^{Sturm et al., 2010}). Omnidirectional images can be acquired using: camera with fisheye lenses; moving cameras or optical elements; catadioptric systems (camera and mirror); and multiple cameras with divergent views (^{Sturm et al. 2010}). (^{Van Den Heuvel et al., 2006}) presented a system developed by Cyclomedia composed of two fisheye lens camera with a FOV (Field of View) of approximately 185°, generating omnidirectional images called Cycloramas.

The fisheye lens camera should be calibrated to be used in applications that require high accuracy. In Photogrammetry, the Collinearity mathematical model, based on perspective projection combined with lens distortion models, is generally used in the camera calibration process. However, fisheye lenses are designed following different projections models such as: Stereo-graphic, Equi-distant, Orthogonal and Equi-solid-angle. In general, the fisheye lenses follow the Equi-distant and Equi-solid-angle projections (^{Abraham and Förstner, 2005}; ^{Schneider et al., 2009}).

(^{Abraham and Förstner, 2005}) presented rigorous mathematical models for the calibration of a stereo system composed of two fisheye lens cameras and for the epipolar rectification of the images acquired by this dual system. (^{Schneider et al., 2009}) presented the calibration of a Kodak DSC 14 Pro with Nikkor 8 mm fisheye lens, which follows the Equi-distant projection. The rigorous mathematical models based on Stereo-graphic, Equi-distant, Orthogonal and Equi-solid-angle projections were used in combination with symmetric radial, decentering and affinity distortion models. (^{Willneff and Wenisch, 2011}) extended the distortion model by a fourth radial distortion coefficient to calibrate fisheye cameras with short focal length (2.6mm - 1.8mm).

(^{Puig et al., 2012}) performed a comparative analysis between some existing generalized (non-physical) mathematical models (^{Barreto and Araujo, 2002}; ^{Scaramuzza et al., 2006}; ^{Mei and Rives, 2007}; ^{Puig et al., 2011}) to calibrate central omnidirectional systems and fisheye lens cameras. It was verified that the calibration methods achieved accurate results in the 3D object reconstruction with GCPs (Ground Control Points) well distributed over the images. (^{Puig et al., 2011}) applied the DLT (Direct Linear Transformation) mathematical model to relate object (3D) and image (2D) spaces in the calibration step.

The main aim of this paper is to present the results achieved in the calibration of a Fuji-Finepix S3pro digital camera with Bower-Samyang 8mm fisheye lens using rigorous mathematical models. The contribution of this work is the assessment of the calibration results of a Bower-Samyang fisheye lens, which is a low cost option in comparison with other fisheye lenses and it is built following the Stereo-graphic projection model (^{Charles, 2009}), which is less commonly used (^{Ray, 2002}). Considering that, experimental assessment on the calibration of Bower-Samyang fisheye lens becomes a relevant issue to enable its use in future projects and in photogrammetric tasks. A correlation analysis between IOPs (Interior Orientation Parameters) and EOPs (Exterior Orientation Parameters) was also conducted.

The mathematical models used in this work are presented in Section 2. Section 3 describes the experiments and results of the fisheye lens camera calibration. Finally, the conclusions are presented in Section 4.

2. Calibration of fisheye cameras

Camera calibration is a procedure to estimate the IOPs, which enable the reconstruction of the perspective bundle that generated the image. The IOPs of digital cameras are the focal length, the principal point coordinates and coefficients for systematic errors correction (lens distortions: symmetric radial and decentering; and affinity). The most used model for camera calibration are the collinearity equations (^{Schmid, 1959}) considering additional parameters - IOPs, as presented in Equation 1. (^{Mikhail et al., 2001}).

where f is the focal length; (X_{C}, Y_{C}, Z_{C}) are the 3D point coordinates in the photogrammetric reference system (Equation 2); (x_{f}, y_{f}) are the image point coordinates in the photogrammetric reference system; (x',y') are the image point coordinates in a reference system parallel to the photogrammetric system which origin is in the image centre; (x_{0},y_{0}) are the coordinates of the principal point (pp) and; Δx and Δy are equations describing the effects of systematic errors (Equation 3).

where r_{ij} (i and j from 1 to 3) are rotation matrix elements that relates the object to the image reference system; (X, Y, Z) are the coordinates of a point in the object reference system; and (X_{CP}, Y_{CP}, Z_{CP}) are the coordinates of the perspective centre (PC) in the object reference system.

where K_{1}, K_{2}, K_{3} represent symmetric radial distortion coefficients; P_{1} e P_{2} are the decentering distortion coefficients; A and B are the affinity parameters; x_{f} = x'-x_{0}; y_{f} = y'-y_{0} and; . The radial symetric and decetering distortions formulation were developed by (^{Brown, 1971}). The decentering distortion model is based on previous work presented in (^{Conrady, 1919}).

The collinearity equations are generally used in the calibration process; however the image acquisition with fisheye lens camera does not follow the collinearity condition. In the perspective projection α = β (see the angles in Figure 1), except by the small deviations caused by lens distortion. With fisheye lens, the rays are deflected toward the optical axis as shown in Figure 1.

In general, fisheye lenses follow the Equi-distant and Equi-solid-angle projections (^{Abraham and Förstner, 2005}; ^{Schneider et al., 2009}), however Bower-Samyang 8mm lens, used in this work, was built following a quasi-Stereo-graphic projection (^{Charles, 2009}). ^{Charles (2009}) presents a discussion on the technical features of Bower-Samyang fisheye lens camera. This author points out that Bower-Samyang is neither a fisheye lens nor a perspective projection lens. ^{Charles (2009}) classifies this lens as quasi-Stereo-graphic, because of the small focal length. Figure 2 depicts the Bower-Samyang fisheye lens. Table 1 presents the mathematical models based on Stereo-graphic, Equi-distant, Orthogonal or Equi-solid-angle projections. More details about the geometric description of these projections is presented in (^{Hughes et al., 2010}).

The analysis of Table 1 shows a difference in the focal length sign between the models. In Photogrammetry, generally, the photogrammetric z axis points to the negative plane, and as a consequence, Z_{c} have negative values. This is appliable only in perspective and Equi-distant projections. In the other projections, this is not possible, because Z_{c} is squared, and Z_{c} ^{2} is a positive value. The focal length sign is a negative value for the perspective and equi-distant projections, when considering the sensor as a diapositive. To avoid inconsistencies, for the other projections the f sign is a positive value, because the reference system is rotated around y axis, as shown in Figure 3, which presents the photogrammetric systems for the different mathematical models, justifying these differences.

3. Experiments and results

The Fuji Finepix S3PRO camera with Bower-Samyang 8mm fisheye lens was calibrated using the Collinearity, Stereo-graphic, Equi-distant, Orthogonal and Equi-solid-angle mathematical models. These mathematical models were implemented in an in-house developed software package called TMS (Triangulation with Multiple Sensors) (^{Ruy et al., 2009}; ^{Marcato Junior and Tommaselli, 2013}).

Twelve (12) images collected by Fuji Finepix S3pro with the fisheye lenses, in three exposure stations were used in the calibration process (see Figure 4). In order to automate the calibration process a special 3D terrestrial calibration field with coded targets was created. This 3D field is composed of 139 coded targets, using the ARUCO style (^{Garrido-Jurado et al., 2014}). These targets have two main parts: an external crown, which is a rectangle and 5x5 internal squares that can code 10 bits of information. With this scheme, 1024 values can be encoded. More details about these coded targets are presented in (^{Tommaselli et al., 2014}) and (^{Silva et al., 2014}). A public existing software (^{Garrido-Jurado et al., 2014}) was adapted to perform automatically the location, identification and accurate measurement of the four corners of the external crown of the calibration field targets (^{Silva et al., 2014}). With the adapted software, most of the existing coded targets were automatically located, recognized and the coordinates of the corners of the bounding rectangle are extracted with subpixel precision. To improve the image quality, and consequently to increase the number of corners automatically detected, the shadows were also interactively segmented and enhanced. Some corners that failed to be detected automatically with the software, were interactively measured using MID software (^{Reiss and Tommaselli, 2003}) to provide enough points with suitable geometry for the camera calibration.

The 3D coordinates of the 556 GCPs on the calibration field (corners of the external square of each target) were estimated using geodetic and photogrammetric methods. To establish the reference frame, four reference points were surveyed during eight hours with a double frequency GNSS (Global Navigation Satellite System) receiver. To verify the precision of the 3D coordinates of these points, the distances between then were measured with a Total Station and the discrepancies among the electronically measured and those computed from the 3D coordinates were around 1mm. Forty-three (43) images of the calibration field were acquired by a Hasselblad H3D (50 Megapixels) 35 mm lens camera, with a GSD (Ground Sample Distance) of 3 mm. The coordinates of the remaining 552 points were estimated with on the job calibration of the Hasselblad camera, being achieved a precision of approximately 3 mm. These points were considered as photogrammetric points in the on the job calibration. More details of this process are described in (^{Moraes et al., 2013}). This set of 3D coordinates generated by photogrammetric calibration was used as ground control in the following experiments.

Fuji Finepix S3pro is a 12.1 Megapixel (4256x2848 pixels) digital camera with a pixel size of 5.4 µm. The sensor frame size is 23.0 mm x 15.5 mm. Experiments were conducted with different sets of IOPs. Table 2 presents the standard deviation of unit weight estimated in the bundle adjustment for each mathematical model (a priori value was set as 1). The coordinates of 325 GCPs were introduced in the bundle adjustment as weighted constraints with a standard deviation of 3 mm. An exception occurred to the four GCPs estimated by GNSS surveying, which were weighted considering a standard deviation of 1 mm.

The analysis of Table 2 shows that the standard deviation of unit weight (a posteriori) estimated with the collinearity model is larger when compared to the other models, because the image coordinates residuals are larger (see Figure 5, that shows the residuals distribution for the image 5 of Figure 4, considering all IOPs). The standard deviation of unit weight for the Stereo-graphic, Equi-distant, Equi-solid-angle and Orthogonal are smaller than 1, which is the a priori value. Table 2 also demonstrates that the results for these four models can be considered similar in the light of an analysis considering the standard deviation of unit weight. Table 2 reveals that the best result under this criteria (smaller ) is achieved when all the IOPs (f, x_{0}, y_{0}, K_{1}, K_{2}, K_{3}, P_{1}, P_{2}, A, B) are considered.

The residuals presented in Figure 5 are compatible with the previous analysis, based on the standard deviation of unit weight. The residuals are larger in perspective model, mainly in the image extremities. Table 3presents the estimated IOPs and standard deviation for the Fuji Finepix S3 pro considering all the IOPs.

The estimated standard deviation of the IOPs with Stereo-graphic, Equi-distant, Equi-solid-angle and Orthogonal models are smaller when compared to the collinearity model, according to Table 3, which can partially explained by the smaller sigma naught. It is also verified that the standard deviation of the focal length is smaller than 0.4 pixels for all models, showing a precise estimation for this parameter. It is important to mention that the estimated radial distortion parameters (K_{1}, K_{2}, K_{3}) for the collinearity model partially absorbs the effect of the rays' refraction toward the optical axis, but this modeling is not enough to recover the inner bundle geometry, in comparison with the other models assessed. In the image limits, these values would vanish to infinite, for an angle of incidence of 180º.

The estimated standard deviations of certain parameters are larger than their own estimated values, as presented in Table 3. This occurs, for example, with the affinity parameter B, when considering the perspective projection. The affinity effects on the x and y photogrammetric coordinates are 0.51 and -0.33 pixel, respectively. The resultant of these values is larger than the measurement error that was performed automatically, with subpixel precision, indicating that the affinity parameters are indeed significant in this case.

The quality of the estimated IOPs was assessed using 13 independent checkpoints. The checkpoint coordinates were estimated through a bundle adjustment with the first and ninth images (see Figure 3) considering the IOPs as absolute constraint. Four GCPs were used to estimate the EOPs and the checkpoint coordinates. The RMSE (Root Mean Square Error) of the checkpoint coordinates are presented in Table 4.

Table 4 demonstrates that the Stereo-graphic, Equi-distant, Equi-solid-angle and Orthogonal models provided better results in the 3D reconstruction of the checkpoint coordinates in comparison to the collinearity model. The analysis of Table 4 also shows that using the sets of IOPs including the parameters P_{1}, P_{2}, A and B did not improve significantly the 3D reconstruction.

Finally, a correlation analysis between the IOPs and EOPs was conducted. Table 5 presents the correlation between the focal length and the EOPs, considering the experiment with all IOPs. The results presented in Table 5 revealed that the correlations between focal length and X_{0}, which is the coordinate representing the depth, are smaller than 0.6. These low correlations value can be explained by the high scale variations caused by the fisheye field of view. The correlations between the other IOPs and EOPs are also small, less than 0.6 in all cases.

4. Conclusion

The aim of this paper was to assess the results of calibration trials performed with a Fuji-Finepix S3pro camera with Bower-Samyang 8mm lens, using rigorous mathematical models. Bower-Samyang 8mm, in comparison to the other fisheye lens, is cheaper and is based in a different projection, Stereo-graphic. ^{Charles (2009}) points out that Bower-Samyang is neither a fisheye lens nor a perspective projection lens, classifying it as a quasi-stereographic lens.

The mathematical models based on Stereo-graphic, Equi-distant, Orthogonal and Equi-solid-angle projections were implemented in an in-house software called TMS. Experiments were conducted with images from a 3D field calibration with coded targets.

The experiments demonstrated that collinearity mathematical model, which is based on perspective projection, presented the less accurate results, which was expected because Bower-Samyang 8mm lens is not based on perspective projection. Stereo-graphic, Equi-distant, Orthogonal and Equi-solid-angle projections presented similar results in the studied cases, although Bower-Samyang fisheye lens was built based on Stereo-graphic projection. The parameters for systematic errors modelling (lens distortions: symmetric radial and decentering; and affinity) absorbed the radial displacements caused by different fisheye projection models.

It was also verified through the experiments a low correlation between IOPs and EOPs, which is justified by the high scale variations caused by a fisheye lens. The experiment aiming the 3D reconstruction showed that Fuji-Finepix S3pro camera with Bower-Samyang 8mm lens, after rigorous calibration with bundle adjustment, can be used for high accuracy applications in close range Photogrammetry.

In future work, techniques will be developed to fully automate the measurement of image points and the calibration process. Experiments will also be performed to compare rigorous and generalized mathematical models to calibrate fisheye lens camera. Line based methods for fisheye calibration is also a topic for further research.