OPEN ACCESS
In this paper we present a robust method for solving the uncalibrated photometric stereo problem in the Lambertian framework, in presence of outliers such as shadows or specular highlights. We show how to detect such outliers so as to accurately estimate the light sources, which are assumed to have uniform magnitude in order to solve for the generalized bas-relief ambiguity. We then propose a way to recover both the normal and the albedo in every pixel in presence of outliers, which is made possible thanks to the accurate estimation of the light sources. Our method is validated both qualitatively and quantitatively on synthetic and realworld datasets, and we compare its efficiency and accuracy to the most state-of-the-art existing techniques.
Extended Abstract
We tackle the problem of 3D-reconstruction from m > 3 images, taken from the same point of view but under m different lightings. This problem, called photometric stereo, is a classical inverse problem in computer vision (Woodham,1980),which can be solved easily when:
– lightings are assumed to be parallel and uniform;
– thereflectanceoftheobjecttoreconstructislambertian(nospecularhighlight);
– the vectors representing the lightings are known, both in norm and direction;
– shadows (both self- and cast-) are neglected.
In this paper, we question the three latter hypothesis. To do so, we combine the methods described in (Quéau, Durou, 2013) and in (Durix et al., 2013) so as to propose a robust method for 3D-reconstruction when the lightings are unknown.
When the lightings are not provided, the problem becomes the so-called uncalibrated photometric stereo problem, in which both the normals, the albedo and the lightings must be estimated simultaneously. Unfortunately, this is an ill-posed problem: those informations can be estimated only up to a global ambiguity. If the normal field is assumed to be sufficiently smooth (integrability hypothesis),this ambiguity reduces to the generalized bas-relief ambiguity, which depends only on three parameters. The estimation of these parameters require an additional constraint to be introduced: we extend the study led in (Quéau, Durou, 2013) and show how to solve the generalized bas-relief ambiguity when all the lightings are assumed to have the same intensity. This method is evaluated by comparing the obtained results with the calibrated case.
Sincethequalityofthe3D-reconstructionobtainedbyphotometricstereodepends strongly on the absence of shadows and specularities, we also extend the work described in (Durix et al., 2013) and propose a robust framework for uncalibrated photometric stereo. To this purpose, we introduce a three-steps method which allows us to reducetheproblemtothatoftherobustestimationofboththenormalsandthealbedo in the calibrated case.
Complex phenomena such as shadows and outliers are treated as outliers to the ideal lambertian model. Such outliers are first identified and excluded from the light estimation process, by comparing the intensity matrix to its rank-3 approximation. Then, normals and albedo are estimated using the previously estimated lightings: we introduce a weighted least square approach which allows to efficiently reduce the influence of outliers on the reconstruction. Finally, the generalized bas-relief ambiguity is solved by assuming that all lightings have the same intensity. This approach is evaluated by reconstructing scenes where shadows and specularities are present, and we compare it to other state-of-the-art approaches.
RÉSUMÉ
Nous présentons une méthode robuste de résolution du problème de la stéréophotométrie non calibrée dans le cadre lambertien, en présence d’écarts à ce modèle tels que les ombres ou les taches spéculaires. Nous montrons d’abord comment détecter de tels écarts afin d’estimer de façon plus précise les paramètres caractéristiques des sources lumineuses, qui sont supposées de même intensité afin de lever l’ambiguïté de bas-relief généralisée inhérente au problème. Nous montrons ensuite comment estimer la normale et l’albédo en chaque point en prenant en compte les écarts au modèle, ce qui est facilité par l'estimation précise des éclairages. Nous validons notre méthode qualitativement et quantativement par la reconstruction 3D d’objets synthétiques et réels, et nous comparons son efficacité et sa précision aux meilleures méthodes actuelles.
3D-reconstruction, shape-from-X, uncalibrated photometric stereo, lighting estimation, generalized bas-relief ambiguity, shadows, specular highlights.
MOTS-CLÉS
reconstruction 3D, shape-from-X, stéréophotométrie non calibrée, estimation de l’éclairage, ambiguïté de bas-relief généralisée, ombres, taches spéculaires.
Abrams A., Hawley C., Pless R. (2012). Heliometric Stereo: Shape from Sun Position. In ECCV 2012, LNCS 7573, p. 357–370.
Ackermann J., Langguth F., Fuhrmann S., Goesele M. (2012). Photometric Stereo for Outdoor Webcams. In CVPR 2012, p. 262–269.
Alldrin N. G., Mallick S. P., Kriegman D. J. (2007). Resolving the Generalized Bas-relief Ambiguity by Entropy Minimization. In CVPR 2007.
Barsky S., Petrou M. (2003). The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. PAMI, vol. 25, no 10, p. 1239–1252.
Belhumeur P. N., Kriegman D. J., Yuille A. L. (1999). The Bas-Relief Ambiguity. IJCV, vol. 35, no 1, p. 33–44.
Chandraker M. K., Agarwal S., Kriegman D. (2007). Shadowcuts: Photometric stereo with shadows. In CVPR 2007.
Chandraker M. K., Kahl F., Kriegman D. J. (2005). Reflections on the Generalized Bas-Relief Ambiguity. In CVPR 2005, vol. 1, p. 788–795.
ChangJ.,LeeK.,LeeS. (2007). Multiviewnormalfieldintegrationusinglevelsetmethods. In CVPR 2007.
Drbohlav O., Chantler M. (2005). Can Two Specular Pixels Calibrate Photometric Stereo? In ICCV 2005, vol. 2, p. 1850–1857.
Durix B., Quéau Y., Durou J.-D., Charvillat V. (2013). Quels prétraitements pour la stéréophotométrie non calibrée? In ORASIS 2013.
Durou J.-D., Aujol J.-F., Courteille F. (2009). Integration of a Normal Field in the Presence of Discontinuities. In EMMCVPR 2009, LNCS 5681, p. 261–273.
FavaroP.,PapadhimitriT. (2012). AClosed-FormSolutiontoUncalibratedPhotometricStereo via Diffuse Maxima. In CVPR 2012, p. 821–828.
Fischer M., Bolles R. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, vol. 24, no 6, p. 381–395.
Frankot R. T., Chellappa R. (1988). A Method for Enforcing Integrability in Shape from Shading Algorithms. PAMI, vol. 10, no 4, p. 439–451.
Georghiades A. S. (2003). Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo. In ICCV 2003, vol. 2, p. 816–823.
Georghiades A. S., Kriegman D. J., Belhumeur P. N. (2001). From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose. PAMI, vol. 23, no 6, p. 643–660. Harker M., O’Leary P. (2008). Least squares surface reconstruction from measured gradient fields. In CVPR 2008.
Hayakawa H. (1994). Photometric stereo under a light-source with arbitrary motion. JOSA A, vol. 11, no 11, p. 3079–3089.
HernándezC.,VogiatzisG.,BrostowG.J.,StengerB.,CipollaR. (2008). MultiviewPhotometric Stereo. PAMI, vol. 30, no 3, p. 548–554.
Hernández C., Vogiatzis G., Stenger B., Cipolla R. (2011). Video normals from colored lights. PAMI, vol. 33, no 10, p. 2104–2114.
Hertzmann A., Seitz S. M. (2005). Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs. PAMI, vol. 27, no 8, p. 1254–1264.
HornB.K.P. (1989). ObtainingShape from ShadingInformation. In,p. 123–171. MITPress. Horn B. K. P. (1990). Height and Gradient from Shading. IJCV, vol. 5, no 1, p. 37–75.
Horovitz I., Kiryati N. (2004). Depth from gradient fields and control points: Bias correction in photometric stereo. Image and Vision Computing, vol. 22, no 9, p. 681–694.
Ikehata S., Wipf D., Matsushita Y., Aizawa K. (2012). Robust photometric stereo using sparse regression. In CVPR 2012, p. 318–325.
Joshi M. V., Chaudhuri S. (2004). Photometric stereo under blurred observations. Pattern Recognition, vol. 3, no 23-26, p. 169–172.
Kozera R. (1991). Existence and uniqueness in photometric stereo. Applied Mathematics and Computation, vol. 44, no 1, p. 1–103.
Mukaigawa Y., Ishii Y., Shakunaga T. (2007). Analysis of photometric factors based on photometric linearization. JOSA A, vol. 24, no 10, p. 3326–3334.
Nehab D., Rusinkiewicz S., Davis J., Ramamoorthi R. (2005). Efficiently combining positions and normals for precise 3D geometry. In SIGGRAPH 2005, p. 536–543.
OnnR.,BrucksteinA.M. (1990). IntegrabilityDisambiguatesSurfaceRecoveryinTwo-Image Photometric Stereo. IJCV, vol. 5, no 1, p. 105–113. PhongB.T. (1975). Illuminationforcomputergeneratedpictures. InCommunicationsofACM 1975, p. 311–317.
Quéau Y., Durou J.-D. (2013). Résolution du problème de la stéréophotométrie non calibrée par estimation de l’intensité des éclairages. In ORASIS 2013.
Quéau Y., Lauze F., Durou J.-D. (2013). Solving the uncalibrated photometric stereo problem using total variation. In SSVM 2013, p. 270–281.
Shi B., Matsushita Y., Wei Y., Xu C., Tan P. (2010). Self-calibrating Photometric Stereo. In CVPR 2010, p. 1118–1125.
Sunkavalli K., Zickler T., Pfister H. (2010). Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows. In ECCV 2010, LNCS 6312, p. 251–264.
Tan P., Mallick S. P., Quan L., Kriegman D. J., Zickler T. (2007). Isotropy, reciprocity and the generalized bas-relief ambiguity. In CVPR 2007.
Verbiest F., Van Gool L. (2008). Photometric stereo with coherent outlier handling and confidence estimation. In CVPR 2008.
WoodhamR.J.(1980).PhotometricMethodforDeterminingSurfaceOrientationfromMultiple Images. Optical Engineering, vol. 19, no 1, p. 139–144.
Wu L., Ganesh A., Shi B., Matsushita Y., Wang Y., Ma Y. (2010). Robust Photometric Stereo viaLow-RankMatrixCompletionandRecovery. InACCV2010,LNCS6494,p.703–717.
Yuille A. L., Snow D. (1997). Shape and Albedo from Multiple Images using Integrability. In CVPR 1997, p. 158–164.
Yuille A. L., Snow D., Epstein R., Belhumeur P. N. (1999). Determining generative models of objectsundervaryingillumination:ShapeandalbedofrommultipleimagesusingSVDand integrability. IJCV, vol. 35, no 3, p. 203–222.