30 ans de démosaïçage

30 ans de démosaïçage

30 years of demosaicing

David Alleysson

Laboratoire de Psychologie et NeuroCognition (LPNC) – CNRS UMR5105 Université Pierre-Mendès France (UPMF) Bâtiment Science de l’Homme et Math (BSHM) 1251, Av. Centrale, BP 47 – 38040 Grenoble Cedex 9 – France

Corresponding Author Email: 
David.Alleysson@upmf-grenoble.fr
Page: 
561-581
 |
Received: 
14 Febuary 2004
| |
Accepted: 
N/A
| | Citation
ts21_6_561-581.pdf

OPEN ACCESS

Abstract: 

This paper proposes a review of thirty years of the development of demosaicing algorithms used in digital camera for the reconstruction of color image. Most recent digital camera used a single sensor in front of a color filter array is placed. This sensor sample a single chromatic value per spatial position and an interpolation algorithm is needed for the definition of a color image with three components per spatial position. This article shows that the whole signal and image processing technics have been used for solving this problem. Moreover, a new method proposed recently by the author and collaborators is decribed. This method based on a model of chromatic sampling by the cones in the retina highlights the nature of spatio-chromatic sampling in digital camera with single sensor.

Résumé

Cet article propose une revue de trente années de développement des algorithmes de démosaïçage utilisés dans les caméras numériques pour la reconstruction des images couleurs. La plupart des caméras numériques actuelles utilisent un seul capteur devant lequel est placée une matrice de filtres couleurs. Ce capteur échantillonne par conséquent une seule couleur par position spatiale et un algorithme d'interpolation est nécessaire pour la définition d'une image couleur avec trois composantes par position spatiale. Cet article montre que l'ensemble des techniques du traitement du signal et des images a été utilisé pour résoudre ce problème. Aussi, une nouvelle méthode proposée récemment par l'auteur et collaborateurs est décrite. Cette méthode, basée sur un modèle d'échantillonnage couleur par les cônes de la rétine, révèle la nature de l'échantillonnage spatio-chromatique dans les caméras couleur à un seul capteur.

Keywords: 

Single-chip digital cameras, interpolation

Mots clés

Caméras numériques couleur mono circuit, interpolation

1. Introduction
2. Historique Des Algorithmes De Démosaïçage
3. Généralisation Du Démosaïçage
4. Démosaïçage Par Sélection De Fréquences
5. Résultats
6. Conclusion
  References

[Acharya00] T. ACHARYA, « Efficient algorithm for color recovery from 8-bit to 24-bit color pixels », US patent 6,091,851, to Intel Corporation, Patent and Trademark Office, Washington, D.C., 2000.

[Adams95] J.E. ADAMS Jr, « Interactions Between Color Plane Interpolation and Other Image Processing Functions in Electronic Photography », Proc. SPIE, Cameras and Systems for Electronic Photography and Scientific Imaging, ed. C.N. Anagnostopoulos/M.P. Lesser, vol. 2416, 1995, pp. 144-151.

[Adams96] J.E. ADAMS Jr, J.F. HAMILTON, « Adaptive color plan interpolation in single sensor color electronic camera », US patent 5,506,619, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1996.

[Adam97-1] J.E. ADAMS Jr., « Design of practical color filter array interpolation algorithms for digital cameras », Proc. SPIE Real Time Imaging II, Vol. 3028, 1997, pp. 117-125.

[Adams97-2] J.E. ADAMS Jr. and J.F. HAMILTON, Jr., « Adaptive color plane interpolation in single sensor color electronic camera, » US patent 5,652,621, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1997.

[Adams98-1] J.E. ADAMS Jr., « Design of practical color filter array interpolation algorithms for digital cameras II », Proc. Int'l Conf. Image Processing, (ICIP'98), Chicago, IL, USA, vol. 1, Oct. 1998, pp. 488-492.

[Adams98-2] J.E. ADAMS Jr., K. PARULSKI, and K. SPAULDING, « Color processing in digital cameras », Proc. Of the IEEE, Vol. 18, no. 6, 1998, pp 20-30.

[Adcock77] W.A. ADCOCK and F.I. SKAGGS, « Single sensor time encoded color image system », US Patent 4,065,785, to Texas Instruments Incopored, Patent and Trademark Office, Washington, D.C., 1954.

[Angelopoulos94] G. ANGELOPOULOS and I. PITAS, « Multichannel Wiener filters in color image restoration », IEEE Trans. Circuit and System for video technology, Vol 4, No 1, 1994, pp. 83-87.

[Alleysson01] D. ALLEYSSON and J. HÉRAULT, « Interpolation d'images couleurs sous échantillonnées par un modèle de perception », Proc. GRETSI'2001, Toulouse, Sept. 2001.

[Alleysson02] D. ALLEYSSON, S. SÜSSTRUNK, and J. HÉRAULT, « Color Domosaicing by estimating luminance and opponent chromatic signals in the Fourier domain. », Proc. IS&T/SID 10th Color Imaging Conf., CIC10, Scottsdale, Arizona, pp. 331-336 , 2002.

[Alleysson03] D. ALLEYSSON, S. SÜSSTRUNK, and J. MARGUIER, « Influence of spectral sensitivity functions on color demosaicing. » Proc. IS&T/SID 11th Color Imaging Conf., CIC11, Scottsdale, Arizona, pp. 351-357 , 2003.

[Alleysson04] D. ALLEYSSON, S. SÜSSTRUNK, and J. HÉRAULT, « Linear Demosaicing inspired by the Human Visual System », IEEE Trans. on Image Processing, to appear in March 2005.

[Banning54] T.A BANNING JR, « Color television and the like », US Patent 2,683,769, Patent and Trademark Office, Washington, D.C., 1954.

[Banning56] T.A BANNING JR, « Color television and the like », US Patent 2,755,334, Patent and Trademark Office, Washington, D.C., 1956.

[Bayer76] B.E. BAYER, « Color imaging array », US Patent 3,971,065, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1976.

[Brainard94] D.H. BRAINARD, « Bayesian method for reconstructing color images from trichromatic samples », Proc. IS&T 47th Annual Conference ICPS, Rochester, NY, 1994, pp. 375-380.

[Brainard95] D.H. BRAINARD, and D. SHERMAN, « Reconstructing Images from Trichromatic Samples, from Basic Research to Practical Applications », Proc. IS&T/SID 3rd Color Imaging Conference, (CIC'95), Scottsdale, Arizona, 1995, pp. 4-10.

[Buchsbaum83] G. BUCHSBAUM and A. GOTTSCHALK, « Trichromacy, opponent colours coding and the optimum colour information transmission in the retina », Proc. R.. Soc. Lond., B220, 1983, pp. 89-113.

[Chang99] E. CHANG, S. CHEUNG, and D. PAN, « Color filter array recovery using a threshold-based variable number of gradients », IS&T/SPIE Conference on Sensors, Cameras and Applications for Digital Photography, San Jose California, Vol. 3650, 1999, pp. 36-43.

[Cok86-1] D.R. COK, « Single-chip electronic color camera with colordependent birefringent optical spatial frequency filter and red and blue signal interpolating circuit. », US Patent 4,605,956, to Eastman Kodak Compagny, Patent and Trademark Office, Washington, D.C., 1986.

[Cok86-2] D.R. COK, « Signal processing method and apparatus for sampled image signals », US patent 4,630,307, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1986.

[Cok87] D.R. COK, « Signal processing method and apparatus for producing interpolated chrominance values in a sampled color image signal », US Patent 4,642,678, to Eastman Kodak Compagny, Patent and Trademark Office, Washington, D.C., 1987.

[Cok91-1] D.R. COK, « Mechanism for producing interpolated color values for a sampled color image signal using a linear equation technique », US Patent 5,032,910, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1991.

[Cok91-2] D.R. COK, « Method of processing sampled signal valves produced by a color imaging device, » US patent 5,040,064, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1991.

[Cok94] D.R. COK, « Reconstruction of CCD Images Using Template Matching », Proc. IS&T's 47th Annual Conference and Int'l Congress Photo. Sciences, (ICPS'94), 1994, pp. 380-385.

[Crane99] H.D. CRANE, J.D. PETER, and E. MARTINEZ-URIEGAS, « Method and Apparatus for Decoding Spatiochromatically Multiplexed Color Images Using Predetermined Coefficients », US patent 5,901,242, to SRI International, Patent and Trademark Office, Washington, D.C., 1999.

[Dillon77] P. DILLON, « Color Imaging Array », US Patent 4,047,203, to Eastman Kodak Compagny, Patent and Trademark Office, Washington, D.C., 1977.

[Dillon79-1] P. DILLON, and B. BAYER, « Signal processing for DiscreteSample-Type-Color-Video Signal », US Patent 4,065,785, to Eastman Kodak Compagny, Patent and Trademark Office, Washington, D.C., 1979.

[Dillon79-2] P. DILLON, and B. BAYER, « Signal processing for DiscreteSample-Type-Color-Video Signal », US Patent 4,176,373, to Eastman Kodak Compagny, Patent and Trademark Office, Washington, D.C., 1979.

[Evans49] EVANS, « The birefringent filter », Journal of Optical Society of America, Vol. 39, 3, pp 229-242, 1949.

[Freeman88-1] W.T. FREEMAN, « Median filter for reconstructing missing color samples », US Patent 4,724,395, to Polaroid Corporation, Patent and Trademark Office, Washington, D.C., 1988.

[Freeman88-2] W.T. FREEMAN, « Method and apparatus for reconstructing missing color samples », US Patent 4,774,565, to Polaroid Corporation, Patent and Trademark Office, Washington, D.C., 1988.

[Goltzbach01] J.W. GOLTZBACH, R.W. SCHAFER, and K. ILLGNER, « A method of color filter array interpolation with alias cancellation properties », Proc. Int'l Conf. on Image Processing, (ICIP'01), Thessaloniki, Greece, Oct. 2001, vol. 1, pp. 141-144.

[Go00] J. GO, K. SOHN, and C. LEE, « Interpolation using neural networks for digital still cameras », IEEE Trans. on Consumer Electronics, Vol. 46, no 3, 2000, pp. 610-616.

[Greivenkamp86] J. GREIVENKAMP, « Optical spatial frequency filter », US Patent 4,575,193, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1986.

[Greivenkamp90] J. GREIVENKAMP, « Color dependant optical prefilter for the suppression of aliasing artifacts », Applied Optics, vol. 29, no. 5, 1990, pp. 676-684.

[Gunturk02] B.K. GUNTURK, Y. ALTUNBASAK, and R.M. MERSEREAU, « Color plane interpolation using alternating projections », IEEE Trans. on Image Processing, Vol. 11, no. 9, 2002, pp. 997-1013.

[Gunturk04] B.K. GUNTURK, J.GOLTZBACH, Y. ALTUNBASAK, R.W. SCHAFER, and R.M. MERSEREAU, « Demosaicking: Color filter array interpolation in single-chip digital cameras », IEEE Signal Processing Magazine, Special Issue on Color Image Processing, to appears in 2004.

[Gupta01] M. GUPTA and T. CHEN, « Vector Color Filter Array Demosaicing », Proc. of the SPIE, Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications II, Vol. 4306, San Jose, ed, CA, M. Blouke, J. Canosa, N. Sampat, 2001, pp. 374-382.

[Hamilton97] J.F. HAMILTON Jr, and J.E. ADAMS Jr, « Adaptive color plan interpolation in single sensor color electronic camera », US Patent 5,629,734, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1997.

[Hamilton00] J.F. HAMILTON Jr. and J.E. ADAMS Jr., « Computing Color Specification (Luminance and Chrominance) Values for Images », US patent 6,075,889, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 2000.

[Herault96] J. HERAULT , « A model of colour processing in the retina of vertebrates : from photoreceptors to colour opposition and colour constancy », Neurocomputing, (1996).

[Hibbard95] R.H HIBBARD, « Apparatus and method for adaptively interpolating a full color image utilizing luminance gradients », US patent 5,382,976, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1995.

[Ingling83] C.R. INGLING Jr. and E. MARTINEZ, « The spatiochromatic signal of the r-g channel », in Colour Vision, J.A. van Esch et al., ed., Academic Press London, 1983.

[Keren99] D. KEREN and M. OSADCHY, « Restoring subsampled color images », Machine Vision and Applications, ed. Springer-Verlag, Vol. 11, 1999, pp. 197-202.

[Kimmel99] R. KIMMEL, « Demosaicing: Image Reconstruction from Color Samples », IEEE Trans. Image Processing, Vol. 8, no. 9, 1999, pp. 1221-1228.

[Kuno99] T. KUNO, H. SUGIURA and N. MATOBA, « New interpolation method using discriminated color correlation for digital still cameras », IEEE Transaction on Consumer Electronics, Vol. 45, no. 1, 1999, pp 259-267.

[Laroche94] C.A. LAROCHE and M. A. PRESCOTT, « Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients », US patent 5,373,322, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1994.

[Lee01] B. LEE, K. SOHN, and C. LEE, « Performance comparison of various interpolation methods for color filter arrays », Proc. Int'l Symposium on Industrial Electronics, (ISIE'01), Pusan, South Korea, 2001, Vol. 1, pp. 232-236.

[Li01] X. LI and M. ORCHARD, « New edge-directed interpolation », IEEE Trans. on Image Processing, Vol. 10, no. 10, 2001, pp. 1521-1527.

[Lu01] W. LU and Y.P. TAN, « Layering-based color filter array interpolation », Proc. Int'l Conf. on Image Processing, Thessaloniki, Greece, Vol.3, 2001, pp. 860-863.

[Lu03] W. LU and Y.P. TAN, « Color filter array demosaicing: New method and performance measures », IEEE Trans. Image Processing, Vol. 12, no. 10, 2003, pp. 1194-1210.

[Longere02] P. LONGERE, X. ZHANG, P.B. DELAHUNT, D.H. BRAINARD, « Perceptual assessment of demosaicing algorithm performance », Proc. of the IEEE, Vol. 90, no 1, January 2002, pp. 123-132.

[Macovski68] A. MACOVSKI, « Monochrome photography system for color television », US Patent 3,378,633, to Stanford Research Inst., Patent and Trademark Office, Washington, D.C., 1968.

[Martinez95] E. MARTINEZ-URIEGAS, H.D. CRANE and J.D. PETERS, « Method and apparatus for compression and decompression of digital color images » US Patent 5,398,066, to SRI International, Patent and Trademark Office, Washington, D.C., 1995.

[Mukherjee01] J. MUKHERJEE, M.S. MOORE and S.K. MITRA, « Color demosaicing with constrained buffering », Proc. Int. Symp. on Signal Processing and its Application, ISSPA, Kuala Lumpur, Malaysia, 2001, pp. 52-55.

[Muresan02] D. MURESAN and T. PARKS, « Optimal recovery demosaicing », Proc. International Association of Science and Technology for Development (IASTED) Signal and Image Processing, Hawaii, 2002.

[Peters96] J.D. PETERS, H.D. CRANE and E. MARTINEZ-URIEGAS, « Method and apparatus for increasing resolution of digital color images using correlated decoding », US patent 5,541,653, to SRI International, Patent and Trademark Office, Washington, D.C., 1996.

[Pritchard71] D.H. PRITCHARD, « Adjustable bandwidth optical filter », US Patent 3,588,224, to RCA Corporation, Patent and Trademark Office, Washington, D.C., 1971.

[Ramanath02] R. RAMANATH and W.E. SNYDER and G.L. BILBRO, « Demosaicking Methods for Bayer Color Arrays », J. of Electronic Imaging, Vol. 11, no. 3, 2002, pp. 303-315.

[Ramanath03] R. RAMANATH and W.E. SNYDER, « Adaptive Demosaicking », J. of Electronic Imaging, Vol. 12, no. 4, 2003, pp. 633-642.

[Seve96] R. SÈVE, « Physique de la couleur: De l'apparence colorée à la technique colorimétrique », Masson, 1996

[Taubman00] D. TAUBMAN, « Generalized Weiner reconstruction of images from colour sensor data using a scale invariant prior », Proc. Int. Conf. Image Proc., (ICIP'00), Vancouver, BC, Canada, vol.3, 10-13 Sept. 2000, pp. 801-804.

[Tao99] B. TAO, I. TASTL, T. COOPER, M. BLASGEN, and E. EDWARDS, « Demosaicing using human visual properties and wavelet interpolation filtering », Proc. IS&T/SID Seventh Color Imaging Conference, (CIC'99), Scottsdale, Arizona, 1999, pp.252-256.

[Topfer98] K. TOPFER, J.E. ADAMS, and B.W. KEELAN, « Modulation transfer functions and aliasing patterns of CFA interpolation algorithms », Proc. IS&T, PICS, Portland Oregon, 1998, pp. 367-370.

[Trussell01] H. TRUSSELL, « A MMSE estimate for demosaicking », Proc. IEEE Int. Conf. on Image Proc., 7-11 Oct. 2001, Thessaloniki, Greece.

[Trussell02] H. TRUSSELL and R.E. HARTWIG, « Mathematics for demosaicking », IEEE Transanctions on Image Processing, vol. 11, no 4, April 2002, pp. 495-492.

[Weerasinghe02] C. WEERASINGHE, I. KHARITONENKO, and P. OGUNBONA, « Method of color interpolation in a single sensor color camera using green channel separation », Proc. Int'l Conf. on Acoustics, Speech, and Signal Processing, (ICASSP'02), 2002, Vol. 4, pp. 3233-3236.

[Weldy87] J.A. WELDY and S.H. KRISTY, « Single sensor color video camera with blurring filter », US Patent 4,663,661, to Eastman Kodak Company, Patent and Trademark Office, Washington, D.C., 1987.

[Weldy89] J.A. WELDY, « Optimized design for a single-sensor color electronic camera system », SPIE Optical sensors and Electronic Photography, vol. 1071, 1989, pp. 300-308.

[Web] Des figures supplémentaires sont disponibles à http://ivrgwww.epfl.ch/index.php?name=IEEE_Demosaicing

Please sign up to receive notifications on new issues and newsletters from IIETA

Select Journal/Journals:

 

Copyright © 2024 IIETA. All Rights Reserved.