Hybrid Decision Systems and Incremental Learning
Systèmes de Décision Hybrides et Apprentissage Incrémental en Données
OPEN ACCESS
This paper presents a multi-classifier system design controlled by the topology of the learning data.Our work also introduces a training algorithm for an incremental self-organizing map (SOM).This SOM is used to distribute classification tasks to a set of classifiers.Thus,the useful classifiers are activated when new data arrives. Comparative results are given for synthetic problems,for an image segmentation problem from the UCI repository and for a handwritten digit recognition problem.
Résumé
Ce papier présente un système de décision multi-classifieurs dont la conception est pilotée par la topologie des données d'apprentissage. Celle-ci est extraite grâce à l'introduction d'un nouvel algorithme d'apprentissage de carte neuronale auto-organisée qui a la propriété d'être incrémentale en données. Cette carte est utilisée en apprentissage pour distribuer la tâche de classification sur un ensemble de classifieurs. Elle permet ensuite d'activer en phase de décision le ou les classifieurs utiles pour une nouvelle donnée. De plus,le système proposé introduit un critère de confiance s'affranchissant totalement du type de classifieurs utilisés. Ce coefficient permet de contrôler plus efficacement le compromis Erreur/Rejet. Des résultats comparatifs sont donnés sur des exemples synthétiques,sur la base de segmentation d'images de l'UCI et sur le problème de reconnaissance de chiffres manuscrits sur des données de la base NIST.
Incremental clustering,distributed learning,self-organizing maps,pattern recognition.
Mots clés
Regroupement incremental,apprentissage distribué,carte auto-organisatrice,reconnaissance de formes.
[AOC+00] S. ADAM, J.M. OGIER, C. CARRIOU, R. MULLOT, J. LABICHE, J. GARDES, Symbol and character recognition: application to engineering drawings. International Journal on Document Analysis and Recognition,pages 89-101, 2000.
[BB82] D.H. BALLARD, C.M. BROWN, Computer Vision. Prentice Hall, 1982. [DUI02] R.P.W. DUIN, The combining classifier: to train or not to train. In IEEE Computer Society, editor, ICPR,pages 765-771, Quebec, 2002.
[FRI94] B. FRITZKE, Growing cell structures – A self-organizing network for unsupervised and supervised learning. Neural Networks, 7(9):1441-1460, 1994.
[FRI95a] B. FRITZKE, Growing grid – a sel organizing network with constant neighborhood range and adaptation strengh. Neural Processing Letter, (2):9-13, 1995.
[FRI95b] B. FRITZKE,A Growing neural gas network learns topologies. In Advances in Neural Information Processing Systems 7,pages 625632, 1995.
[GA95] M. GUILLOT, R. AZOUZI, Improving on-line adaptation in neurocontrol using a combination of self-organizing map and multilayer feedforward network. Intelligence Engeneering Systems Throu Artificial Neural Networks, pages 915-922, 1995.
[GR00] G. GIACINTO, F. ROLL, Dynamic classifier selection. In Springer, editor, Proc. of the First Int. Workshop on Multiple Classifier Systems, pages 177-189, jun 200.
[GS98] M. GORI, F. SCARSELLI, A Multilayer perceptron adequate for pattern recognition and verification. IEE Transactions on pattern analysis and machine intelligence, 20:1121-1132, 1998.
[HAY95] S. HAYKIN, Neural Networks – A Comprehensive Fundations. IEE Press, 1995.
[HPG99] J.F. HEBERT, M. PARIZEAU, N. GHAZZALI, An hybrid architecture for active and incremental learning: The self-organizing perceptron (sop) network. In IJCNN99,Washington, DC, USA, July 1999.
[JJNH91] R.A. JACOBS, M.J. JORDAN, S.J. NOWLAN and G.E. HINTON, Adaptative mixtures of local experts. Neural Computation, 3(1):79-87, 1991.
[KOH82] T. KOHONEN, Sel-organized formation of topologically correct feature maps. Biological Cybernetics,43:59-69, 1982.
[KUN02] L.I. KUNCHEVA,A theoretical study on six classifier fusion strategies. In IEEE Trans. Pattern Analysis and Machine Intelligence, volume Pattern Recognition, Feb 2002.
[LESG95] Y. LECOURTIER, A. ENNAJI, E. STOCKER, F. GILLES, Réseaux d'yprels et apprentissage incrémental. Traitement du Signal, 12(6):597-608, 1995.
[LW03] K.W. LAU, Q.H. WU, Online training of support classifier. Pattern Recognition, 36:1913-1920, 2003.
[MA92] P.M. MURPHY, D.W. AHA, UCI repository of machine learning databases. Machines-readable data repository, University of California, Department of Information and Computer Science, Irvine, CA, 1992.
[MAC67] J.B. MACQUEEN, Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Symposium on Mathematics Statistics and Probability,pages 281-297, 1967.
[MAR93] T. MARTINEZ, Competitive Hebbian learning rule forms perfectly topology perservings maps. In Proc. ICANN'93,pages 427434, Springer, 1993.
[MS91] T. MARTINEZ, K. SCHULTEN, A "Meural-Gas" network learns topologies. In Proc. ICANN, volume I, pages 397-402, Amsterdam, Netherlands, 1991.
[MS94] T. MARTINEZ, K. SCHULTEN, Topology representing networks. Neural Networks, 7(2), 1994.
[PE04] Y. PRUDENT, A. ENNAJI, Clustering incrémental pour un apprentissage distribué. In Conférence CAp 2004, 2004.
[RG02] F.ROLL, G. GIANCINTO, Design of multiple classifier systems. Hybrid Methods in Pattern Recognition,pages 199-226, April 2002.
[RIB98] A. RIBERT, Structuration Evolution de données: Application à la construction de classifieurs distribués. PhD thesis, Université de Rouen, 1998.
[TFCJW03] W. TING-FAN, L. CHIH-JEN, RUBY C.W., Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research,5 ,2004.
[VAP95] V.N. VAPNIK, The nature of statistical theory. Springer, 1995.