MHI-CNN Model for Fine-grained Classification of Product Image

MHI-CNN Model for Fine-grained Classification of Product Image

Mingxia LinCuihua Li 

School of Information Science and Technology, Xiamen University, Xiamen, P.R.China

School of Business and management, JiMei University, Xiamen, P.R.China

15 March 2017
15 April 2017
31 March 2017
| Citation



Fine-grained classification (FGC) is a current focus of research, but the problem in the application of product image especially in women's clothing is a great challenge. Because women clothing contain more style, color and details than other clothes, in addition constructing a proper training set for these is very difficult. We propose a new model named MHI-CNN which is multi-scale and heterogeneous integration based on the traditional Convolutional Neural Network (CNN) model. We use multi-scale to get more detail feature and heterogeneous integrate three models to achieve higher accuracy. The experiment results show our model improved the performance of  women clothing image classification.


fine-grained classification (FGC), convolutional neural network CNN), deep learning

1. Introduction
2. Related Work
3. Framework of MHI-CNN Model
4. Experiments
5. Conclusion

1. P.H. Gosselin, N. Murray, H. Jégou, F. Perronnin, Revisiting the Fisher vector for fine-grained classification, 2014, Pattern Recognition Letters, vol. 49, pp. 92-98.

2. M.E. Nilsback, A. Zisserman, Automated flower classification over a large number of classes, 2008, Computer Vision, Graphics & Image Processing, ICVGIP'08. Sixth Indian Conference on. IEEE.

3. C. Wah, S. Branson, P. Welinder, P. Perona, S. Belongie, The caltech-ucsd birds-200-2011 dataset, 2011. 

4. B. Schiele, M. Andriluka, S. Amin, M. Rohrbach, A database for fine grained activity detection of cooking activities, 2012, Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE.

5. O.M. Parkhi, A. Vedaldi, A. Zisserman, C. Jawahar, Cats and dogs, 2012, Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE.

6. A.S. Razavian, H. Azizpour, J. Sullivan, S. Carlsson, CNN features off-the-shelf: an astounding baseline for recognition, 2014, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops.

7. B. Fernando, E. Fromont, T. Tuytelaars, Effective use of frequent itemset mining for image classification, 2012, European conference on computer vision.

8. Q. Chen, Z. Song, Y. Hua, Z. Huang, S. Yan, Hierarchical matching with side information for image classification, 2012, Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE.

9. J. Nchez, F. Perronnin, T. Mensink, J. Verbeek, Image classification with the fisher vector: Theory and practice, 2013, International journal of computer vision, vol. 105, no. 3, pp. 222-245.

10. X.T. Yuan, X.B. Liu, S.C. Yan, Visual classification with multitask joint sparse representation, 2012, IEEE Transactions on Image Processing, vol. 21, no. 10, pp. 4349-4360.

11. M. Varma, R. Debajyoti, Learning the discriminative power-invariance trade-off, 2007, Computer Vision, ICCV 2007, IEEE 11th International Conference on. IEEE.

12. P. Gehler, N. Sebastian, On feature combination for multiclass object classification, 2009, Computer Vision, 2009 IEEE 12th International Conference on. IEEE.

13. A. Iscen, G. Tolias, P.H. Gosselin, H. Jegou, A comparison of dense region detectors for image search and fine-grained classification, 2015, IEEE Transactions on Image Processing, vol. 24, no. 8, pp. 2369-2381.

14. M.D. Zeiler, R. Fergus, Visualizing and understanding convolutional networks, 2014, European conference on computer vision.

15. A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep convolutional neural networks, 2012, Advances in neural information processing systems.

16. K. Chatfield, K. Simonyan, A. Vedaldi, A. Zisserman, Return of the devil in the details: Delving deep into convolutional nets, 2014, arXiv: 14053531.

17. D. Ciregan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image classification, 2012, Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE. 

18. J.C. Chen, C.F. Liu, Visual-based deep learning for clothing from large database, 2015, Proceedings of the ASE BigData & SocialInformatics 2015.

19. K. Lin, H.F. Yang, K.H. Liu, J.H. Hsiao, C.S. Chen, Rapid clothing retrieval via deep learning of binary codes and hierarchical search, 2015, Proceedings of the 5th ACM on International Conference on Multimedia Retrieval.

20. N. Khosla, V. Venkataraman, Building Image-Based Shoe Search Using Convolutional Neural Networks, 2015, CS231n Course Project Reports.

21. A. Borràs, F. Tous, J. Lladós, M. Vanrell, High-level clothes description based on colour-texture and structural features, 2013, Iberian Conference on Pattern Recognition and Image Analysis.

22. Q. Chen, J. Huang, R. Feris, L.M. Brown, J. Dong, S. Yan, Deep domain adaptation for describing people based on fine-grained clothing attributes, 2015, Proceedings of the IEEE conference on computer vision and pattern recognition.

23. S. Lazebnik, C. Schmid, J. Ponce, Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, 2016, CVPR 2006

24. C. Rother, V. Kolmogorov, A. Blake, Grabcut: Interactive foreground extraction using iterated graph cuts, 2004, ACM transactions on graphics (TOG), vol. 23, no. 3.

25. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun, Overfeat: Integrated recognition, localization and detection using convolutional networks, 2013, arXiv:13126229.

26. T. Zahavy, A. Magnani, A. Krishnan, S. Mannor, Is a picture worth a thousand words? A Deep Multi-Modal Fusion Architecture for Product Classification in e-commerce, 2016, arXiv:1611.09534.

27. S.C. Zhao, Y.J. Xu, Y.H. Han, 2016, Large-Scale E-Commerce Image Retrieval with Top-Weighted Convolutional Neural Networks, 2016, Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval.