Bin-Bin Jia, Min-Ling Zhang. Multi-Dimensional Classification via Selective Feature Augmentation. International Journal of Automation and Computing.
Citation: Bin-Bin Jia, Min-Ling Zhang. Multi-Dimensional Classification via Selective Feature Augmentation. International Journal of Automation and Computing.

Multi-Dimensional Classification via Selective Feature Augmentation

More Information
  • Author Bio:

    Bin-Bin Jia received the B. Sc. degree in electronic information science and technology from North China Electric Power University, China in 2010, and the M. Sc. degree in information and communication engineering from Beihang University, China in 2013. He joined the College of Electrical and Information Engineering, Lanzhou University of Technology in 2013. Currently, he is a Ph. D. candidate in the School of Computer Science and Engineering at Southeast University. His main research interests include machine learning and data mining, especially in multi-dimensional classification. E-mail: jiabb@seu.edu.cn ORCID: https://orcid.org/0000-0003-3302-9398

    Min-Ling Zhang received the B. Sc., M. Sc., and Ph. D. degrees in computer science from Nanjing University, China in 2001, 2004 and 2007, respectively. Currently, he is a Professor at the School of Computer Science and Engineering, Southeast University, China. His main research interests include machine learning and data mining. In recent years, Dr. Zhang has served as the General Co-Chairs of ACML′18, Program Co-Chairs of PAKDD′19, CCFICAI′19, ACML′17, CCFAI′17, PRICAI′16, Senior PC member or Area Chair of AAAI 2017−2020, IJCAI 2017-2020, KDD 2021, ICDM 2015-2020, etc. He is also on the editorial board of ACM Transactions on Intelligent Systems and Technology, Neural Networks, Science China Information Sciences, Frontiers of Computer Science, etc. Dr. Zhang is the Steering Committee Member of ACML and PAKDD, secretary general of the CAAI Machine Learning Society, standing committee member of the CCF Artificial Intelligence & Pattern Recognition Society. He is a Distinguished Member of CCF, CAAI, and Senior Member of ACM, IEEE. E-mail: zhangml@seu.edu.cn (Corresponding author) ORCID: https://orcid.org/0000-0003-1880-5918

  • Received Date: 2021-04-17
  • Accepted Date: 2021-09-08
  • In multi-dimensional classification (MDC), the semantics of objects are characterized by multiple class spaces from different dimensions. Most MDC approaches try to explicitly model the dependencies among class spaces in output space. In contrast, the recently proposed feature augmentation strategy, which aims at manipulating feature space, has also been shown to be an effective solution for MDC. However, existing feature augmentation approaches only focus on designing holistic augmented features to be appended with the original features, while better generalization performance could be achieved by exploiting multiple kinds of augmented features. In this paper, we propose the selective feature augmentation strategy that focuses on synergizing multiple kinds of augmented features. Specifically, by assuming that only part of the augmented features is pertinent and useful for each dimension′s model induction, we derive a classification model which can fully utilize the original features while conduct feature selection for the augmented features. To validate the effectiveness of the proposed strategy, we generate three kinds of simple augmented features based on standard kNN, weighted kNN, and maximum margin techniques, respectively. Comparative studies show that the proposed strategy achieves superior performance against both state-of-the-art MDC approaches and its degenerated versions with either kind of augmented features.

     

  • loading
  • [1]
    D. Xu, Y. X. Shi, I. W. Tsang, Y. S. Ong, C. Gong, X. B. Shen. Survey on multi-output learning. IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 7, pp. 2409–2429, 2020. DOI: 10.1109/TNNLS.2019.2945133.
    [2]
    J. Read, C. Bielza, P. Larra\~{n}aga. Multi-dimensional classification with super-classes. IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 7, pp. 1720–1733, 2014. DOI: 10.1109/TKDE.2013.167.
    [3]
    B. B. Jia, M. L. Zhang. Maximum margin multi-dimensional classification. IEEE Transactions on Neural Networks and Learning Systems, 2021. DOI: 10.1109/TNNLS.2021.3084373.
    [4]
    J. D. Rodr\’{i}guez, A. P\’{e}rez, D. Arteta, D. Tejedor, J. A. Lozano. Using multidimensional Bayesian network classifiers to assist the treatment of multiple sclerosis. IEEE Transactions on Systems,Man,and Cybernetics,Part C (Applications and Reviews), vol. 42, no. 6, pp. 1705–1715, 2012. DOI: 10.1109/TSMCC.2012.2217326.
    [5]
    H. Borchani, C. Bielza, C. Toro, P. Larra\~{n}aga. Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers. Artificial Intelligence in Medicine, vol. 57, no. 3, pp. 219–229, 2013. DOI: 10.1016/j.artmed.2012.12.005.
    [6]
    H. Shatkay, F. X. Pan, A. Rzhetsky, W. J. Wilbur. Multi-dimensional classification of biomedical text: Toward automated, practical provision of high-utility text to diverse users. Bioinformatics, vol. 24, no. 18, pp. 2086–2093, 2008. DOI: 10.1093/bioinformatics/btn381.
    [7]
    F. Serafino, G. Pio, M. Ceci, D. Malerba. Hierarchical multidimensional classification of web documents with multiwebclass. In Proceedings of the 18th International Conference on Discovery Science, Springer, Banff, Canada, pp. 236-250, 2015. DOI: 10.1007/978-3-319-24282-8_20.
    [8]
    Z. Lian, Y. Li, J. H. Tao, J. Huang, M. Y. Niu. Expression analysis based on face regions in real-world conditions. International Journal of Automation and Computing, vol. 17, no. 1, pp. 96–107, 2020. DOI: 10.1007/s11633-019-1176-9.
    [9]
    Z. W. He, L. Zhang, F. Y. Liu. DiscoStyle: Multi-level logistic ranking for personalized image style preference inference. International Journal of Automation and Computing, vol. 17, no. 5, pp. 637–651, 2020. DOI: 10.1007/s11633-020-1244-1.
    [10]
    Y. Zhang, X. Y. Shi, S. Y. Mi, X. Yang. Image captioning with transformer and knowledge graph. Pattern Recognition Letters, vol. 143, pp. 43–49, 2021. DOI: 10.1016/j.patrec.2020.12.020.
    [11]
    A. H. Al Muktadir, T. Miyazawa, P. Martinez-Julia, H. Harai, V. P. Kafle. Multi-target classification based automatic virtual resource allocation scheme. IEICE Transactions on Information and Systems, vol. E102-D, no. 5, pp. 898–909, 2019. DOI: 10.1587/transinf.2018NTP0016.
    [12]
    J. Arias, J. A. Gamez, T. D. Nielsen, J. M. Puerta. A scalable pairwise class interaction framework for multidimensional classification. International Journal of Approximate Reasoning, vol. 68, pp. 194–210, 2016. DOI: 10.1016/j.ijar.2015.07.007.
    [13]
    B. B. Jia, M. L. Zhang. Multi-dimensional classification via stacked dependency exploitation. Science China Information Sciences, vol. 63, no. 12, Article number 222102, 2020. DOI: 10.1007/s11432-019-2905-3.
    [14]
    B. B. Jia, M. L. Zhang. MD-KNN: An instance-based approach for multi-dimensional classification. In Proceedings of the 25th International Conference on Pattern Recognition, IEEE, Milan, Italy, pp. 126-133, 2021.
    [15]
    J. H. Zaragoza, L. E. Sucar, E. F. Morales, C. Bielza, P. Larranaga. Bayesian chain classifiers for multidimensional classification. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, IJCAI, Barcelona, Spain, pp. 2192-2197, 2011.
    [16]
    J. Read, L. Martino, D. Luengo. Efficient Monte Carlo methods for multi-dimensional learning with classifier chains. Pattern Recognition, vol. 47, no. 3, pp. 1535–1546, 2014. DOI: 10.1016/j.patcog.2013.10.006.
    [17]
    C. Bielza, G. Li, P. Larra\~{n}aga. Multi-dimensional classification with Bayesian networks. International Journal of Approximate Reasoning, vol. 52, no. 6, pp. 705–727, 2011. DOI: 10.1016/j.ijar.2011.01.007.
    [18]
    J. H. Bolt, L. C. Van Der Gaag. Balanced sensitivity functions for tuning multi-dimensional Bayesian network classifiers. International Journal of Approximate Reasoning, vol. 80, pp. 361–376, 2017. DOI: 10.1016/j.ijar.2016.07.011.
    [19]
    M. Benjumeda, C. Bielza, P. Larra\~{n}aga. Tractability of most probable explanations in multidimensional Bayesian network classifiers. International Journal of Approximate Reasoning, vol. 93, pp. 74–87, 2018. DOI: 10.1016/j.ijar.2017.10.024.
    [20]
    B. B. Jia, M. L. Zhang. Multi-dimensional classification via kNN feature augmentation. Pattern Recognition, vol. 106, Article number 107423, 2020. DOI: 10.1016/j.patcog.2020.107423.
    [21]
    H. B. Wang, C. Chen, W. W. Liu, K. Chen, T. L. Hu, G. Chen. Incorporating label embedding and feature augmentation for multi-dimensional classification. In Proceedings of the 34th AAAI Conference on Artificial Intelligence, AIAA, New York, USA, pp. 6178-6185, 2020. DOI: 10.1609/AAAI.V34I04.6083.
    [22]
    M. L. Zhang, Z. H. Zhou. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, vol. 26, no. 8, pp. 1819–1837, 2014. DOI: 10.1109/TKDE.2013.39.
    [23]
    E. Gibaja, S. Ventura. A tutorial on multilabel learning. ACM Computing Surveys, vol. 47, no. 3, Article number 52, 2015. DOI: 10.1145/2716262.
    [24]
    M. L. Zhang, Y. K. Li, X. Y. Liu, X. Geng. Binary relevance for multi-label learning: An overview. Frontiers of Computer Science, vol. 12, no. 2, pp. 191–202, 2018. DOI: 10.1007/s11704-017-7031-7.
    [25]
    S. Gil-Begue, C. Bielza, P. Larra\~{n}aga. Multi-dimensional Bayesian network classifiers: A survey. Artificial Intelligence Review, vol. 54, no. 1, pp. 519–559, 2021. DOI: 10.1007/s10462-020-09858-x.
    [26]
    J. Huang, G. R. Li, Q. M. Huang, X. D. Wu. Learning label-specific features and class-dependent labels for multi-label classification. IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 12, pp. 3309–3323, 2016. DOI: 10.1109/TKDE.2016.2608339.
    [27]
    H. H. Bauschke, J. Bolte, M. Teboulle. A descent lemma beyond Lipschitz gradient continuity: First-order methods revisited and applications. Mathematics of Operations Research, vol. 42, no. 2, pp. 330–348, 2017. DOI: 10.1287/moor.2016.0817.
    [28]
    A. Beck, M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183–202, 2009. DOI: 10.1137/080716542.
    [29]
    Z. C. Ma, S. C. Chen. Multi-dimensional classification via a metric approach. Neurocomputing, vol. 275, pp. 1121–1131, 2018. DOI: 10.1016/j.neucom.2017.09.057.
    [30]
    R. E. Fan, K. W. Chang, C. J. Hsieh, X. R. Wang, C. J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, vol. 9, pp. 1871–1874, 2008.
    [31]
    K. Crammer, Y. Singer. On the algorithmic implementation of multiclass kernel-based vector machines. The Journal of Machine Learning Research, vol. 2, pp. 265–292, 2001.
    [32]
    J. Dem\v{s}ar. Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research, vol. 7, pp. 1–30, 2006.
    [33]
    J. Zhao, X. J. Xie, X. Xu, S. L. Sun. Multi-view learning overview: Recent progress and new challenges. Information Fusion, vol. 38, pp. 43–54, 2017. DOI: 10.1016/j.inffus.2017.02.007.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(4)  / Tables(5)

    Article Metrics

    Article views (80) PDF downloads(7) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return