﻿
 doi:

DOI: 10.3724/SP.J.1016.2008.00476

Chinese Journal of Computers (计算机学报) 2008/31:3 PP.476-485

## The Key Theorem and the Bounds on the Rate of Uniform Convergence of Statistical Learning Theory on Quasi-Probability Spaces

• HA Ming-Hu 1   FENG Zhi-Fang 2   SONG Shi-Ji 3   GAO Lin-Qing 1
• 1.College of Mathematics and Computer Sciences, Hebei University,Baoding, Hebei,071002,China
• 2.Department of Mathematics and Information Sciences, Langfang Normal School,Langfang, Hebei,065000,China
• 3.Department of Automation, Tsinghua University,Beijing,100084,China

Abstract：
Some properties of quasi-probability are further discussed. The definitions and properties of quasi-random variable and its distribution function, expected value and variance are then presented. Markov inequality, Chebyshev's inequality and the Khinchine's law of large numbers on quasi-probability spaces are also proved. Then the key theorem of learning theory on quasi-probability spaces is proved, and the bounds on the rate of uniform convergence of learning process on quasi-probability spaces are constructed. The investigations will help lay essential theoretical foundations for the systematic and comprehensive development of the quasi-statistical learning theory.

Key words：quasi-probability,empirical risk functional,expected risk functional,key theorem,bounds on the rate of uniform convergence

ReleaseDate：2014-07-21 14:43:30

[1] Vapnik V N, Xu Jian-Hua, Zhang Xue-Gong translate. Statistical Learning Theory. Beijing: Publishing House of Electronics Industry, 2004(in Chinese) (Vapnik V N著. 许建华,张学工译. 统计学习理论. 北京:电子工业出版社, 2004)

[2] Vapnik V N, Zhang Xue-Gong translate. The Nature of Statistical Learning Theory. Beijing: Tsinghua University Press, 2000(in Chinese) (Vapnik V N著.张学工译. 统计学习理论的本质. 北京:清华大学出版社, 2000)

[3] Vapnik V N. An overview of statistical learning theory. IEEE Transactions on Neural Networks, 1999, 10(5): 988-999

[4] Bian Zhao-Qi, Zhang Xue-Gong. Pattern Recognition. Beijing: Tsinghua University Press, 1999(in Chinese) (边肇祺,张学工.模式识别.北京:清华大学出版社, 1999)

[5] Zhang Xue-Gong. Introduction to statistical learning theory and support vector machines. Acta Automatica Sinica, 2000, 26(1): 32-42(in Chinese) (张学工.关于统计学习理论与支持向量机. 自动化学报, 2000, 26(1): 32-42)

[6] Liu Hui-Chun, Ma Shu-Yuan. Introduction to support vector machines. Journal of Image and Graphics, 2002, 7(6): 618-623(in Chinese) (柳回春, 马树元. 支持向量机的研究现状. 中国图象图形学报, 2002, 7(6): 618-623)

[7] Ha Ming-Hu, Li Yan, Li Jia, Tian Da-Zeng. The key theorem and the bounds on the rate of uniform convergence of learning theory on Sugeno measure spaces. Science in China(Series E), Information Sciences, 2006, 36(4): 398-410(in Chinese) (哈明虎,李颜,李嘉,田大增.Sugeno测度空间上学习理论的关键定理和一致收敛速度的界. 中国科学(E辑):信息科学, 2006, 36(4): 398-410)

[8] Evgeniou T, Poggio T, Pontil M, Verri A. Regularization and statistical learning theory for data analysis. Computational Statistics & Data Analysis, 2002, 38: 421-432

[9] Wechsler H, Duric Z, Li Fa-Yin, V. Cherkassky. Motion estimation using statistical learning theory. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(4): 466-478

[10] Raudys S. How good are support vector machines? Neural Networks, 2000, 13(1): 17-19

[11] Tay Francis E H, Cao Lijuan. Application of support vector machines in financial time series forecasting. Omega, 2001, 29(4): 309-317

[12] Tsai C F. Training support vector machines based on stacked generalization for image classification. Neurocomputing, 2005, 64: 497-503

[13] Kim Jong Kyoung, Raghava G P S, Bang Sung Yang, Choi Seungjin. Prediction of subcellular localization of proteins using pairwise sequence alignment and support vector machine. Pattern Recognition Letters, 2006, 27(9): 996-1001

[14] Zhan Yi-Qing, Shen Ding-Gang. Design efficient support vector machine for fast classification. Pattern Recognition, 2005, 38(1): 157-161

[15] Jeng Jin-Tsong. Hybrid approach of selecting hyper-parameters of support vector machine for regression. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2006, 36(3): 699-709

[16] Jin Bo, Tang Y C, Zhang Yan-Qing. Support vector machines with genetic fuzzy feature transformation for biomedical data classification. Information Sciences, 2007, 177: 476-489

[17] Castro J L, Flores-Hidalgo L D, Mantas C J, Puche J M. Extraction of fuzzy rules from support vector machines. Fuzzy Sets and Systems, 2007, 158: 2057-2077

[18] Choquet G. Theory of capacities. Annales de l'Institut Fourier, 1954, 5: 131-295

[19] Sugeno M. Theory of fuzzy integrals and its applications[Ph.D. dissertation]. Tokyo Institute of Technology, Tokyo, 1974

[20] Wang Zhen-Yuan, George J K. Fuzzy Measure Theory. New York: Plenum Press, 1992

[21] Zadeh L A. Fuzzy sets as a basis for a theory of possibility. Fuzzy Sets and Systems, 1978, 1: 3-28

[22] Wang Zhen-Yuan. Une classe de measures floues—les quasi-mesures. Busefal, 1981, 6: 28-37

[23] Liu Bao-Ding. Theory and Practice of Uncertain Programming. Heidelberg: Physica-Verlag, 2003

[24] Ha Ming-Hu, Wu Cong-Xin. Fuzzy Measure and Fuzzy Integral Theory. Beijing: Science Press, 1998(in Chinese) (哈明虎,吴从炘. 模糊测度与模糊积分理论. 北京:科学出版社, 1998)

[25] Grabisch M, Murofushi T, Sugeno M. Fuzzy Measure and Integrals: Theory and Applications. New York: Physica-Verlag, 2000

[26] Sirbiladze G, Gachechiladze T. Restored fuzzy measures in expert decision-making. Information Sciences, 2005, 169: 71-95

[27] Liu Zhi-Qiang, Bruton L T, Bezdek J C et al. Dynamic image sequence analysis using fuzzy measures. IEEE Transactions on Systems, Man, and Cybernetics-Part. B: Cybernetics, 2001, 31(4): 557-572

[28] Liu Bao-Ding, Liu Yan-Kui. Expected value of fuzzy variable and fuzzy expected value models. IEEE Transactions on Fuzzy Systems, 2002, 10(4): 445-450

[29] Zhang Qiang, Gao Long-Chang, Du Wen. Quasi-probabilities and conditional quasi-probabilities. Journal of Southwest Jiaotong University, 1998, 33(4): 436-441(in Chinese) (张强,高隆昌,杜文. 拟概率和条件拟概率. 西南交通大学学报, 1998, 33(4): 436-441)

[30] Devroye L. Exponential inequalities in nonparametric estimation//Roussas G ed. Proceedings of the Nonparametric Functional Estimation and Related Topics. Dordrecht: Kluwer Academic Publishers, 1991: 31-44