doi:

DOI: 10.3724/SP.J.1089.2010.11032

Journal of Computer-Aided Design & Computer Graphics (计算机辅助设计与图形学学报) 2010/22:9 PP.1619-1627

A Survey on Machine Learning in the Synthesis of Human Motions


Abstract:
In recent years, machine learning techniques have been widely used in the synthesis of human motions and many excellent results are achieved, by taking advantage of the mature and the wide use of motion capture. Motion capture allows the data-driven methods become the mainstream techniques for human motion generation. Moreover, machine learning techniques are capable of handling effectively the high dimensionality of human motion data and their correlations in spatial and temporal domain. Based on the characteristics of various machine learning techniques, a survey on the application of machine learning in human motion synthesis is given in this paper. The future development in this research area is also discussed.

Key words:human motion,machine learning,motion capture,data driven,virtual humans

ReleaseDate:2014-07-21 15:25:43



[1] Xia S H, Wang Z Q. Recent advances on virtual human synthesis[J]. Science in China Series F: Information Sciences, 2009, 52(5): 741-757

[2] Dinerstein J, Egbert P K, Cline D. Enhancing computer graphics through machine learning: a survey[J]. The Visual Computer, 2007, 23(1): 25-43

[3] Hertzmann A. Machine learning for computer graphics: a manifesto and tutorial[C]// Proceedings of Pacific Graphics. New York: IEEE Computer Society Press, 2003: 22-36

[4] Duda R O, Hart P E, Stork D G. Pattern classification[M]. 2nd ed. Beijing: China Machine Press, 2003 (in Chinese) (Duda R O, Hart P E, Stork D G. 机器学习[M]. 曾华军, 张银奎, 译. 北京: 机械工业出版社, 2003)

[5] Mitchell T M. Machine learning[M]. New York: McGraw-Hill Campanies, 1997

[6] Xiao Jun, Zhuang Yueting, Wu Fei. Computer vision and machine learning in 3D human animation: a survey[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(3): 281-290 (in Chinese) (肖 俊, 庄越挺, 吴 飞. 计算机视觉与机器学习技术在三维人体动画中的应用综述[J]. 计算机辅助设计与图形学学报, 2008, 20(3): 281-290)

[7] Rose C, Cohen M F, Bodenheimer B. Verbs and adverbs: multidimensional motion interpolation[J]. IEEE Computer Graphics and Applications, 1998, 18(5): 32-41

[8] Rose C F, Sloan P P J, Cohen M F. Artist-directed inverse-kinematics using radial basis function interpolation[J]. Computer Graphics Forum, 2001, 20(3): 239-250

[9] Torresani L, Hackney P, Bregler C. Learning motion style synthesis from perceptual observations[C]// Proceedings of NIPS 19. Cambridge: The MIT Press, 2007: 1393-1400

[10] Wang Y, Liu Z G, Zhou L Z. Key-styling: learning motion style for real-time synthesis of 3D animation[J]. Computer Animation and Virtual Worlds, 2006, 17(3-4): 229-237

[11] Mukai T, Kuriyama S. Geostatistical motion interpolation[J]. ACM Transactions on Graphics, 2005, 24(3): 1062-1070

[12] Liu C K, Hertzmann A, Popovic Z. Learning physics-based motion style with nonlinear inverse optimization[J]. ACM Transactions on Graphics, 2005, 24(3): 1071-1081

[13] Cooper S, Hertzmann A, Popovic Z. Active learning for real-time motion controller[J]. ACM Transactions on Graphics, 2007, 26(3): Article No 5

[14] Ikemoto L, Arikan O, Forsyth D. Generalizing motion edits with Gaussian processes[J]. ACM Transactions on Graphics, 2009, 28(1): Article No 1

[15] Brand M, Hertzmann A. Style machines[C]// Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH. New York: ACM Press, 2000: 183-192

[16] Li Y, Wang T S, Shum H Y. Motion texture: a two-level statistical model for character motion synthesis[J]. ACM Transactions on Graphics, 2002, 21(3): 465-472

[17] Hsu E, Pulli K, Popovic J. Style translation for human motion[J]. ACM Transactions on Graphics, 2005, 24(3): 1082-1089

[18] Chai J X, Hodgins J K. Constraint-based motion optimization using a statistical dynamic model[J]. ACM Transactions on Graphics, 2007, 26(3): Article No 8

[19] Wang J M, Fleet D J, Hertzmann A. Gaussian process dynamical models for human motion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 283-298

[20] Lau M, Bar-Joseph Z, Kuffner J. Modeling spatial and temporal variation in motion data[J]. ACM Transactions on Graphics, 2009, 28(5): Article No 171

[21] da Silva M, Abe Y, Popovic J. Interactive simulation of stylized human locomotion[J]. ACM Transactions on Graphics, 2008, 27(3): Article No 82

[22] Ren L, Patrick A, Efros A A,et al. A data-driven approach to quantifying natural human motion[J]. ACM Transactions on Graphics, 2005, 24(3): 1090-1097

[23] Glardon P, Boulic R, Thalmann D. PCA-based walking engine using motion capture data[C]// Proceedings of the Computer Graphics International. Washington D C: IEEE Computer Society Press, 2004: 292-298

[24] Urtasun R, Glardon P, Boulic R,et al. Style-based motion synthesis[J]. Computer Graphics Forum, 2004, 23(4): 799-812

[25] Li Chunpeng, Wang Zhaoqi, Xia Shihong. Motion synthesis for virtual human using functional data analysis[J]. Journal of Software, 2009, 20(6): 1664-1672 (in Chinese) (李淳芃, 王兆其, 夏时洪. 人体运动的函数数据分析与合成[J]. 软件学报, 2009, 20(6): 1664-1672)

[26] Mori H, Hoshino J. ICA-based interpolation of human motion[C]// Proceedings of IEEE International Symposium on Computational Intelligence in Robotics and Automation. New York: IEEE Press, 2003: 453-458

[27] Shapiro A, Cao Y, Faloutsos P. Style components[C]// Proceedings of the Conference on Graphics Interface. Quebec: Canadian Human-Computer Communication Society, 2006: 33-39

[28] Liu G D, Pan Z G, Lin Z Y. Style subspaces for character animation[J]. Computer Animation and Virtual Worlds, 2008, 19(3-4): 199-209

[29] Shin H J, Lee J. Motion synthesis and editing in low-dimensional spaces[J]. Computer Animation and Virtual Worlds, 2006, 17(3-4): 219-227

[30] Grochow K, Martin S L, Hertzmann A,et al. Style-based inverse kinematics[J]. ACM Transaction on Graphics, 2004, 23(3): 522-531

[31] Li Chunpeng, Wang Zhaoqi, Xia Shihong,et al. Inverse kinematics using local support poses[J]. Chinese Journal of Computers, 2007, 30(11): 1982-1988 (in Chinese) (李淳芃, 王兆其, 夏时洪, 等. 基于局部支撑姿态的逆运动学求解[J]. 计算机学报, 2007, 30(11): 1982-1988)

[32] Tournier M, Wu X, Courty N,et al. Motion compression using principal geodesic analysis[J]. Computer Graphics Forum, 2009, 28(2): 355-364

[33] Carvalho S R, Boulic R, Thalmann D. Interactive low-dimensional human motion synthesis by combining motion models and PIK[J]. Computer Animation and Virtual Worlds, 2007, 18(3-4): 493-503

[34] Raunhardt D, Boulic R. Motion constraint[J]. The Visual Computer, 2009, 25(5-7): 509-518

[35] Liu G D, Pan Z G, Li L. Motion synthesis using style-editable inverse kinematics[M]// Lecture Notes in Artificial Intelligence. Berlin: Springer, 2009, 5773: 118-124

[36] Safonova A, Hodgins J K, Pollard N S. Synthesizing physically realistic human motion in low-dimensional behavior-specific spaces[J]. ACM Transactions on Graphics, 2004, 23(3): 514-521

[37] Ye Y, Liu C K. Animating responsive characters with dynamic constraints in near-unactuated coordinates[J]. ACM Transactions on Graphics, 2008, 27(5): Article No 112

[38] Zhu Dengming, Wang Zhaoqi. Extraction of keyframe from motion capture data based on motion sequence segmentation[J]. Journal of Computer-Aided Design & Computer Graphics, 2008, 20(6): 787-792 (in Chinese) (朱登明, 王兆其. 基于运动序列分割的运动捕获数据关键帧提取[J]. 计算机辅助设计与图形学学报, 2008, 20(6): 787-792)

[39] Assa J, Caspi Y, Cohen-or D. Action synopsis: pose selection and illumination[J]. ACM Transactions on Graphics, 2005, 24(3): 667-676

[40] Tang B, Pan Z G, Zheng L,et al. Interactive Generation of Falling Motions[J]. Computer Animation and Virtual Worlds, 2006, 17(3-4): 271-279

[41] Pan Zhigeng, Cheng Xi, Tang Bing. A real-time algorithm for character reactive animation generation[J]. Journal of Computer Research and Development, 2009, 46(1): 151-158 (in Chinese) (潘志庚, 程 熙, 唐 冰. 一种实时虚拟人反应式动画生成算法[J]. 计算机研究与发展, 2009, 46(1): 151-158)

[42] Sok K W, Kim M, Lee J. Simulating biped behaviors from human motion data[J]. ACM Transactions on Graphics, 2007, 26(3): Article No 107

[43] Arikan O, Forsyth D A, O'Brien J F. Motion synthesis from annotations[J]. ACM Transactions on Graphics, 2003, 22(3): 402-408

[44] Zhang L, Xiao J, Zhuang Y T,et al. Competitive motion synthesis based on hybrid control[J]. Computer Animation and Virtual Worlds, 2009, 20(2-3): 225-235

[45] Peng J Y, Lin I C, Chao J H,et al. Interactive and flexible motion transition[J]. Computer Animation and Virtual Worlds, 2007, 18(3-4): 549-588

[46] Tang J K T, Leung H, Komura T,et al. Emulating human perception of motion similarity[J]. Computer Animation and Virtual Worlds, 2008, 19(3-4): 211-221

[47] Ong E J, Hilton A. Learnt inverse kinematics for animation synthesis[J]. Graphical Models, 2006, 68(5-6): 472-483

[48] Lee J, Chai J X, Reitsma P S A,et al. Interactive control of avatars animated with human motion data[J]. ACM Transactions on Graphics, 2002, 21(3): 491-500

[49] Barbic J, Safonova A, Pan J Y. Segmenting motion capture data into distinct behaviors[C]// Proceedings of the Conference on Graphics Interface. Quebec: Canadian Human-Computer Communications Society, 2004: 185-194

[50] Sakamoto Y, Kuriyama S, Kaneko T. Motion map: image-based retrieval and segmentation of motion data[C]// Proceedings of Eurographics-ACM SIGGRAPH Symposium on Computer Animation. New York: ACM Press, 2004: 259-266

[51] Wu S Y, Xia S H, Wang Z Q,et al. Efficient motion data indexing and retrieval with local similarity measure of motion strings[J]. The Visual Computer, 2009, 25(5-7): 499-508

[52] Kovar L, Gleicher M, Pighin F. Motion graphs[J]. ACM Transactions on Graphics, 2002, 21(3): 473-482

[53] Lee J, Lee K H. Precomputing avatar behavior from human motion data[C]// Proceedings of Eurographics-ACM SIGGRAPH Symposium on Computer Animation. New York: ACM Press, 2004: 79-87

[54] McCann J, Pollard N. Responsive characters from motion fragments[J]. ACM Transactions on Graphics, 2007, 26(3): Article No6

[56] Lo W Y, Zwicker M. Real-time planning for parameterized human motion[C]// Proceedings of Eurographics-ACM SIGGRAPH Symposium on Computer Animation. New York: ACM Press, 2008: 29-38

[57] Cheng X, Liu G D, Pan Z G,et al. Fragment-based responsive character motion for interactive games[J]. The Visual Computer, 2009, 25(5-7): 479-485

[58] Lee Y J, Lee S J, Popovic Z. Compact character controllers[J]. ACM Transactions on Graphics, 2009, 28(5): Article No 169

PDF