Artificial intelligence (or AI) has opened up possibilities that were previously unthinkable at home, vehicles, education, and more. AI technology has also been evaluated as a key informational technology in the video game industry as well. As a leader of technology and culture, Netmarble has also been developing AI technology to international acclaim in the form of speech recognition, computer graphics, computer vision, and more. 

Before we look into the meeting of gaming and AI, let's learn what AI is. AI is a blanket term used to indicate informational technology surrounding artificial intelligence. These technologies are divided into a number of fields with examples like speech recognition in smartphones, computer vision technologies that understand images and videos, language processing programs that allow computers to understand human language, reinforcement learning that find solutions in dynamic situations, and more. There are many other categories as well, and Netmarble is currently focusing on speech, language processing, reinforced learning, computer vision, computer graphics, and other adjacent technology. 

* Shounan An presenting “Robust Keyword Spotting via Recycle-Pooling for Mobile Games” at INTERPSECCH 2019 (Author: Shounan An, Youngsoo Kim, Hu Xu, Jinwoo Lee, Myungwoo Lee, Insoo Oh, Balancing AI team 1, Magellan Division, AI Center, Netmarble)

1. On-Device Speech Recognition

The most notable piece of technology in Netmarble's artificial intelligence is on-device speech recognition. Did you know you can access speech recognition technology in Netmarble games like other applications such as "Hey Google," "Hey Siri," and "Hi Bixby"? <A3: STILL ALIVE> is the first mobile game to automatically hunt, enter shops, and more through voice recognition technology by using the keyword "Monica." In Korea, the Monica keyword and game command recognition technology was announced at the INTERSPEECH 2019 and IEEE ICASSP 2020 conferences. Not stopping there, research continues to create technology that is capable of recognizing voice commands from the device itself rather than through a network, which would be a better fit for mobile games. Recently another research paper titled “MONICA: Mobile Neural voIce Command Assistant for mobile games” got accept to NeurIPS 2020 demonstration session with “Strong Accept” from all reviewers. We anticipate that even better voice recognition software will be available in a number of games very soon. 

* “Robust Eed-To-Eed Keyword Spotting and Voice Command Recognition for Mobile Game” at IEEE ICASSP 2020 (Author: Hu Xu*, Youshin Lim*, Shounan An, Hyemin Cho, Yoonseok Hong, Insoo Oh, Balancing AI team 1, Magellan Division, AI Center, Netmarble)

2. Computer Graphics - Audio Driven Facial Animation

The second piece of AI technology is "Audio Driven Facial Animation." Players can see high detailed dialogue scenes between characters in games now. Basic animation scenes just have characters open and close their mouths, but with "Facial Animation," things become better. It becomes possible to make facial muscles move automatically to match what is being said, which makes the scene much more natural. Netmarble presented audio driven facial animation technology at ACM SIGGRAPH 2019, the most famous conference in computer graphics field. 

* “Multi-Task Audio-Driven Facial Animation” at ACM SIGGRAPH 2019 (Author: Youngsoo Kim* Shounan An*, Youngbak Jo, Seungje Park, Shindong Kang, Insoo Oh, Duke Donghyun Kim, Balancing AI team 1, Magellan Division, AI Center, Netmarble)

3. Computer Vision - Dense Pose

DensePose is one of the computer vision technology which detects humans from 2D images and maps it to a 3D coordinates of the human model to make various poses through deep learning. By participating in the DensePose Challenge, Netmarble developed technology that allows the fine stacking of Up-sampled (increasing the pixel count of a low-resolution image to change it into a high-resolution image) layers to overcome the limitation to Feature Mapping, which has low resolutions. This was recognized as the runner up for the "COCO DensePose Challenge (2018)" hosted by ECCV 2018, one of the major computer vision conferences. 

▲ Result of Mapping a Human's 3D Coordinates using DensePose
* (Author: Shounan An, Seungje Park, Youngbak Jo, Balancing AI team 1, Magellan Division, AI Center, Netmarble)
 
The various AI technologies being developed by Netmarble will be continuously improved to increase game immersion and overall quality. As AI technology continues to garner interest in the gaming industry, we expect these technologies to be the future of the industry which will increase development efficiency and player convenience.