KAIST (President Kwang Hyun Lee) announced on the 25th that a research team led by Prof. Jaemin Hwangbo of the Department of Mechanical Engineering has developed control technology for a four-legged robot that can walk nimbly even on a deformed terrain such as a sandy beach.
Professor Hwangbo’s research group has developed a technology to simulate the force perceived by a walking robot on the ground, made of granular materials such as sand, and simulate it with a quadruped robot. In addition, the team worked on an artificial neural network framework suitable for making the real-time decisions needed to adapt to different types of terrain without prior information during simultaneous walking, and applied it to reinforcement learning. The trained neural network controller is expected to expand the field of application of quadruped walking robots by proving its stability in changing terrain, such as the ability to move at high speed even on a sandy beach, and to walk and turn on soft, air-like ground. mattress without losing balance.
This research from Ph.D. Student Soo Young Choi from KAIST’s Department of Mechanical Engineering was published as first author in January at Scientific robotics. (Title of article: Learning quadrupedal locomotion on deformed terrain).
Reinforcement learning is an AI training method used to create a machine that collects data about the results of different actions in an arbitrary situation and uses that data set to perform a task. Since the amount of data required for reinforcement learning is very large, a simulation-based data collection technique that approximates physical phenomena in a real-world environment is widely used.
In particular, learning-based controllers in the field of walking robots have been applied in a real-world environment after being trained using data collected during simulation to successfully perform walking control on various terrains.
However, since the performance of a learning-based controller degrades rapidly if the actual environment has any discrepancies with the learned simulation environment, it is important to implement a real-world-like environment during the data collection phase. Therefore, to create a learning-based controller that can maintain balance on deformed terrain, a simulator must provide a similar contact experience.
The research team defined a contact model that predicts the force generated at contact from the dynamics of the motion of a walking body, based on a ground reaction force model that takes into account the additional mass effect of granular media identified in previous studies.
In addition, by calculating the force generated by one or more contacts at each time step, terrain deformation was effectively modeled.
The research team also presented an artificial neural network framework that implicitly predicts ground characteristics using a recurrent neural network that analyzes time series data from the robot’s sensors.
The trained controller was mounted on the “RaiBo” robot, which was built in practice by the research team to demonstrate high-speed walking of up to 3.03 m/s on a sandy beach, where the robot’s legs were fully immersed in the sand. Even when used on firmer ground such as grassy fields and a running track, it was able to perform stably, adapting to the characteristics of the ground without additional programming or revision of the control algorithm.
In addition, it rotated with a stability of 1.54 rad/s (about 90° per second) on the air mattress and demonstrated its quick adaptability even in a situation where the terrain suddenly became soft.
The research team demonstrated the importance of providing a suitable contact in the learning process compared to a controller that assumes the ground is rigid, and proved that the proposed recurrent neural network changes the controller’s walking method depending on the ground properties.
The modeling and learning methodology developed by the research team is expected to contribute to practical tasks by robots as it expands the range of conditions under which different walking robots can operate.
The first author, Suyoung Choi, said, “It has been shown that providing a learning-based controller with experience in close contact with real deformed ground is very important for deformed terrain applications.” He further added that “the proposed controller can be used without prior terrain information, so it can be applied to various robot walking studies.”
This research was supported by the Samsung Electronics Research Funding and Incubation Center.
 "The bills exclude most students with waivers from graduation credits")
