18th February 2020
We last spoke with researchers at the EU H2020 ICT AnDy project in 2019, discussing the creation of collaborative robotics that possess cutting-edge, anticipatory capabilities. To dive deeper into the project’s research, we sat down with Serena Ivaldi, Research Scientist at AnDy, to find out more about their open-source data center that hosts motion capture recordings of both individual movements and sequences. This biomechanical data can be used to inform ergonomic factors and predict movements before they’re completed.
Open-source data
Researchers at AnDy use Xsens MVN Analyze to track the motion of human posture in real-time. This translates to Activity Recognition used when performing an automatic assessment of full-body ergonomics. By recording multiple variations of body movements and common movement sequences, researchers can create a data set that can reference these movements and predict future exertions ahead of time.
“We want an automatic system that can recognize and predict the activity the human is exerting using data-driven methods that are trained using an open-source data-set,” explained Serena.
The data set is freely available online and consists of more than 5 hours of recorded motions from people with different anthropomorphic features. The data set—which is reusable—is annotated with labels that describe the activity, posture, and ergonomic evaluation. Pauline Maurice, a postdoctoral researcher who was instrumental in making the data set reusable, stated:
“Building the data set was already a huge task—taking months—but to process the data and write the documentation in a way that makes it usable was challenging. Even if the user doesn’t have an Xsens mocap suit, they can play the data and use it for analysis.”
Complete picture
An advantage of inertial motion capture is its ability to pick up and record full-body movements without cameras, meaning no aspect of a movement is left unseen. Blackspots in motion analysis can be detrimental to any research conducted on biomechanics because a lack of data means less-reliable results.
“MVN Analyze is particularly good at solving the problems commonly associated with camera-based motion tracking. There are occlusions, and you need a huge number of cameras to track the human-movement successfully. If the human is moving and performing different postures—such as crouching on the ground or bending to manipulate an object—some parts of the body aren’t visible to the cameras and those parts of the tracking are gone. It certainly highlights the advantages of inertial motion-tracking for this type of research,” explained Serena.
The researchers utilize other measuring technologies alongside Xsens to obtain an even greater depth of posture analysis. One such measuring device is a glove with sensors:
“When a person is manipulating or lifting an object, the glove measures the moment of contact made with the tool—it’s important when differentiating the movements.”
Moment of prediction
Predicting movements is a scientific art that draws upon a pool of previously recorded data to ensure prediction is implemented accurately. With the data set produced by AnDy, it’s possible to reference-check common sequences and deduce the most likely outcome or continuation of a movement.
“We use MVN Analyze to retrieve the postural information and this information is then fed into our machine-learning tools. We have a technique enabled to automatically determine the features that are relevant when building models of activity.”
“We look at the most probable trajectory that the human is supposed to perform. Imagine you are picking up a heavy box, we have the classical recommendation that you use your knee instead of bending your back. But if we see that the human is going to pick up the box while bending the back, we can compute the future trajectory. Let’s say in 400 milliseconds, the person is going to complete a bad posture—we can say at the moment of prediction: stop” said Serena.
© Inria/Loria
This technology can be used in industrial jobs for correction and training, teaching the employees to make the correct movements. But it’s also possible to improve movements in sports, enhancing movements within a fitness criterion.
“The software has to be adapted to the data that users want to promote. It can optimize trajectory, but you need to select the criteria.”
Current Research
Data collection has been one of the primary functions of the team’s work—it fuels the predictive capabilities of their models. An integral part of producing such a large data set is measuring how effective the data is at providing improvements in human biomechanics and ergonomics. The abundance of data recorded and labeled can provide a broad set of functions and applications for research.
“The data set we have collected is very rich; you can achieve human motion analysis, machine learning, and prediction. Other open data sets provide repetitions of single movements, our set has entire sequences.”
The ability to predict movements has positive implications for modes of robotics. This includes collaborative robotics that work with humans and exoskeletons worn by users to complete tasks.
© Inria/Loria
“We are continuing to improve the prediction aspect at the whole-body level—we want to improve anticipatory information in robots. With predictive functions, the robot can plan the appropriate assistance for specific tasks. If the robot can predict the way the user is committing an action then it can help in a specified way. Humans do this all the time intuitively, it’s encoded in your brain. The capability of predicting fast is going to be really crucial, the robot needs to be immediate.”
With the continual advancement of workplace robotics, an anticipatory function with a high level of precision is a necessary part of an increasingly safe working environment.
Xsens Motion Capture Data Files
Are you actively looking for a motion capture system? Download Xsens motion capture data files to convince you about the quality of our data.
Related papers:
Malaisé, A.; Maurice, P.; Colas, F.; Ivaldi, S. (2019) Activity Recognition for Ergonomics Assessment of Industrial Tasks with Automatic Feature Selection IEEE Robotics and Automation Letters. Volume 4 , Issue 2 , Pages 1132 – 1139, April 2019.
Dermy, O.; Chaveroche, M.; Colas, F.; Charpillet, F.; Ivaldi, S. (2018) Prediction of Human Whole-Body Movements with AE-ProMPs. Proc. IEEE/RAS International Conf. on Humanoid Robots (HUMANOIDS).
Maurice, P.; Malaisé, A.; Amiot, C.; Paris, N.; Richard, G.-J.; Rochel, O.; Ivaldi, S. (2019) Human Movement and Ergonomics: an Industry-Oriented Dataset for Collaborative Robotics. The International Journal of Robotics Research. Volume 38, Issue 14, Pages 1529-1537.
European Union's Horizon 2020
An.Dy has received funding from the European Union's Horizon 2020 Research and Innovation Programme under Grant Agreement No. 731540.
Related articles
