How Xsens technology brought the 21 Pilots’ virtual concert to life in the Roblox Metaverse
21 Pilots have performed for thousands of fans at the Grammys, Wembley Arena and Reading festival, but they have never had an experience quite like taking to the virtual stage in Roblox and their 35 million daily users. As the media and technology industries attempt to find their ‘Metaverse’, the digital world of Roblox provides a window into what the concert-going experience might look like in the next internet’s next stage.
Virtual concerts aren’t new, but previous renditions have had their share of challenges, with the biggest hurdle being trying to encapsulate the energy of an in-person show in the digital realm. Despite this, it was essential for 21 Pilots that the emotion of their performance transcended across into the virtual space. This is why Actor Capture and GSU’s CMII Studios sought out Xsens, implementing our inertia-based technology to bring the concert experience into players’ homes.
We sat down with long-time Xsens collaborator, and Actor Capture Technical Director, James Martin, to discuss how the project came about, and he was quick to point out how essential Xsens technology was in capturing the 21 Pilots’ performance.
“It is essential for drummers and specifically for musical performances to utilize inertia based and furthermore, a hand solution that allows you to get the grip or things like notation on a fretboard or on a keyboard. It's very, very, very important. All of these things are prevalent and this performance.”
Making use of Xsens’ tools
Lil Nas X’s Roblox show was a rousing success for Symbol Zero, but with their next virtual concert, they wanted to take things to the next stage, and engage the player at an unprecedented level. Having built up an in-depth understanding of Xsens through his work on past projects ‘Suicide Squad 2’ and Cardi B’s ‘Cardi Tries’, James knew that our technology was the best way to address the challenges 21 Pilot’s virtual performance might have brought.
“We knew that the work would have over 15 minutes of animation. We also knew that they were very interested in doing face bonding and hands because instruments would be involved. They (Symbol Zero) weren't just looking for somebody who was versed in the moves and the Xsens workflow. They were looking for somebody who could utilize things like the Manus gloves that now carry the timecode over, which is very important to get the finger articulation. And then, of course, a face solution that was robust enough to get all of the lyric performance.”
Capturing the articulation of the performance was an important factor for Roblox and 21 Pilots. The band wanted to carry across the emotions of a concert to the virtual space, so it was essential the right tools and methods were chosen. MVN Link suits were worn by the band as they performed, while Xsens Gloves by Manus were also used to ensure hand movements were accurately captured. Alongside this, James made use of the iPhone’s ARkit to track the face movements of 21 Pilots.
The Metaverse experience
In the virtual spaces that will make up the metaverse, it will be the individualities that the user will seek out to make themselves stand apart. For now, these come in the form of digital clothes, or NFT art, but as the user base grows, so does the adoption of aspects of reality. This is where the 21 Pilots Roblox concert might serve as a preview for the role Xsens can play in the metaverse. During the recording process, it was clear that there was value attributed to the movements of 21 Pilots. It was essential for the player that the way the drums or the guitar was played, matched that of the band. It’s not out of the question that the metaverse will see similar demand for realistic, and unique avatar movements, which is where technology like the Manus gloves will come into play.
Recording the performance
Whether the stage is real or virtual, 21 Pilots have always sought to have a connection with their loyal fan base. As explained by Mark Eshleman, the band’s Creative Director, they wanted “to honor the band’s fans, its lore, and history to make sure this experience aligns with what people can expect from a show.”
The question is no longer whether performance can be captured for the virtual space, but more so, can the virtual concert capture the sophistication and emotion that comes with an in-person show? Thanks to Xsens’ inertia-based solutions, James was able to address these concerns and created an environment that allowed 21 Pilots to perform just as they would in any arena around the world.
“In the behind-the-scenes, you'll see that we had all their instruments in the CMII Studios stage, and we didn’t hinder their performance. That's really the true magic of inertia-based solutions; we bring a solution to a performer and meet them in their own native environment.”
The success of the metaverse will be dictated by the ease of adoption and use. It will be the technology, like the Xsens tools that allowed James and Actor Capture to record the essential, intricate details of performing, that will be necessary for the metaverses’ ability to maintain a sustained suspension of disbelief.
Engaging the player
Hosting a concert within the world of Roblox allowed 21 Pilots new opportunities to interact with their audience in a way only possible in the virtual space. It was important for Symbol Zero that the player remained a part of the performance, rather than being a passive spectator.
“Where at first you're just a captive audience member just watching a performance, the interactivity that is offered through Roblox engages the game player. In the opening track that they utilized, one of the options was that the player would go up and down alongside the hook of the song, and if the player did not jump when Tyler said the lyric ‘up’, then they would fall through the floor. It's smaller interactivity that makes the game player have a little bit more of an intimate experience with the performance.”
Users will venture into virtual worlds in the metaverse for an enhanced experience of their reality, not just a 1-1 depiction of their reality. Roblox implemented quests, rewards and virtual merchandise into the 21 Pilots concert, not just as a way to sustain and market the performance but to keep players engaged in a way that differs from an in-person experience.
Actor Capture used Xsens MVN Animate as their solution. It enables you to mocap anywhere, at any time. Want to know what we can do for you? Get in touch!
Are you actively looking for a motion capture system and want to compare data? Download Xsens motion capture data files to convince you about the quality of our data.