The Tech Behind Lady Gaga’s Jaw-Dropping Grammys Performance

While the digital makeup and special effects filters that Snapchat offers are all the rage these days, Lady Gaga and Intel accomplished pioneering work in a similar vein earlier this year with augmented reality and face tracking — and they had the additional challenge of doing it live and on national television.

In February, at the 2016 Grammy Awards, Lady Gaga and Intel teamed up to put on a tech-fueled spectacle that was unlike any other performance seen before. Rock legend David Bowie died the month before, and Gaga was given the role of honoring the music icon at the awards show.

The performance opened with a tight shot of Gaga’s face as Bowie’s signature red lightning bolt manifested over her eye while she sang his classic song “Space Oddity.” The lightning bolt, however, was just the beginning. A spider appeared and moved around on Gaga’s face, and more conventional things like blush and eyeshadow were rendered digitally and tracked digitally, following her every movement.

“We were really pushing the envelope; it's unlike anything done before,” says Lakshman Krishnamurthy, Intel Fellow in the New Devices Group. “If you look at some of the face-tracking work with the people we collaborated with before, most of it was with static models. Nobody was moving their lips and mouth and shaking and singing a song. Getting that technology to work and work in a way that 25 million people are going to watch live, and however many more millions later, you know it can't fail, and getting it to that level of functionality required the engineers to solve new algorithms on the fly.”

When Technology and Art Combine

Art and science are often thought of as two completely separate worlds. But some of the best technology is driven by the smartest, most innovative creative. So why can’t some of the best creative be fueled by some of the most cutting-edge technology?

Intel, as a brand, has a storied history as a chipmaker. Their processors are in countless desktop and notebook computers, servers and mobile devices across the world. But Intel is eager to move beyond its chip processor history and move forward into a truly boundary-breaking role as technology innovator.

“Our new brand promise is to connect Intel’s longstanding legacy from being focused on Intel Inside to asserting and owning responsibility for actually creating experiences,” says Penny Baldwin, Vice President, Global Marketing and Communications Group and General Manager, Brand, Reputation and Partner Marketing.

With a focus on “high-impact” opportunities, Intel identified the Grammys as a major opportunity to leverage its engineering talents and also shift perceptions of its organization on a global scale.

“The idea was to mash up amazing technology with amazing creativity and reinvent the live performance experience at the Grammys because it has a built-in audience and a massive social and digital following,” says Baldwin. “That in turn led to reaching out to Lady Gaga because we knew she was the type of artist that would embrace technology and really respond to the concept overall, and she did.”

When Baldwin says Gaga is the type of artist to embrace technology, she’s not exaggerating. In 2013, Gaga and her creative team, the Haus of Gaga, unveiled a flying dress called Volantis as part of her marketing campaign for her album Artpop.

For Gaga, the melding of creativity and technology isn’t so far-fetched to her as it is for other pop stars.

“I think what’s so exciting about this collaboration is I get to shine a light on all of these scientists, that are artists, in my opinion. All of the things that they develop, that they research, that they invent; they are dreamers. And that is at the essence of being an artist,” said Gaga in a behind the scenes video from Intel.

Pushing Live Entertainment to the Next Frontier

The face-tracking and augmented reality technology might have taken center stage, but there was a lot of technology that Intel and its partners brought to the table that went undetected by the average viewer.

Gaga was wearing the Curie ring, a wearable technology ring from Intel that leverages the Intel Curie processor, which Baldwin describes as its “smallest processor” since it’s the size of a button. In order to track her movements and ensure that data was transmitted from the ring to the servers, Intel had to turn to ultra-wideband technology (UWB), since Wi-Fi and Bluetooth wouldn’t suffice.

“One of the cool things about ultra-wideband and the technology that we used is it has the ability to give you precise location,” says Krishnamurthy. “People have been doing location tracking with Wi-Fi indoors, which allows for sub-meter resolution. For what we wanted to do, we wanted sub-10-centimeter resolution. We were able to actually track location within a standard deviation of plus or minus 2 centimeters.”

To fuel this tracking system along with the intensive graphics rendering required to pull off the performance, Intel put together a heavy-duty back-end system.

“There were several server racks backstage that were running the performance. The face-tracking computer was one high-end computer but we had to render the content, literally the spider and all the effects that you see, in a server farm,” says Krishnamurthy. “This content that was used needed a lot of server capacity. It was almost like running a data center that we rolled into Universal Studios.”

Compounding the complexities associated with implementing emerging technology is the fact that the decision to turn Gaga’s performance into a Bowie tribute came just prior to showtime, which forced Gaga and Intel to rethink numerous aspects of plans that had been in the works for nine months.

“Changing up the artist at the last minute did pose technology challenges for our engineering team, but they literally ate, slept, and lived on the sound stage that we created for all the rehearsals, for every second for at least a month, leading up to the live performance,” says Baldwin.

Logistically, things worked out differently on the rehearsal stage in Universal Studios than they did on the actual Grammys stage at the Staples Center.

“Many of those constraints were not known until we built the practice stage and we sort of duplicated the Grammys stage at Universal Studios. But it's only when we started putting the show together that we discovered some of the things that we thought would work would completely not work, and we either had to cut them off or adapt them,” says Krishnamurthy.

The dancing piano was a challenge for the Intel team in particular, since they had to convert robots that were typically used in manufacturing, and robots in a factory aren’t typically tasked with doing a one-two step.

“We had to make those robots dance like she was dancing. So we had to tap the data from her choreographer and the choreographer said, ‘This is how I'm going to move. The robots should move this way,’ ” says Krishnamurthy.

Tapping Creativity from Unexpected Places

While your organization might not be tasked with singing and dancing to David Bowie classics at the Staples Center like Lady Gaga, there are opportunities to learn from the ingenuity and innovative spirit that helped make the collaboration between art and technology in this scenario such a success.

“I feel that with innovation, new things can come from anywhere. And if you just look at the maker community, and you go to one of these maker faires, you have anybody with something in their garage coming up with something amazing and showing it to the world,” says Krishnamurthy.

Intel is actively supporting startups and entrepreneurs as part of its involvement in America’s Greatest Makers which is both a digital hub and a reality TV show that airs on TBS. The Curie ring that Gaga used in the Grammys performance was in fact put to use by inventors for other applications on the show.

“What’s coming out of America’s Greatest Makers is a whole range of new ideas and innovations and products that Intel would not ever necessarily pursue but that our technology makes possible, and we will ultimately help the winning contestant then take that product to market,” says Baldwin.

Above all, perhaps the biggest lesson learned is that creativity and technology don’t have to live on opposite sides of the spectrum. They can, and should, be partners and collaborators. Whether that’s for an application in fueling a jaw-dropping concert experience or spearheading an incredible month for your small business, technology helps make it possible.

“When I was a kid I used to watch the Grammys, and to be part of this as a geek was an incredible experience,” says Krishnamurthy, reflecting on the entire experience. “You need to have creativity, but without the hard work it’s just impossible to pull these things off. From a personal standpoint, this is something I will remember as a high point in my professional career.”