A new music video uses 200 drones, 3D animation in Cinema 4D and the Moves by Maxon AR toolkit to create a truly unique music experience.
"High in Heaven" is sung by an anonymous artist who goes by the name VWLS and can be seen in the video as an animated drone face over a beautiful midnight sky. While the concept for the music video came before the COVID hit, it was the perfect solution for a music video that was recorded during quarantine as this video does not contain any actual actors.
Hobbes Creative Director Nick Forshee is the creative mastermind behind the music video.
questions and answers
FXG: Congratulations on the project. How difficult was it to resolve the face to be animated? How many faces did you consider viable options?
NF: We narrowed down from about 6-8 initial sketches. The first few looked like Josh, but after a week of exploring, we realized that a more deliberate design would be better suited for drones.
FXG: The main face has strong nasolabial folds which, together with the single strong drone light for each eye, give the face a droid-like / AI look – was that your intention or am I bringing my own worldview to bear too strongly?
NF: Funny you should say that! The first time we tried a face (in 2019) was purposely made to look like AI as it was part of the finals for an AI conference drone show in California. It didn't sing, but it taught us that strong line choices really sold the dimension of a face. We really wanted the face to feel like a mix of AI and a cosmic entity. The cheek lines and eyes of individual drones were definitely droids / AI inspired, but the eyebrow line gave it personality / wisdom.
FXG: I am fascinated by the subject of lip synchronization. Human lips can move very quickly, but you would have had physical limits on how far (or fast) a drone could move – which in turn would limit the range of the visages I assumed?
NF: We decided to animate the 3-minute song in real time and then slow down the finished flight paths 5x, which corresponds to a 15-minute show. After filming, we sped up the footage 5x to perfectly restore lip sync. This gave us the flexibility to sync the lips and keep the drones throttled to 15 mph to meet Firefly's requirements.
FXG: What were your thoughts on overall facial density? I'm assuming there are safe distances between the drones, but you could have made the face denser by bringing them closer … and yet, as the later face of the exploded view shows, the effect also works as a sparse cloud very well?
NF: For security reasons, Firefly requires that the drones stay 15 feet apart. This is very difficult for density so we squeezed as many as possible. But even with the distance maximized, the resulting face was still about 300 feet tall.
FXG: Given the 3D face, how directed were the lights? How far away from the front position could a viewer be and still “read” the face?
NF: The lights are on the bottom of the drones, which makes them almost invisible from above (apart from small LEDs on top for planes). From the front, the face works from almost any distance. The illusion breaks when you look at the face from the side where you would see overlaps and unobstructed lines. The cameramen, especially the video drone operator, have been given instructions to avoid these angles. Plus, we knew we'd be shooting on the ground from the hill and tilted the entire showdown 8 degrees for better movie options.
FXG: One of the immense aspects of the clip is that it is real. So lighting up the hill with the Comms Tower seems really important. How much did this "locating and seeing the face above the physical hill" play a role in your planning?
NF: This was 1000% intentional. The size would have felt like nothing without context. Ski lifts are a good indicator of the size and depth of a remote urban area. Against the sky, the drones look like dots, but when they illuminate a whole hill you can really feel how big it is.
FXG: Did you even research flying in the face with the camera drone? You get pretty close to the page, but hard to see the lens on the camera drone?
NF: Yes! But the camera drone had to keep a safe distance from the swarm. And since the owner of the mountain did us a favor, we showed our best behavior.
FXG: This seems to have been shot on a slope in the summer?
NF: Bingo, Pine Knob Ski Area in Michigan.
FXG: Have you examined the blinking of the eyes?
NF: We investigated a two-state blink where a set of drones was turned off to indicate another closed state that was hidden (similar to a zebra crossing sign). But that ate up too many drones so we decided not to blink.
FXG: How much control did you have over the lights?
NF: We have complete control over color and intensity. But with great force comes difficult design decisions to keep it simple and to create the impression of a "huge blue face".
FXG: Which drones did you use? How tall were they
NF: The drones were specially designed, built and operated by Firefly Drone Shows for this type of application. A total of 200 drones + a camera drone were used. They are each about a foot in diameter.
FXG: How many takes do we see in the clip? Was this a multiple take edit?
NF: We shot 3 takes in 2 locations. One at the Pine Knob Ski Area and the other at the Firefly Test Field in Holly, Michigan. The entire performance can be viewed as 1 take, but editing it would help keep the audience a little more engaging.
FXG: Can you outline the key technical steps – how did you use C4D and the AR toolkit to adjust the physical limits of the drones and the speed at which they could move? And what software programmed the drones?
NF: We used features from Maxon like a Swiss Army knife. An important milestone is Moves By Maxon's real-time face capture, which allowed us to record 4 unique performances. With Pose Morphs we could choose the "energetic" performance when the song was recorded and choose the "stoic" performance when the face needed to be more monolithic / attentive. We were able to use a surface deformer to glue individual dots (drones) onto the face mesh. From there we monitored speed and collisions. After everything was checked out, we sent our finished show to Firefly, who converted the data for 200 individual flight routes. We were able to achieve a lot without really delving into C4D. Most of the features we've used have been proprietary for over a decade.
FXG: What was the communication solution? If a drone lost communication, there was a plan – a safe return to base? I am sure there would be safety concerns about such things?
NF: Firefly uses specially designed double redundant data links to ensure this never happens. However, Firefly has many proprietary fail-safe features and procedures in place to correct such a problem should it occur.
FXG: How long would the drone and light battery run a sequence before having to return to base?
NF: At warm temperatures 16 minutes.
FXG: What was the clip shot on? How did you balance the brightness of the light with the ISO of the filming and thus the lighting of the ground cover and trees?
NF: Hands down, the Sony A7Sii used by the Firefly crew caught the light best. They use this camera to record most of the shows for this reason.
FXG: Was there a lot of post-production for the footage? Classification or power windows for better readability?
NF: The only grading was to keep the blues consistent. We suspected that people would assume the drones were CGI, so we purposely kept them very raw. Even its amazing glow towards the end is only due to the lens becoming foggy on the night the temperature dropped.
FXG: I'm assuming the entire VWLS animation from the face pose was a problem as well, as the letters have to sit coplanar and the drones would risk a collision if they only moved from one important 3D cloud location to the letters. Was this collision detection just a function of the C4D or a custom process?
NF: Unfortunately, this is also a trade secret … but it took almost a year to come up with something that could solve this complex problem. A drone moving out of the path of another drone can also cut the trajectory of another drone. This problem keeps getting worse.
FXGUIDE: Thank you very much