Virtual Set Design – fxguide

James Connelly is a Primetime Emmy Award winner, production designer, art director and set designer, and founder of JP Connelly. The company specializes in pre-vis and digital set design for television and special events. The company prides itself on combining its art-oriented architectural solutions that focus on storytelling with physical and virtual sets. Her notable projects include NBC's The Voice and Bravos Top Chef, as well as Celebrity Show Off and The Masked Singer. The company uses real-time game engines to produce live sets based primarily on green screen stages.

JP Connelly recently faced a number of complex design issues related to shooting in a pandemic environment where the studio audience is not allowed and social distancing is required on set. The use of a complex, large-scale virtual production enabled programs like Mayim Bialik's Celebrity Show-Off on TBS to film in an almost empty phase while presenting an interactive game show.

Fxguide's Mike Seymour walked through James Connelly's personal process in this exclusive clip below, from the initial visualizations of a set to the final real-time UE4 rendering and some of the processes behind the sets for Masked Singer and their other virtual productions.

For many people, Virtual Production suggests LED volume levels, especially on otherwise empty sound stages. While JP Connelly has taken a close look at LED stages and created several music videos on LED stages, Connelly points out that LED stages can be problematic for multi-camera recording. “For Celebrity Show Off, it was very efficient to use green screen during COVID. We had to make sure Mayim was completely isolated and that we had a very experienced actress. She didn't necessarily have to see the world around her (on LED screens), ”explains Connelly. “I think LEDs are an efficient way of taking photos and a lot of creativity. But there are still a few bugs to fix. The nature of my TV is all multi-camera. So with a large set of LEDs, you have to be very careful about which camera is recording what, and you can remove the option to edit later as you have to set recordings from all of the live cameras. So it can be a bit tricky and the solution is an algorithm that I can't quite figure out just yet. "The production dilemma is that an LED screen, controlled by a game engine like UE4, adjusts the image to be correct for the lens. Aside from the exact location of the camera, the image can be sent to the LED Walls appear distorted or incorrect, which means that the one LED screen cannot simultaneously render two viewing cones for two cameras live in perspective. The only solution would be to switch LED screens live while the director switches cameras. This enables both The background for the actors on set is constantly fading in and the ability to change the editing later because editing is locked. But Connelly With 160 new LED screen levels in the US by the end of the year there will be an incredible amount of inventory that people can explore further, the option that JP Connelly is currently exploring is to manage a master set and only green ta Tracking fields on the LED screens for each of the actors so that the ambient lighting is correct but the actual image can be adjusted for each camera in the mail. “Allow the LED screen to turn green. They have a really nice key, but even for cross-shots, you can get into the virtual world they were put into, ”he adds. "I'm assuming that if we go into a completely virtual space, we'll be looking at a lot of mixed media to get this to work and that means we'll have to add quite a bit of technology."

An LED set also doesn't capture the same illusion of space that many JP Connelly projects have experimented with. The virtual designs no longer assume that the set is visually set up on the ground floor of the studio. Celebrity Show Off is a prime example and one of the second or third fully virtual productions the team has recently done. "I'm excited to see what we can do," he explains. “If you walk through the Warner Brothers or Paramount stages and look into a soundstage, you can tell what a movie set is and what a TV. Film sets are typically six or seven feet tall because the DP wants to put their camera on the floor and we give them creative freedom. On TV, you build right on the ground. You just don't have that kind of budget. “As a designer, Connelly is excited to see what he can do with an“ infinite floor ”.“ We have practically an endless budget, so we're making our own city … I have a passion for pushing these boundaries of depth and negative space and that definitely determined the design and concept of Show Off. "

The company doesn't just focus on stage design. With cinematography, art direction, and visual effects increasingly merging, especially during COVID, the team is addressing a whole range of new topics such as the virtual audience. "We are currently working on a project for Facebook that has a virtual audience, and it is very important that the host of this project connects with you when necessary, sees the action and reacts," explains Connelly. Before COVID, many shows gained in production value and the feeling of being an "event" due to a large studio audience. Shows often featured sweeping footage from the back of the studio with what appeared to be hundreds of not thousands of people. "This has been our constant discussion for the past six months – something I talked about every day," he says. “There is a negative space to fill a theater, and even in Greek amphitheatres you would fill that space with an audience for thousands of years. You feel your energy even more than you do for the landscape. And so suddenly with the pandemic you don't have that anymore, so there is a bias towards technology. "With the Masked Singer when Connelly designed the first set of the first season, the audience was a" character "in designing the program." They were three-quarters of the room. And as we approached this season, – oh boy, me couldn't even tell you how many versions we went through to figure out what it looked like without an audience … it was a challenge. It's a real design challenge. "

JP Connelly uses the Unreal Engine for their virtual sets and dynamic lighting. The company is keen to investigate the latest hardware GPU cards, and UE5 in particular, as they release. Mainly because of its ability to push the boundaries of what can be done in the team's real-time render budgets. When it comes to real-time ray tracing, Connelly believes his team has about a year off. "It's a little early for us, we still use some dynamic lights, but we bake our lights and play it safe to make sure they work perfectly in real time."

Interestingly, when asked for his advice to the VFX community on dealing with virtual art departments and digital production design, Connelly points out that VFX often thinks in terms of recordings and keyframes while production designers think in terms of creating behavior. “We set up productions to create behavior, and the recordings come from that. And so it is only in the final steps of our process when we get a chance to really influence the "recordings" … but we create the worlds first to get to those recordings, so don't fret us! "he jokes." It's really exciting when you marry the two worlds. It's really exciting because I think VFX really takes some technical aspects into account, but there is an art of world building that I think is production design and Art department bring with them the table ".


Please enter your comment!
Please enter your name here