Epic Games releases the free Live Link Face iOS app for UE4
Thursday July 9, 2020 | Written by Jim Thacker
Epic Games has released Live Link Face, a free app for streaming facial animation data from footage of a live actor shot with TrueDepth cameras in modern iPhones to characters in Unreal Engine.
In addition to facial expressions, the app can record head and neck rotation data and offers a number of professional production functions, including the integration of TentacleSync and the support of Open Sound Control.
A powerful app for large virtual productions as well as individual animators
Live Link Face isn't the first tool to use the depth detection features of modern iPhones to collect facial animation data: Rococo and Maxon have both released their own free apps.
However, it is probably the most comprehensive feature, with features aimed at both online streamers and individual animators looking to collect facial reference data, as well as large virtual productions.
Stream facial animations directly from an iPhone to Unreal Engine
For the first user group, Live Link Face transfers facial animation data from the live actor to a 3D character in Unreal Engine for use in real time or to collect reference data that can be manually refined.
The data can be streamed over a wireless network or by connecting the iPhone to a workstation over Ethernet. The 3D avatar can be placed over the phone's video display to preview performance.
As with other apps of this type, it is based on Apple's ARKit augmented reality framework. Therefore, the 3D character requires the standard face blends specified in the ARKit documentation.
According to Epic, the app can be "natively customized when performers are sitting at their desks instead of wearing a head-mounted rig with a mocap suit" and can also stream head and neck rotation data.
The app not only transfers animations directly to the Unreal Engine take recorder, but also captures the actor's reference video material in MOV format and raw facial animation data in CSV format.
Users can browse the library of recorded takes and play the reference video directly in the app.
Supports TentacleSync and OSC to ensure precise synchronization with other Mocap hardware during production
For the second user group, Live Link Face also supports a number of key technologies that are used in professional motion capture sessions and in collaborative virtual production.
The app not only offers "robust timecode support and precise image accuracy", but also TentacleSync, so that a connection to the master clock can be established in one acquisition phase.
This should ensure that the facial animation data recorded via Live Link Face is accurately synchronized with the whole body data recorded by the actors' inertial detection suits or an optical mocap system.
The app also supports the Open Sound Control (OSC) protocol so that other software can control the app remotely: for example, to start or stop recording on the iPhones of several actors at the same time.
In addition to animation data, the app can also record production metadata such as slate names and adopt numbers.
System requirements and availability
Live Link Face is available free of charge for iOS 13.0 and higher. To use it, you need an iPhone with a TrueDepth camera, ie an iPhone X or higher. It is compatible with Unreal Engine 4.25.
Using the Unreal Engine editor itself is free, as is rendering non-interactive content. For game developers, Epic makes 5% of gross over the first $ 1 million earned over the life of a product.
For more information on Live Link Face, visit the Epic Games blog
For more information on the face capture workflow via Live Link Face, see the online documentation
Download the free Live Face Face iOS Mocap app from the App Store
Tags: animation, app, ARKit, CSV, download, epic games, facial animation, facial animation data, face detection, facial mocap, facial performance, face tracking, free, head rotation, iOS, iPhone, iPhone X, live link face, master clock, metadata, mocap, Motion detection, MOV, neck rotation, open sound control. OSC, reference material, slate name, system requirements, take number, take recorder, TentacleSync, timecode, timecode server, TrueDepth, UE4, Unreal Engine, Unreal Engine 4.25, virtual production