The Prague technology company Secret Weapons has released the beta version of its new tool EbSynth, with which rotoscopy can be automated in any style. Animators provide a video reference, but they don't need to paint over each frame, they just paint a series of keyframes. EbSynth creates all intermediate images in the same artistic style, which can be completely arbitrary – oil paint, watercolor, pastel or digital media. The algorithm respects the artist's vision and preserves his style in all details. This workflow significantly reduces the time an animator would have to spend on a project.
Jakub Javora, main artist at Secret Weapons, explains: “The motivation is simple: We would like to see picturesquely animated films in the cinemas. And we think the trend is moving in that direction – Klaus, for example, is such a great film. The new animated Spider-Man was amazing. And then there is Loving Vincent who was painted frame by frame and people thought it was crazy. Hand-drawn animation has been an incredibly tedious process so far. This need not be. With EbSynth, we want to help animators speed up their work while preserving their authentic handwriting. We want each of our tools to be that “secret weapon” for artists, helping them focus on creation and eliminate routine work. "
EbSynth is currently in the beta version, which can be downloaded for free at https://ebsynth.com.
The technology behind EbSynth is not artificial intelligence. It is a texture synthesis algorithm. "You can think of it as a puzzle," explains the company's senior researcher, Ondřej Jamriška. “EbSynth receives a video frame and its stylized pair of versions. It breaks down the stylized frame into small parts and knows the positional and color-based correspondences with the original frame. Other video images are also assembled from the stylized parts. "
"By using synthesis, we ensure that the end result is rich in texture and maintains the artist's style with great fidelity. This is a quality that most neural methods don't have," he adds.
Originally intended as a tool for rotoscopy, the range of applications for the program has expanded to include the VFX industry and 3D animation since its alpha release last year. “3D artists started using EbSynth to enhance their smooth renderings with brush strokes. Compositors use it to quickly retouch or colorize videos. Even people who have nothing to do with art often use it together with FaceApp, for example to create videos with their older ones or younger selves, ”notes Šárka Sochorová, manufacturer of secret weapons. “It's great to see people come up with new and original uses for EbSynth. The number of different use cases makes sense because the idea is so simple: do an edit in one frame, EbSynth distributes it to the rest of the video. It's free 2D tracking that can be used in so many fields. However, we would love to see these hand-drawn feature films most. "
Source: Secret Weapons
Dan Sarto is the Editor and Editor-in-Chief of Animation World Network.