The free EbSynth tool turns videos into hand-painted animations
Friday, August 14, 2020 | Written by Jim Thacker
Originally published July 9, 2019. Scroll down for details on the official beta version.
The start-up Secret Weapons has published EbSynth, an interesting free tool for converting video material into stylized "motion painting".
The software transfers the appearance of one or more manually generated style frames to those of the video, creating the appearance of a hand-painted animation.
Based on recent research on style transfer and texture synthesis
EbSynth is based on research by the Czech Technical University in Prague and Adobe Research under the direction of Ondřej Jamriška, founder of Secret Weapons, whose work has been published on Greenscreen in the past.
The EbSynth website shows the demo video from the team's latest article, "Stylizing Video by Example".
In contrast to other work in this area, it is not based on machine learning, but uses a "state-of-the-art implementation" of nonparametric texture synthesis algorithms.
Secret Weapons informs us that EbSynth is now to be developed as a stand-alone tool.
Works from source material and a single hand-painted frame of reference
EbSynth only needs the source video as input – converted to an image sequence in standard 2D file formats – and at least one still image keyframe for reference.
The software then automatically applies the style of the keyframe to the remaining frames of footage, preserving "texture coherence, contrast, and high frequency details".
Since EbSynth does not work with underlying 3D data, it cannot accurately stylize parts of objects that are not shown in the keyframe or that are completely missing.
This means that the keyframe should closely match the footage – ideally it should be overpainting an actual video frame – and that a new keyframe is recommended after every significant change in perspective.
In addition, the user can provide masks to indicate which parts of the video frame the stylization should be applied to: for example, to stylize a character but not the background.
Judging from the demo video at the top of the story, the results are ready to use right out of the box: there are notable transitions when the software switches from one keyframe to another, and a halo region between character and background when it doesn't work with input masks, but the output is very observable.
Updated August 14, 2020: EbSynth has now officially gone into beta.
The new features include automatic filename filling so that directories of images can be dragged into the software and processed automatically instead of having to manually enter file paths.
The software now also exports directly to After Effects, with the recordings automatically sequenced in the After Effects timeline. processing is now "10x faster" than the original version.
Interestingly, Secret Weapons tells us that since EbSynth was originally released, some visual effects artists have started using the software to automate tasks like sorting and removing bugs.
"You edit an image and EbSynth spreads the changes across the entire scene, so it's useful to retouch, colorize, and generally edit video."
Availability and system requirements
EbSynth is available in beta for Windows and MacOS. The software is free for commercial use.
Download EbSynth from the product website
Tags: Adobe, Adobe Research, After Effects, Beta, Troubleshooting, Cleanup, Color Correction, Convert Video to Animation, Convert Video to Motion Graphics, Convert Video to Motion Painting, Czech Technical University in Prague, Download, EbSynth, Featured Articles, Free, Review , new functions, Ondrej Jamriska, retouching, rig removal, source code, style transfer, stylizing videos using examples, system requirements, texture synthesis, VFX, visual effects