Reallusion has introduced free Audio2Face plugins for iClone
Audio2Face is an artificial intelligence-based application for creating facial animation from audio, available as part of the NVIDIA Omniverse platform. While it can also animate changes in facial expression, its primary purpose is to generate lip-sync animation from an actor's voice recording. Unlike many automated lip-sync systems, it is not limited to English language only - in the 2023.1 update, a new AI model based on a female Chinese character was added. Reallusion's announcement highlights its "exceptional ability" to generate animation from any language, including "songs and gibberish."
While Audio2Face already supports Reallusion characters and can transfer generated facial animation to other applications, the new plugins significantly streamline the workflow.
The first plugin, CC Character Auto Setup, exports CC3+ characters from Character Creator or iClone to Audio2Face, "reducing the manual 18-step process to just one."
The second plugin, Audio2Face (A2F) for iClone, transfers facial animation created by Audio2Face back to iClone and includes controls for adjusting or smoothing the animation. This workflow serves as an alternative to iClone's built-in AccuLips automatic lip-syncing system.
The Audio2Face plugins for iClone are compatible with iClone 8.4 and above, Audio2Face 2023.2 and above, and Character Creator 4.4 and above. They can be downloaded for free.
To use them, you also need the free Character Creator and iClone Omniverse Connectors.
Character Creator and iClone are available for Windows 7 and above. The cost of new licenses is $299 and $599 respectively. Audio2Face is available for Windows 10 and above and is a free part of Omniverse.