SayMotion - converting text into animation

The company DeepMotion, founded in 2014 by gaming industry veteran Kevin He, is one of the pioneers in the field of motion capture using AI and has paved the way for other recent browser-based mocap tools. With SayMotion, the company ventured into the realm of generative AI, allowing users to create animated clips by inputting textual descriptions of movements they want to generate. The descriptions are divided into three subsections: theme, action, and details.

The theme describes the character's nature (for example, "woman," "child," or "swimmer"), while the action describes the performed action. Detail alters the animation by specifying the direction or orientation in which it should occur, as well as the style or emotions with which the character should perform it. When combined, they can generate quite complex action sequences.
One example from the online documentation uses the description: "A person walks in a circle, tense, and then extends their left hand to grab a chair, abruptly sitting down on it." The original animation can be refined using the SayMotion inpainting system, which allows users to expand, modify, or blend animations with additional textual prompts. In the tutorial example, it demonstrates how the tilting motion, to lift an object, is applied to the animation of a character walking in a circle. The system also supports some additional Animate 3D options from DeepMotion mocap AI service, including motion smoothing and foot locking.

In addition to using pre-made characters from DeepMotion or those created using built-in avatar generation tools, users can upload their own 3D characters for animation. Models from stock sites like Mixamo, Sketchfab, and TurboSquid will also work, provided they have standard humanoid body proportions. Animated characters can be exported in FBX, GLB, or BVH formats, allowing the animation to be used in DCC applications like Blender or Maya, as well as in game engines like Unity and Unreal Engine. Additionally, animations can be exported as MP4 videos.

DeepMotion is currently limited to generating full-body movements for human characters, although the company is considering adding support for facial and hand animation. The company also plans to add support for video prompts, generating new unique animations based on the interpretation of SayMotion's original video.
Currently, SayMotion is in open beta. It is expected to work with any standard desktop web browser, with support for mobile browsers planned for the future. During the beta testing period, all users receive 2,000 credits per month. Creating animation or MP4s, uploading animation, or using inpainting all consume one credit. Animation created using beta accounts is licensed for use in commercial projects. DeepMotion has not announced the release date or final price of SayMotion.