Today we are excited to announce Puppets, an exciting new feature for SwiftKey Beta allowing users to create their own virtual character that mimics their facial expressions and head movements.
Android users can choose between five playful characters: Dinosaur, Panda, Cat, Owl and Dog to record a video with. Puppets can be shared widely and are not limited to any specific messaging app.
“People want to type fast, and SwiftKey is perfect for that,” said Deepak Paramanand, Product Manager at Microsoft. “They also want to express themselves in a fun and interesting way, and now SwiftKey is perfect for that too.”
SwiftKey worked with the Microsoft Computer Vision and Microsoft Research Asia teams to bring Puppets to life. Unlike other facial tracking software, SwiftKey’s Puppets does not rely on users having a device with an inbuilt depth sensor in their camera and instead uses an RGB camera found in most Android smartphones. This means that more people than ever before with a wide range of smartphones will be able to share messages as a friendly character.
Puppets’ AI technology has been trained using thousands of volunteers from around the world recording expressive videos of changing facial expressions. Using these images and recordings, SwiftKey has trained a Deep Neural Network (DNN) to learn how to identify facial movements and transfer these onto an expressive animal character.
“The traditional approach of converting 2D training images into a 3D puppet uses two separate Deep Neural Networks, meaning it would be challenging for even the most powerful modern smartphones to run,” said Baoyuan Wang, Principle Researcher and Manager in Microsoft’s Computer Vision research group who were one of the teams to contribute to Puppets. “We overcame this with a unique approach that has the same outcome but with a much more compact, single model. This means Puppets is available to many more people across a greater range of phones.”
More information about this unique approach can be found in this research paper.
Whilst Puppets is available to all users on Android N and above, those with more recent and powerful smartphones will have the best experience.
We welcome you to get involved in developing Puppets. After you’ve tried out the feature you can provide feedback through this dedicated feedback survey:
SwiftKey Beta is an experimental app that provides early access to SwiftKey updates and new unreleased features for anyone to try and test before they are released to all users. Beta features are still a work in progress and may change over time before full release.
*SwiftKey Beta is a secondary app that provides early access to SwiftKey performance updates, new unreleased features, customisations and special themes before released to all users. Features are still a work in progress and may change over time before full release.
The SwiftKey team