How Are AI and Machine Learning Used in the Animation Industry?

How Are AI and Machine Learning Used in the Animation Industry

From creating a database to rotoscoping, here’s how AI and machine learning can speed up the production process in the animation industry.

While artificial intelligence (AI) and machine learning (ML) are commonly used for facial recognition or creating personalized shopping experiences in e-commerce, have you thought about how they can also be used in the animation industry? 

The animation industry is known for using innovative and ground-breaking technologies and techniques for its works, and it is constantly searching for new ways to improve and speed up the production process. In particular, AI and ML can be helpful tools that can take care of the labor-intensive work so that animators can focus on creating and animating instead. 

Creating a database for future research

In the past, the archive process required writers and animators to manually tag all the necessary information into the database, which was very time-consuming. Tagging involves identifying the contextual story and character information from structured data, like storylines, character archetypes or motivations.

Thanks to technology, things can get done so much quicker. In 2018, Disney formed the Direct-to-Consumer & International Organization (DTCI). It aims to use ML and deep learning (a type of AI and ML that teaches computers to acquire knowledge like humans do—through examples) tools to automate metadata tagging and speed up the archiving process, where people input different data into a centralized database for easy reference.

Together with Amazon Web Services (AWS), they have created an archive called Content Genome that can automate the digital archival of all Disney content. This archive can help animators find a specific reference shot or sequence from all the Disney animated films or concept arts.

The new archive from Disney uses AI and ML to automate the tagging process, so that animators do not need to manually tag information themselves. This program can automatically tag the content into the database to speed up the tagging process. The tags, which contain descriptions of animated films, can identify different characters in a film, their relationships, the film soundtrack and the background scenery.

As a result, writers and animators can now quickly search for and familiarize themselves with the sources without manually going through every film. This also reduces the amount of work required to organize the Disney library since people just need to approve the algorithms’ tags instead of manually creating tags by themselves. 

Generating 3D animation and face models

Besides creating databases, AI and ML can also generate highly accurate and realistic 3D face models and character movements. In particular, Disney has used such technologies to make designing and simulating 3D faces easier. 

One of the most important and challenging aspects of animating a character is getting their facial expressions right. As humans, we spend so much time recognizing faces and can tell when a face is off or unnatural. Hence, to create a convincing facial animation, animators must go the extra mile and ensure that their 3D rendered faces are as realistic and accurate as possible. 

While animators can easily model a still 3D face, it is a tremendous task for them to animate one filled with facial expressions while capturing all the subtle details. However, with ML, researchers at Disney can build a semantic deep face model that can generate and manipulate 3D faces through 3D face modeling.

This model combines various modeling techniques, such as facial performance transfer, performance editing and 3D face synthesis, which is the process of using still images to create facial expressions. It uses the neutral 3D geometry of a face with the target expression and recreates it into the desired expression. For example, it can take a still 3D face and turn it into a smile or a frown, depending on the animators’ input. 

Animation tech startup Midas Interactive has launched an automated animation engine, Midas Creature, that can create complex 2D/3D animations for films and video games. They aim to use AI and ML to choreograph and animate complex character animations so that it can speed up production time and make modeling a quicker process. For instance, Midas Creature can animate hair and cloth motions instantly. The powerful animation tools allow artists and animators to build, pose, warp and bring their animated characters to life with realistic and natural movements at less cost. 

Automating the rotoscoping process

Rotoscoping is an animation technique that lets animators create realistic characters that move like humans. It creates animations by tracing animated sequences over the live-action sequence and drawing the animated characters according to the movements of humans. 

Automating the rotoscoping process
Example of the rotoscoping process (Image Source: Vox)

The animation industry has been using rotoscoping to make visual effects flow smoothly on the screen. But the problem is that it is a very tedious task and usually requires thousands of hours of manual artist labor. However, recent AI and ML developments allow animators to automate that process. 

Laika Studios, a U.S.-based animation company famous for producing films, like Coraline and Kubo and the Two Strings, has partnered with Intel to combine ML and AI to develop a program that can streamline the animation process. 

When producing the 2019 stop-motion animated film Missing Link, the production team at Laika needed to swap out magnetic faceplates on the puppets, which are small facial components with different expressions. For instance, when a character needs to show a smile, the puppeteers need to swap out the different expressions of that puppet, from a resting face to a smiling face. But since the faceplates are not seamless, there is a gap between the different parts of the puppet’s face, which creates a line around the eyes of all Laika’s puppets as shown below. 

Automating the rotoscoping process 1
Example of removing the facial lines through the AI program (Image Source: Enterprise AI)

The artists will then need to rotoscope the line individually and make them blend in with the rest of the puppet’s face, which will be very tedious. But with AI and ML, animators don’t have to sit at their desks to rotoscope out the lines manually. The 3D modeling program powered by Intel can shorten the time required to rotoscope 70 frames from five to six hours to five minutes and 25 seconds. As a result, it can cut production time by a margin and massively reduce the animators’ workload. 

The above examples show how AI and ML can help automate and speed up the production process and save a lot of time for animators. This way, they can spend more time producing animations instead of doing labor-intensive work. In recent years, we have witnessed the speed at which AI and ML have been expanding the modern scope of animation to unseen levels, and we are excited to see what AI and ML can do in the future. 

Also Read:

Image Courtesy of Unsplash


Share on facebook
Share on twitter
Share on linkedin
Share on email


5 More Business Podcasts You Need to Listen To

5 More Business Podcasts You Need to Listen To

Entrepreneurs have one thing in common—they always look for opportunities to learn and grow. To gain an understanding of running a viable business, you need to listen to the experiences of those who have emerged triumphant in the business world. A great way to do so is by tuning into a vast number of business and entrepreneurship podcasts scattered across the internet today.

5 Hobbies To Maximize Your Entrepreneurial Drive

5 Hobbies To Maximize Your Entrepreneurial Drive

Entrepreneurs are a special breed of person. They’re go-getters, risk-takers and dreamers. Yet, even the most driven entrepreneur needs to take a break from time to time. Spending every second of your day on your business is stressful, and you will lose your drive very soon. If you want to stay energetic and ready to take on new challenges at work, hobbies are what you need.

Making the Metaverse a Force for Good with the Metaverse Charity Foundation

Making the Metaverse a Force for Good with the Metaverse Charity Foundation

If you are a frequent reader of our website, you must have seen us mention The Sandbox, Decentraland and Axie Infinity. These are some of the most popular metaverses out there, and the one thing they all have in common is that they all have their origins in Asia. While Asia is home to all these metaverse projects, it suffers two main issues: the region has major rural-urban inequality as well as a significant wage difference between high-skilled and low-skilled occupations.

How Do Stores Get You to Spend More Money

How Do Stores Get You to Spend More Money?

Have you ever entered a grocery store to buy some milk but ended up with a bag full of items you didn’t even need? You’re not alone if you feel guilty for spending more money than you intended. It’s not our fault that we keep putting items in our shopping carts; we are manipulated to do so! To make sure you buy more, retailers will go all the way to carefully engineer every aspect of their store.

6 Lessons All Entrepreneurs Can Learn from Amazon Founder Jeff Bezos

6 Lessons All Entrepreneurs Can Learn from Amazon Founder Jeff Bezos

Whether you love or hate him, there’s no denying that Jeff Bezos is one of the most successful entrepreneurs of our generation. Bezos, the founder of Amazon, has built a business empire encompassing everything from ecommerce and cloud computing to artificial intelligence (AI). What can we learn from Bezos, one of the world’s wealthiest men with a net worth of US$151 billion? Here are six key lessons that all entrepreneurs can learn from the billionaire:

9 Signs of a Toxic Workplace

9 Signs of a Toxic Workplace

Workplace trends like “hustle culture” and “quiet quitting” are shining the light on unhealthy, toxic work environments and their effects on employees. Hustle culture is when you feel like you have to work yourself to the bone and do way more than others all the time; quiet quitting, a counteragent to hustle culture, is about doing the bare minimum at your job instead of going above and beyond for an exploitative employer.