A start-up company has a tool that it says can recreate lip sync - matching a speaker’s lip movements with pre-recorded audio - in films. The artificial intelligence, or AI, tool may change how film producers replace recorded speech in a movie with speech recorded in another language. This process is known as dubbing. Start-up Flawless AI, co-founded by film director Scott Mann, has a tool that it says can recreate lip sync without changing the performance of the actors. The tool studies how actors move their mouths. It then changes the movements to sync with the replaced, or dubbed, words in different languages. The result could be that Tom Hanks looks like he can speak Japanese or Jack Nicholson looks like he can speak French. The idea for the tool came to Mann when he saw how dubbing affected his 2015 film "Heist," starring Robert De Niro. I hate dubbing as it stands, Mann told Reuters. He went on to describe the problems that dubbing can create. You have to change so many things to try and catch sync. You're changing words filmmakers and performers have thought about so deeply. They're thrown out to find a different word that fits, but it never really does. Mann decided to do something about this issue. After some research, he discovered a paper written by Christian Theobalt of the Max Planck Institute for Informatics in Germany. The paper lays out a new method that uses artificial intelligence to recreate human visual effects. The tool worked to translate a De Niro scene in Heist. Man said the tool was able to take an 'ooh' sound from De Niro 20 minutes earlier and place that in a different moment in the film. He added, "It measures at the same time and blends it so that the performance is the same, but it's a different mouth movement." Blend is a term that means to mix something, often with good results. Flawless AI is now working with producers and studios to use the technology after filming is complete. The first films using this tool are around a year away. Mann believes this technology will help to remove foreign-language categories at award shows. It could create a more open film industry with a real global stage.
Title: AI Tool Could Change Language in Films
Content: <div id='article-page'><div id='article-content' data-media-url='//news-app-staging.s3.amazonaws.com' data-base-url='//news-app-staging.herokuapp.com' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/C.wav' data-capture-url='//staging.analytics.lingraphica.com/events/capture_news' data-article-title='AI Tool Could Change Language in Films' data-article-id='5661'><script src='//news-app-staging.herokuapp.com/javascripts/getscripts.js'></script><link rel='stylesheet' href='//news-app-staging.herokuapp.com/stylesheets/article.css' type='text/css' /><div class='article'><p><span data-start-time='0' data-end-time='13771'>A start-up company has a tool that it says can recreate lip sync - matching a speaker’s lip movements with pre-recorded audio - in films.</span> <span data-start-time='13771' data-end-time='27917'>The artificial intelligence, or AI, tool may change how film producers replace recorded speech in a movie with speech recorded in another language. </span> <span data-start-time='27917' data-end-time='31292'>This process is known as dubbing.</span> <span data-start-time='31292' data-end-time='46875'>Start-up Flawless AI, co-founded by film director Scott Mann, has a tool that it says can recreate lip sync without changing the performance of the actors.</span> <span data-start-time='46875' data-end-time='50938'>The tool studies how actors move their mouths. </span> <span data-start-time='50938' data-end-time='59604'>It then changes the movements to sync with the replaced, or dubbed, words in different languages. </span> <span data-start-time='59604' data-end-time='70854'>The result could be that Tom Hanks looks like he can speak Japanese or Jack Nicholson looks like he can speak French.</span> <span data-start-time='70854' data-end-time='81979'>The idea for the tool came to Mann when he saw how dubbing affected his 2015 film "Heist," starring Robert De Niro.</span> <span data-start-time='81979' data-end-time='86958'>I hate dubbing as it stands, Mann told Reuters. </span> <span data-start-time='86958' data-end-time='92188'>He went on to describe the problems that dubbing can create. </span> <span data-start-time='92188' data-end-time='96792'>You have to change so many things to try and catch sync.</span> <span data-start-time='96792' data-end-time='103125'>You're changing words filmmakers and performers have thought about so deeply. </span> <span data-start-time='103125' data-end-time='109083'>They're thrown out to find a different word that fits, but it never really does.</span> <span data-start-time='109083' data-end-time='113479'>Mann decided to do something about this issue.</span> <span data-start-time='113479' data-end-time='125104'>After some research, he discovered a paper written by Christian Theobalt of the Max Planck Institute for Informatics in Germany.</span> <span data-start-time='125104' data-end-time='134354'>The paper lays out a new method that uses artificial intelligence to recreate human visual effects.</span> <span data-start-time='134354' data-end-time='139396'>The tool worked to translate a De Niro scene in Heist. </span> <span data-start-time='139396' data-end-time='150854'>Man said the tool was able to take an 'ooh' sound from De Niro 20 minutes earlier and place that in a different moment in the film.</span> <span data-start-time='150854' data-end-time='161583'>He added, "It measures at the same time and blends it so that the performance is the same, but it's a different mouth movement."</span> <span data-start-time='161583' data-end-time='167792'>Blend is a term that means to mix something, often with good results.</span> <span data-start-time='167792' data-end-time='177479'>Flawless AI is now working with producers and studios to use the technology after filming is complete. </span> <span data-start-time='177479' data-end-time='182917'>The first films using this tool are around a year away.</span> <span data-start-time='182917' data-end-time='191833'>Mann believes this technology will help to remove foreign-language categories at award shows.</span> <span data-start-time='191833' data-end-time='198167'>It could create a more open film industry with a real global stage. </span> </p></div><div class='control-buttons-sticky' style='display:none;'><div class='control-buttons'><button title='Back' class='back' disabled='disabled'></button><button title='Play' class='play' disabled='disabled'></button><button title='Pause' class='pause' style='display:none;'></button><button title='Forward' class='forward' disabled='disabled'></button><button class='finished-reading' style='display:none;'>Done</button></div></div></div><div id='article-media'><div id='media-image'><img src='https://news-app-production.s3.amazonaws.com/articles/5093/C.jpeg'/></div></div><div id='question-content' style='display:none;'><div class='vocabulary_question question-container' data-question-id=17485 data-hint-location='161583' style='display:none;'><div class='question' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/what_does_the_term_blend_mean.wav'>What does the term blend mean?</div><div class='choices' data-correct-choice='To mix something, often with good results'><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/to_mix_something_often_with_good_results.wav'><span>To mix something, often with good results</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/lacking_strong_features_or_characteristics_and_therefore_uninteresting.wav'><span>Lacking strong features or characteristics and therefore uninteresting</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/fair_or_pale_yellow.wav'><span>Fair or pale yellow</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/to_repair_something_that_is_broken_or_damaged.wav'><span>To repair something that is broken or damaged</span></div></div></div><div class='multiple_choice_question question-container' data-question-id=17486 data-hint-location='27917' style='display:none;'><div class='question' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/what_is_the_process_of_replacing_recorded_speech_in_a_movie_with_speech_recorded_in_another_language_called.wav'>What is the process of replacing recorded speech in a movie with speech recorded in another language called?</div><div class='choices' data-correct-choice='dubbing'><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/dubbing.wav'><span>dubbing</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/relanguage.wav'><span>re-language</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/erasing.wav'><span>erasing</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/over_and_under.wav'><span>over and under</span></div></div></div><div class='sentence_completion_question question-container' data-question-id=17487 data-hint-location='182917' style='display:none;'><div class='question' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/mann_believes_this_technology_will_help_to_remove_foreignlanguage_categories_at__blank_.wav'>Mann believes this technology will help to remove foreign-language categories at <span class='blank'>awards shows</span>.</div><div class='choices' data-correct-choice='awards shows'><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/pageants.wav'><span>pageants</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/awards_shows.wav'><span>awards shows</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/movie_theaters.wav'><span>movie theaters</span></div><div class='choice' data-sound-name='https://news-app-production.s3.amazonaws.com/articles/5093/starbucks.wav'><span>Starbucks</span></div></div></div><div class='question-buttons'><button class='skip-button'></button><button class='hint-button'></button><button class='speak-button'></button></div></div></div>
Sound Name: https://news-app-production.s3.amazonaws.com/articles/5093/C.wav
Preview
