Deepfake makes Robert De Niro speak in fluent German! I almost forgot he is American

Latest update time：2021-05-31

Reads：

Fengse from Aofei Temple
Quantum Bit Report | Public Account QbitAI

When a movie is dubbed in another language, the actors' facial movements and lip movements will be very inconsistent in the new language.

If you don't believe me, just look at this: when Zhang Ziyi's Crouching Tiger, Hidden Dragon was dubbed into English:

Forrest Gump in German, Japanese and Spanish:

</p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> If you haven't seen either of these movies, would you doubt that neither of them is actually an original soundtrack? </p> <h2 style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);text-align: left;margin-top: 40px;margin-bottom: 40px;line-height: 1.5;font-weight: bold;padding-left: 15px;word-break: break-all ;border-left: 6px solid rgb(0, 153, 127) ;font-size: 20px ;letter-spacing: 1px ;word-spacing: 1px ;"> Deepfake has another place to go </h2> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> The thing is, the director of the movie "The Untold Story" was shocked after watching its foreign version, saying that the dubbing that did not match the characters' lip movements ruined many of the carefully crafted scenes in the movie. </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> You should know that when dubbing a movie into other languages, the character dialogue is sometimes drastically changed in order to make it closer to the actors' lip movements. </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> "I was very sad at the time," the director said. "You make a small change in a line or a performance, and it can have a big change in the rhythm of the story for a character, and then affect the entire movie." </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> In the end, he couldn't stand watching the "distorted" foreign dubbing in his own movies anymore, <span style="text-decoration: line-through;"> so he decided to stop dubbing and just listen to the original sound and read the subtitles </span> . He then began to "focus" on the latest <strong style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 153, 127);"> Deepfake </strong> technology. </p> <p style="text-align: center;"> </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> He began researching and participating in an AI dubbing project led by a professor at Germany's Max Planck Institute, <span style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(143, 143, 143);font-size: 16px;"> which has produced 18 Nobel Prize winners in science .</span> <br/> </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> Their AI dubbing work is much more complex than traditional deepfakes. It requires capturing the facial expressions and movements of an actor in a scene, as well as someone speaking the same lines in another language. This information is then combined to create a 3D model that combines the actor's <strong style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 153, 127);"> face and head </strong> with the voice actor's <strong style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 153, 127);"> lip movements .</strong></p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> The director and Flawless, a company he co-founded, ultimately drew inspiration from the project and discussed with studios the decision to apply the technology to films in multiple languages. </p> <p style="text-align: center;"> </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> Then we have the demo effect we saw earlier, where the characters’ dialogues are dubbed in German and other languages, but the actors’ lip movements and facial movements closely match the language, making it impossible for the audience to realize that the original version was actually filmed in English. <br/> </p> <p style="text-align: center;"> </p> <h2 style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);text-align: left;margin-top: 40px;margin-bottom: 40px;line-height: 1.5;font-weight: bold;padding-left: 15px;word-break: break-all ;border-left: 6px solid rgb(0, 153, 127) ;font-size: 20px ;letter-spacing: 1px ;word-spacing: 1px ;"> The actor was very disturbed after watching it </h2> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"><strong style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 153, 127);"> This technology can help film companies avoid expensive reshoots when producing multilingual versions. However, some actors expressed surprise and concern </strong> when they saw themselves being manipulated by AI and "speaking" fluent foreign languages . </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> It’s not hard to understand why Deepfakes have always been controversial. </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> This readily available face-changing technology has been used to synthesize celebrity pornography, as well as amateur pornography videos for revenge or other purposes. In the political field, it has also been used to synthesize celebrity videos to make inappropriate remarks and influence political events such as elections. This has had a very bad impact on society and ethics. </p> <p style="text-align: center;"> </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> The use of Deepfake facial manipulation in the film industry may also cause controversy. </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> The head of a company that is also working on AI dubbing said that it is impossible to determine how some actors will react to seeing their performances changed in this way, especially when they don't know that it is done by a computer. Some more famous actors may not agree with this at all. </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> The Screen Actors Guild <span style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(143, 143, 143);font-size: 16px;"> (SAG) </span> general counsel said the technology can be used legally and ethically. Any use of such technology must be done <strong style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 153, 127);"> with the consent of the actors involved and with appropriate compensation </strong> . </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> I wonder if everyone is looking forward to this technology being legally used in the film industry? </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> <span style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(143, 143, 143);font-size: 14px;"> Reference Links: </span> </p> <p style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(0, 0, 0);font-family: Arial;white-space: normal;background-color: rgb(255, 255, 255);font-size: 16px;text-align: left;letter-spacing: 1px ;word-spacing: 1px ;line-height: 2 ;min-height: 1.5em ;box-sizing: border-box ;overflow-wrap: break-word ;margin: 20px 16px ;"> <span style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(143, 143, 143);font-size: 14px;"> [1] </span> <span style="-webkit-user-drag: none;-webkit-app-region: no-drag;color: rgb(143, 143, 143);font-size: 14px;"> https://www.wired.com/story/ai-makes-de-niro-perform-lines-flawless-german/ </span> </p> <div> <div powered-by="xiumi.us"> <div> <div powered-by="xiumi.us"> <div> <div> <div powered-by="xiumi.us"> <div> <div> <div powered-by="xiumi.us"> <div> <div powered-by="xiumi.us"> <div> <div powered-by="xiumi.us"> <div> <div> <div powered-by="xiumi.us"> <div> <div style="box-sizing: border-box;font-size: 16px;"> <div powered-by="xiumi.us" style="margin: 15px 0% 25px;opacity: 0.8;box-sizing: border-box;"> <div style="text-align: center;box-sizing: border-box;"> <p style="box-sizing: border-box;"> -over- <strong style="box-sizing: border-box;"> </strong> </p> <p style="box-sizing: border-box;text-align: left;"> <span style="font-size: 12px;color: rgb(178, 178, 178);"> This article is the original content of [Quantum位], a signed account of NetEase News•NetEase's special content incentive plan. Any unauthorized reproduction is prohibited without the account's authorization. </span> </p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> <p style="box-sizing: border-box;text-align: center;"> <span style="color: rgb(178, 178, 178);font-size: 13px;"> <strong style="font-size: 16px;"> <span style="color: rgb(0, 153, 127);"> click here <span style="color: rgb(178, 178, 178);"> </span> </span> </strong> </span> </p> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> </div> <div> <hr/> <p><h2>Latest articles about</h2></p> <div class="hdlist"> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388733.jspx" target="_blank" title="Domestic 4o large model, understand the national style Li Ziqi in seconds"><span class="serial-num">■</span>Domestic 4o large model, understand the national style Li Ziqi in seconds</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388734.jspx" target="_blank" title="The search engine for life is free to use, the open source version of Harry Potter's "Pensieve" is on the GitHub hot list, and it supports Chinese"><span class="serial-num">■</span>The search engine for life is free to use, the open source version of Harry Potter's "Pensieve" is on the GitHub hot list, and it supports Chinese</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388735.jspx" target="_blank" title="iPad can use AI painting interactive editing tool to become popular, netizens: tremble PS"><span class="serial-num">■</span>iPad can use AI painting interactive editing tool to become popular, netizens: tremble PS</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388736.jspx" target="_blank" title="Real data for various tasks, large-scale online shopping benchmark Shopping MMLU open source｜NeurIPS&KDD Cup 2024"><span class="serial-num">■</span>Real data for various tasks, large-scale online shopping benchmark Shopping MMLU open source｜NeurIPS&KDD Cup 2024</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388737.jspx" target="_blank" title="Scheduled for December 11, registration for the MEET2025 Smart Future Conference has opened!"><span class="serial-num">■</span>Scheduled for December 11, registration for the MEET2025 Smart Future Conference has opened!</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388738.jspx" target="_blank" title="2499, AI concentration is off the charts! Wear this pair of glasses, order coffee/real-time translation/AR navigation in one sentence"><span class="serial-num">■</span>2499, AI concentration is off the charts! Wear this pair of glasses, order coffee/real-time translation/AR navigation in one sentence</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388739.jspx" target="_blank" title="Terminus launches its first universal intelligent agent, achieving high-dimensional perception of the physical world"><span class="serial-num">■</span>Terminus launches its first universal intelligent agent, achieving high-dimensional perception of the physical world</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388740.jspx" target="_blank" title="HKUST's embodied robotics team receives billions of yuan in funding"><span class="serial-num">■</span>HKUST's embodied robotics team receives billions of yuan in funding</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388741.jspx" target="_blank" title="ChatGPT paid features are free! Mistral copied Canvas and Artifacts"><span class="serial-num">■</span>ChatGPT paid features are free! Mistral copied Canvas and Artifacts</a> </dt> </dl> <dl class="subject" style="width: auto;"> <dt><a class="ov-astrict" style="display: inline;" href="https://en.eeworld.com/mp/QbitAI/a388742.jspx" target="_blank" title="Qwen2.5 updates millions of super-long contexts, speeding up inference by 4.3 times. Netizens: RAG is going to be outdated"><span class="serial-num">■</span>Qwen2.5 updates millions of super-long contexts, speeding up inference by 4.3 times. Netizens: RAG is going to be outdated</a> </dt> </dl> </div> </div> <script src="/mp/countpv/109776"></script> </div> </div> </div> </div> </div> <div class="clear"></div> </div> <section id="newsfooter"> <div class="boxfit boxline2 boxbg1"> <div class="boxwrapmix" style="width:1400px;"> <div class="newsfooter clearfix"> <div class="newsftln"> <div class="qrcode"><p> <br>EEWorld WeChat Subscription</p><img src="//6.eewimg.cn/news/statics/qrcode/ffdyh.jpg"></div> <div class="qrcode" ><p> <br>EEWorld WeChat Service Number</p><img src="//6.eewimg.cn/news/statics/qrcode/fwh.jpg" /></div> <div class="qrcode"><p> <br>AutoDevelopers</p><img src="//6.eewimg.cn/news/statics/qrcode/car.jpg"></div> </div> <div class="newsftr"> <p class="newf_p1"> <span><a href="/about_us/about_engver.html">About Us</a></span> <span><a href="/about_us/service_engver.html">Customer Service</a></span> <span><a href="/about_us/contact_engver.html">Contact Information</a></span> <span><a href="/datasheet/">Datasheet</a></span> <span><a href="/sitemap.html">Sitemap</a></span> <span><a href="/latestnews/">LatestNews</a></span> </p> <p class="newf_p3"> <span><img src="/statics/images/channel_footpic2.jpg" /></span> <span><img src="/statics/images/channel_footpic3.jpg" /></span> <span><img src="/statics/images/channel_footpic4.jpg" /></span> <span>Room 1530, Zhongguancun MOOC Times Building,Block B, 18 Zhongguancun Street, Haidian District,Beijing, China</span> <span>Tel:(010)82350740</span> <span>Postcode：100190</span> </p> </div> </div> </div> </div> <div class="clearfix"></div> </section> <section id="newsfooterbtm"> <div class="boxfit boxbg2"> <div class="boxwrapmix"> <div class="footcopy txtc6"> <span>Copyright © 2005-2024 EEWORLD.com.cn, Inc. All rights reserved</span> <span>京ICP证060456号</span> <span><a href="https://beian.miit.gov.cn/" target="_blank">京ICP备10001474号-1</a></span> <span><a href="//www.eeworld.com.cn/corp/pifu.html" target="_blank">电信业务审批[2006]字第258号函</a></span> <span><a target="_blank" href="http://www.beian.gov.cn/portal/registerSystemInfo?recordcode=11010802033920" ><img src="//www.eeworld.com.cn/statics/img/beian.png" style="margin:0;" />京公网安备 11010802033920号</a></span> </div> </div> </div> </section> <script type="text/javascript"> var recommend = 0; $(function(){ $("#owl-example").owlCarousel({ items: 1, autoPlay: true, pagination: true, navigation: true, navigationText: [" "," "], }); $(function(){ $('#owl-demo').owlCarousel(); }); $('.leftbar-bottom').slimScroll({ height: $(window).height()-$(".leftbar-top").height()-$(".leftbar-middle").height() -70 }); $('.leftbar-bottom').slimScroll().bind('slimscroll', function(e, pos){ if(pos=='bottom'){ $(".leftbar-shadow-bottom").hide(); } else{ $(".leftbar-shadow-bottom").show(); } }); }) //左侧悬浮初始代码 $(window).scroll(function() { var s_top = $(window).scrollTop(); if (s_top >39) { $(".leftbar").css("top","0"); } else{ $(".leftbar").css("top","80px"); } }); </script> <script type="text/javascript" src="//static.eewimg.cn/news/mp/static/js/show.js"></script> <script type="text/javascript">(function(d,c){var a=d.createElement("script"),m=d.getElementsByTagName("script")[0],eewurl="//counter.eeworld.com.cn/pv/count/";a.async=1;a.src=eewurl+c;m.parentNode.insertBefore(a,m)})(document,230)</script> <script type="text/javascript">(function(d,c){var a=d.createElement("script"),m=d.getElementsByTagName("script")[0],eewurl="//counter.eeworld.com.cn/pv/count/";a.async=1;a.src=eewurl+c;m.parentNode.insertBefore(a,m)})(document,282)</script> <script> var _hmt = _hmt || []; (function() { var hm = document.createElement("script"); hm.src = "https://hm.baidu.com/hm.js?35c04b29d8ecb826fee3a2975f2b06af"; var s = document.getElementsByTagName("script")[0]; s.parentNode.insertBefore(hm, s); })(); </script> </body> </html>

Deepfake makes Robert De Niro speak in fluent German! I almost forgot he is American

Fengse from Aofei Temple Quantum Bit Report | Public Account QbitAI

Fengse from Aofei Temple
Quantum Bit Report | Public Account QbitAI