Demo: The magic of AI neural TTS and holograms at Microsoft Inspire 2019

Demo: The magic of AI neural TTS and holograms at Microsoft Inspire 2019


so it’s a pleasure to be here in Las
Vegas to present to you now I get
invited to do keynotes across the globe
and while it’s easy for me to be here in
Las Vegas it isn’t always easy for me to
travel across the world and even when I
do I can’t always speak the local
language well what if neither language
nor distance mattered for me to deliver
a fantastic keynote what if technology
could help me be anywhere I needed to be
and speak any language I wanted well it
can we are bringing together the power
of mixed reality and Azure AI services
to create a truly game-changing
experience what you’re about to see is
an exact hologram of me wearing the same
outfit that we recently captured at a
mixed reality studio and I don’t speak
Japanese but what if I wanted to deliver
my keynote in Japanese using Azure AI
technology I can translate my English
into Japanese and train it to sound
exactly like me the same voice tones
those same inflections now we brought
this together my hologram and as Rai to
show you what’s possible so first I’m
gonna put on my hololens to hear and
then we’ll flip in the room to the
special camera so you can see exactly
what I’m seeing let’s get started first
let me introduce you to mini-me there
she is my perfect holograph and thanks
to the power of hololens – she just
floats right with me I’m literally
holding my hologram so natural now she’s
a little small to do a keynote so let’s
get her up so she can do full-sized
Japanese keynote render keynote
providing us what does that about sizing
of Google gains its capture Gidget socio
see what does in a hologram was accessed
Amos did I know Jim boots ahora gramma
to estimate a kotoba are commercially
Masanga
what a cigar design in nihongo Hannah
stay tuned Agha a Pharisee notice what
does your new home go Hannah seem a
singer but the Sunoco a hologram you WA
can pick in any home Hoda honest Amos
Korea a neurotic estoy yo me agita Oh
buddy resides in notes in casino digits
you irony at ATT socio schemas Podesta
to achieve una corner ago COC
personality covetous additional codes in
taking on session a success mass nihongo
conference ago boruto gargle madonna
Sema’s EEMA Koenig is its wha what a
static ah Hatake sob say katsu ho holy
cast a sec I dunno cope Kyoya so he
Kyoto in Osaka Tokido Kyoto so so steamy
Takeda say Missoni see hygge
games it’s an inheritor goddess
as you can see this is mind blowing
technology and what you just saw was my
life-size hologram my exact replica
rendered here in real time speaking
Japanese with my unique voice signature
to do this we use mixed reality
technology to create my hologram and
render it here live then we use Azure
speech-to-text capability in English
transcription to get my speech then use
Azure translate to get this in this
Beach into Japanese and finally applied
neural text-to-speech technology so it
sounded exactly like me
just speaking Japanese and the most
amazing part all of these technologies
exists today the future is here

100 thoughts on “Demo: The magic of AI neural TTS and holograms at Microsoft Inspire 2019

  • July 20, 2019 at 2:52 pm
    Permalink

    I want to work for Microsoft 😍 please hire me 😭😭

    Reply
  • July 20, 2019 at 4:19 pm
    Permalink

    this is damm amazing

    Reply
  • July 20, 2019 at 5:02 pm
    Permalink

    Don't attack me but I seriously heard "Genjutsu ni Naruto korodes" in the end. Idek what that means, I only know the two key words Naruto and Genjutsu…

    Reply
  • July 20, 2019 at 6:52 pm
    Permalink

    Next generation. This product will replace smart phone. everybody use smart glasses.

    Reply
  • July 20, 2019 at 8:58 pm
    Permalink

    Nice combination of technologies, but I would have liked to see them add one more in there that re-rendered the mouth to match the native Japanese tones like the "deep fakes".

    Reply
  • July 20, 2019 at 9:02 pm
    Permalink

    This is the game changer technology introduced by MS. It will change the perspective for monitors. Please make it available for public sooner and inexpensive.

    Thank you MS,
    For developing such tech.

    Reply
  • July 21, 2019 at 1:17 am
    Permalink

    Azure Speech and HoloLens 2 coming together so amazing well

    Reply
  • July 21, 2019 at 5:44 am
    Permalink

    https://youtu.be/6roRB7SlJPs

    Bollywood had this in the nineties.. Aishwarya Rai in Jeans. (2 minutes mark)

    Funny how in that scene they use this to scam the target…

    Reply
  • July 21, 2019 at 7:15 am
    Permalink

    Future is here at Microsoft!!

    Reply
  • July 21, 2019 at 7:29 am
    Permalink

    SAO coming soon

    Reply
  • July 21, 2019 at 12:41 pm
    Permalink

    Why didn't you try Thai as a live???

    Reply
  • July 21, 2019 at 2:25 pm
    Permalink

    Fake af

    Reply
  • July 21, 2019 at 2:46 pm
    Permalink

    This was in a Star trek movie. I'm so glad to see we actually have it now!

    Reply
  • July 21, 2019 at 3:21 pm
    Permalink

    Great job 👍!
    1. So how can you transport round the globe with you behind your hologram?
    2. How many places can you be at the same time?
    The does not match, more robotic than soft.
    3. If you can be at many places or some place other than you original location, will it not be in teleportation. Again, can it be a form of legal time travel with time?
    4. What happens, if something goes wrong with the real person?
    5. Can you have more than one hologram transported to the different places at the same time?

    Reply
  • July 21, 2019 at 4:35 pm
    Permalink

    Isn't this copy from Hua Wei. Hrha, US stealing the technology from China. If not, then what about security threat? No security threats? Then, what the phark of accusing HuaWei as a security threat. Don't lie to the people of US and the world.

    Reply
  • July 21, 2019 at 4:36 pm
    Permalink

    It does not sound like you speaking in English. it was like another person speaking.

    Reply
  • July 21, 2019 at 7:13 pm
    Permalink

    > I will be in love with mine.

    > Next target: A Doll with AI

    Reply
  • July 21, 2019 at 7:52 pm
    Permalink

    Yes with my democratic right I AM STRONGLY opposing ALL THESE AI initiative and has also disliked this video (though I KNOW that NOW it matters too little when the entire Human race is running the RAT race for more POWER, more WEALTH, more DOMINATION on others in this small planet).
    People in the auditorium has burst OUT of excitement without EVEN UNDERSTANDING the consequential THREAT these AI are gradually going to gift MANKIND.
    ☝☝… We are walking steadily fast enough towards our own destruction, the destruction of the human race… 😳☹😠

    Reply
  • July 21, 2019 at 10:03 pm
    Permalink

    When seeing this in full screen, it appears this is a different person – their faces are different and the hologram is missing the zipper at the bottom of her jacket.. The AI may have enhanced the original image or this is something else…

    Reply
  • July 21, 2019 at 10:57 pm
    Permalink

    It would be cheaper to hire a translator!

    Reply
  • July 22, 2019 at 2:21 am
    Permalink

    Woww this is why I wanna explore and study AI Along with my cs

    Reply
  • July 22, 2019 at 6:30 am
    Permalink

    can the audience see the " hologram body" without goggles ?
    this is confusing.
    why is the lady wearing the glasses… cant she see her hologram ?

    Reply
  • July 22, 2019 at 10:32 am
    Permalink

    日本人
    ⬇️

    Reply
  • July 22, 2019 at 10:57 am
    Permalink

    Great. But in the hologram the sweater didn't have the thread hanging below.

    Reply
  • July 22, 2019 at 11:23 am
    Permalink

    Things are happening… looking forward for the X-Men era…

    Reply
  • July 22, 2019 at 1:21 pm
    Permalink

    после этого не надо звать братка чтобы разобрался с гопниками, а будет достаточно включить его голограмму

    Reply
  • July 22, 2019 at 1:30 pm
    Permalink

    The ramifications of this technology on porn will "blow" everyone away!

    Reply
  • July 22, 2019 at 3:14 pm
    Permalink

    Wow this is amazing…..

    Reply
  • July 22, 2019 at 4:59 pm
    Permalink

    Actually, It does not inspire me much. Applications in games is mainly

    Reply
  • July 22, 2019 at 5:02 pm
    Permalink

    This may fool an AI expert but not a physicist.

    This is projected on screen rather than hologram (she mentioned it). It is not physically possible to produce an effective hologram in open space.

    p.s. Light will need a medium to reflect (shine and bounce back from screen, mist, smoke, etc) or refract (absorbed and re-emit at atomic level) which is not possible in open space.

    Reply
  • July 22, 2019 at 6:08 pm
    Permalink

    "Neural TTS" == Neural titties XD

    Reply
  • July 22, 2019 at 7:34 pm
    Permalink

    Fighting Vipers

    Reply
  • July 22, 2019 at 7:54 pm
    Permalink

    Amazing technology … but let's not forget this might have a really serious security impact !

    Reply
  • July 22, 2019 at 8:27 pm
    Permalink

    Impresionante

    Reply
  • July 22, 2019 at 10:28 pm
    Permalink

    This is amazing. Saw this technology live in Microsoft Ready and its spectacular ❤️

    Reply
  • July 22, 2019 at 11:42 pm
    Permalink

    So beautiful and tech improved

    Reply
  • July 23, 2019 at 1:10 am
    Permalink

    感動しました。もうここまで進んでるなんて

    Reply
  • July 23, 2019 at 1:13 am
    Permalink

    I was in first year of english-japanese translation. I think I will quit.

    Reply
  • July 23, 2019 at 6:55 am
    Permalink

    How the hell did they project without a screen? Am i missing something?

    Reply
  • July 23, 2019 at 7:58 am
    Permalink

    The Black Mirror reality just got closer

    Reply
  • July 23, 2019 at 8:41 am
    Permalink

    Ok, so they recorded speaker, they made a script, vfx team created this 3d model, they used TTS to create this speech. Remind me where it becomes microsoft's technology? And why do we need hololens except for seeing a hologram.

    Reply
  • July 23, 2019 at 10:34 am
    Permalink

    It will be definitely needed in India for election campaigns 😅😅 remembering campaigns in Tamil Nadu and Kerala

    Reply
  • July 23, 2019 at 11:18 am
    Permalink

    he is the first HYBRID on Earth. Stop him by ANY MEANS. a lion in musk.
    GOD by the HOLY SPIRIT is way more Powerful and Effective. DON'T EVEN TRY to manipulate U S .

    Reply
  • July 23, 2019 at 12:59 pm
    Permalink

    future is here, imagine book of Acts , Chapter 2:1-11, has the same feeling

    " When the Day of Pentecost had fully come, they were all with one accord in one place. And suddenly there came a sound from heaven, as of a rushing mighty wind, and it filled the whole house where they were sitting. Then there appeared to them divided tongues, as of fire, and one sat upon each of them. And they were all filled with the Holy Spirit and began to speak with other tongues, as the Spirit gave them utterance.

    And there were dwelling in Jerusalem Jews, devout men, from every nation under heaven. And when this sound occurred, the multitude came together, and were confused, because everyone heard them speak in his own language. Then they were all amazed and marveled, saying to one another, “Look, are not all these who speak Galileans? And how is it that we hear, each in our own language in which we were born? Parthians and Medes and Elamites, those dwelling in Mesopotamia, Judea and Cappadocia, Pontus and Asia, Phrygia and Pamphylia, Egypt and the parts of Libya adjoining Cyrene, visitors from Rome, both Jews and proselytes, Cretans and Arabs – we hear them speaking in our own tongues the wonderful works of God.”

    Reply
  • July 23, 2019 at 2:49 pm
    Permalink

    not extra just illusion

    Reply
  • July 23, 2019 at 4:31 pm
    Permalink

    WINTERMUTE IS COMING

    Reply
  • July 23, 2019 at 5:16 pm
    Permalink

    AI might be able to perform certain, limited tasks better than a person can, but there is no logical, philosophical, or biblical reason to think it can be “better” in a meaningful sense. AI might emulate the patterns human beings use when we think, but it can never replace the prowess, dexterity, and creativity of the human mind. Despite fears and speculations, the weight of science, observation, and Scripture refutes the possibility of true artificial intelligence or a technological singularity. In short, the concept of AI makes for entertaining fiction, but not much else.

    Reply
  • July 23, 2019 at 7:09 pm
    Permalink

    my CI, or computer intelligence, is not so fun and easy, but it would do away with most of these irrelevant advances in technology. CI goes directly to automation of everything desirable, so the humans can create the paradise thru cultural pursuits.
    technology has about 50 years of more life span, the minute CI is launched.

    Reply
  • July 23, 2019 at 8:33 pm
    Permalink

    The future is here indeed

    Reply
  • July 23, 2019 at 9:56 pm
    Permalink

    Amazing 😉

    Reply
  • July 23, 2019 at 10:18 pm
    Permalink

    If this is not magic, then what is?
    अगर यह माया नहीं है, तो क्या है?
    இது மாயம் இல்லேன்னா அப்போ எதுதான் மாயம்?

    ❤️

    Thank you. Thank you. Thank you for sharing.

    &

    Reply
  • July 24, 2019 at 3:24 am
    Permalink

    I wish one Japanese person could confirm that this lady's holograph is actually speaking Japanese properly?! 😏 (itboost.com.au)

    Reply
  • July 24, 2019 at 4:24 am
    Permalink

    I hope someone with hologram only can see mixed reality object not the audience otherwise it will be a disaster.

    Reply
  • July 24, 2019 at 5:02 am
    Permalink

    All of this technology only harms the whole of humanity. My opinion here too? Well, the allegedly modern man of the 21st century, should first mature himself mentally and develop mentally to social people before he gets into the hands of technology that he can neither handle nor use this for the benefit of all of humanity on this planet , Any form of technology was and will continue to be used primarily for the destruction of human beings / surveillance … and that is exactly what will always remain so, as long as humanity is mentally at the level of barbaric primeval human beings. I do not oppose the technology, on the contrary … but as long as you have this wonderful technology, not just for the benefit of all
    People instead of suffering, misery and the associated extermination (wars) sets in, I can not express myself positive about modern technology and consider it in the first place always as well as in principle, as misanthropic.
    As long as the destruction of humans, the only starting point for the development of high-quality technology is the driving force, … the human being is not up to his mental maturity to deal with it or to handle it correctly.

    Reply
  • July 24, 2019 at 6:22 am
    Permalink

    It is amazing but not surprising,we thought technology makes our life easy, are you sure ???If Brain doesn,t work imagine what will happen to humans in future ….🤔🧐🤗

    Reply
  • July 24, 2019 at 8:46 am
    Permalink

    Wow

    Reply
  • July 24, 2019 at 8:51 am
    Permalink

    すげぇ、震える。。。

    Reply
  • July 24, 2019 at 8:57 am
    Permalink

    Next step is to match movement (mouth, body and hand movements) with the Japanese text and culture. They recorded the English keynote presentation and therefore mismatching the timing of hand-movements in Japanese. Also, her lips are not in sync with the Japanese text. Impact of a presentation is in the details. Sounds great, but for the looks it needs more work to exactly match all the small nuances' which are so important in communication.

    Reply
  • July 24, 2019 at 9:26 am
    Permalink

    it was not realtime… and it definitely did NOT sound like her

    Reply
  • July 24, 2019 at 1:28 pm
    Permalink

    Promoting amazing technology with a bad audio video 🙏

    Reply
  • July 24, 2019 at 5:10 pm
    Permalink

    I hope criminally minded people will not use this to impersonate innocent persons or create an alibi while they accually commit crimes… Travails of technology

    Reply
  • July 24, 2019 at 6:11 pm
    Permalink

    I like how negative everybody in the comments is about this. 15 years ago this would have been considered magic…

    Reply
  • July 24, 2019 at 8:50 pm
    Permalink

    Orijinal life, natural life please…

    Reply
  • July 24, 2019 at 10:15 pm
    Permalink

    Yeah, I'm calling that was fake–as in you almost certainly prerecorded all of that and it has little to nothing to do with anything than can happen now in real-time on the Hololens as it actually exists. I doubt there was any direct translation going on there at all, and I actually think this was just some Japanese-speaking chick saying the stuff directly reading from some transcript and then you guys pretending that was actually some computer-based speech to translation to speech magic going on.

    Just like those initial bullshots of Hololens when it was first shown off, I''m calling bullsh*t here again.

    Also, it honestly really didn't sound like you at all when it was in Japanese. 😮

    Reply
  • July 25, 2019 at 4:49 am
    Permalink

    Is that real??

    Reply
  • July 25, 2019 at 5:12 am
    Permalink

    So in "reality", you rotoscope a foreground, instead of a background which Hollywood has been doing for decades, and call it AI

    Reply
  • July 25, 2019 at 5:41 am
    Permalink

    Why does the stage look so graphical?

    Reply
  • July 25, 2019 at 1:36 pm
    Permalink

    🛑Tip: This is CGI people
    You can’t see the hologram objects without the glasses.

    Reply
  • July 25, 2019 at 1:37 pm
    Permalink

    What a great time to be here!

    Reply
  • July 25, 2019 at 6:19 pm
    Permalink

    I don't want to be that guy but the hologram did not sound even remotely like her.

    Reply
  • July 25, 2019 at 7:37 pm
    Permalink

    Soo cool!

    Reply
  • July 26, 2019 at 2:37 am
    Permalink

    Its a post produced VFX not a realtime hologram even not VR.

    Reply
  • July 26, 2019 at 4:34 am
    Permalink

    She said it was "rendered here in real time" ?? 03:16

    Reply
  • July 26, 2019 at 8:40 am
    Permalink

    This is not amusing. It is pointless when fundamental needs are becoming challenges and becoming bigger problems. Future generation needs a gift of basic problems being addressed like, water scarcity, food, health, climate change, CO2 emission, forest green cover, dipleating natural resources, terrorism, animal and human rights violations and many more.

    Reply
  • July 26, 2019 at 8:44 am
    Permalink

    Congratulations #Microsoft

    Reply
  • July 29, 2019 at 8:08 am
    Permalink

    real time?, I lost something.

    Reply
  • July 29, 2019 at 8:22 am
    Permalink

    They never showed the demonstrator and the audience in the same frame.
    I will say it's fake.

    Reply
  • July 29, 2019 at 8:50 am
    Permalink

    I guess it doesnt support hair yet

    Reply
  • July 29, 2019 at 9:50 am
    Permalink

    чо то не впечатляет!

    Reply
  • July 30, 2019 at 1:40 am
    Permalink

    "Wtf just happened?"-peter parker

    Reply
  • July 30, 2019 at 7:49 am
    Permalink

    There's something fake about all this!!! Imagine stopping people on street to ask questions, by the time you pull out the lens to put it on the local will be running like hell…😂😂😂

    Reply
  • July 30, 2019 at 10:59 pm
    Permalink

    Fake d+++++

    Reply
  • August 1, 2019 at 1:42 am
    Permalink

    It is just a demo, not for real world.

    Reply
  • August 1, 2019 at 9:02 am
    Permalink

    there's a whole big population that both Google and Microsoft ignore, which Apple takes over gladly, — the Cantonese people lol

    Reply
  • August 1, 2019 at 5:05 pm
    Permalink

    This is so goddamn stupid

    Reply
  • August 1, 2019 at 10:46 pm
    Permalink

    凄いけど。
    違和感しかない。
    やっぱり、日本人が日本語を喋っているようにはならないんですね。

    Reply
  • August 2, 2019 at 1:41 pm
    Permalink

    I am quite surprised to see the ignorance in the comments. I guess it shows the broadening of the audience of Microsoft events by time.

    Reply
  • August 2, 2019 at 10:53 pm
    Permalink

    Wow…really appreciated👏👏👏👏

    Reply
  • August 2, 2019 at 11:22 pm
    Permalink

    Awesome NiCeLy dOnE🎉👏👏👏👏👏👏👏💖🚀

    Reply
  • August 3, 2019 at 6:11 am
    Permalink

    ここまでできるのすごいと思う

    Reply
  • August 3, 2019 at 9:40 pm
    Permalink

    It's a lie and a cartoon

    Reply
  • August 4, 2019 at 10:35 pm
    Permalink

    Somebody call Captain Disillusion!

    Reply
  • August 7, 2019 at 4:18 am
    Permalink

    No mames la inteligencia artificial más cerca en humanos

    Reply
  • August 7, 2019 at 4:29 am
    Permalink

    I just listened: Naruto do you genjutsu kokorono orochimaru. 🀄

    Reply
  • August 8, 2019 at 10:44 pm
    Permalink

    I don't speak Japanese so I can't verify what was being said, but I'd like to hear this in Pidgin English.

    Reply
  • August 8, 2019 at 10:46 pm
    Permalink

    I once thought a mobile phone in a car was fake because it didn't have a cord attached to it. This can only become more realistic in time.

    Reply
  • August 9, 2019 at 3:05 am
    Permalink

    check this A.I. talking (google colab notebook included) https://youtu.be/ly2iLmz3ncA

    Reply
  • August 9, 2019 at 5:34 pm
    Permalink

    Learning and mastering a language or languages will always be of very great value. Language is a fundamental tool of all learning – its acquisition or the transmission of learning. The real value of this technology is the opportunity and mechanism for the rapid and extensive dissemination/exchange of information in multiple languages and places – all at once! But it cannot obviate the requirement for properly calibrated nuance in self-expression, debate and negotiation….in politics, social interaction or business.

    Many countries, including USA, still organise Spelling Bees (Competitions) for school children despite the availability of spelling- and grammar-checking software…for good reasons!

    In short, the old statement remains true regarding computers…"Garbage in, garbage out". Meanwhile, how FAITHFULLY does the hologram generate and synchronise the hand gesticulations and facial expressions with the what is being said across languages and cultures?

    In summary, these technologies are very welcome and are to be celebrated. But they cannot replace learning and hard work.

    Reply
  • August 9, 2019 at 7:53 pm
    Permalink

    I believe the VR/AI technology is able to deliver nearly real-time (simultaneous) interpreting in foreigner language now, and might do better job than a blood-flesh trained professional (linguistics expert/bilingual-interpreter).

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *