{"id":2542,"date":"2020-09-03T18:18:00","date_gmt":"2020-09-03T18:18:00","guid":{"rendered":"https:\/\/techclot.com\/index.php\/2020\/09\/03\/facebook-wants-its-ar-glasses-to-give-wearers-superhearing\/"},"modified":"2020-09-03T18:18:00","modified_gmt":"2020-09-03T18:18:00","slug":"facebook-wants-its-ar-glasses-to-give-wearers-superhearing","status":"publish","type":"post","link":"https:\/\/techclot.com\/index.php\/2020\/09\/03\/facebook-wants-its-ar-glasses-to-give-wearers-superhearing\/","title":{"rendered":"Facebook wants its AR glasses to give wearers superhearing"},"content":{"rendered":"

Facebook wants its AR glasses to give wearers superhearing<\/a><\/p>\n

A new Facebook experiment shows how sound could play a major role in augmented reality. Chief scientist Michael Abrash and his team at FRL Research (formerly Facebook Reality Labs) released details today<\/a> about what the team calls \u201cperceptual superpowers\u201d \u2014 AR systems that figure out what you\u2019re trying to hear, then amplify it and dampen background noise. Combined with spatial audio features, the device creates the aural equivalent of a hologram overlay in a pair of glasses.<\/p>\n

Facebook has pursued high-quality virtual sound for years, largely through its Oculus virtual reality headsets. FRL Research\u2019s latest work focuses on AR applications. To give one example, \u201cimagine being able to hold a conversation in a crowded restaurant or bar without having to raise your voice to be heard or straining to understand what others are saying,\u201d the company explains.<\/p>\n

AR glasses could do this by picking up audio with microphones, using contextual clues to gauge which sounds are important, and feeding those sounds through a noise-canceling earpiece. Conversely, if you\u2019re on a phone or video call, improved spatial sound could project participants\u2019 voices or other audio to specific parts of the room, increasing the sense that you\u2019re really with somebody else \u2014 or \u201caudio presence,\u201d in FRL Research\u2019s terms.<\/p>\n

\n <\/p>\n

<\/p>\n\"Facebook<\/p>\n<\/picture>\n

<\/span><\/p>\n

<\/span><\/p>\n<\/figure>\n

As Facebook acknowledges, the lab\u2019s \u201cperceptual superpowers\u201d pitch is very similar to the function of existing hearing aids, which also amplify sound and reduce background noise. (One experimental system even uses brain implants<\/a> to focus on specific voices.) <\/p>\n

AR glasses, however, present unique opportunities for people with and<\/em> without impaired hearing. Directional sensors and outward-facing or eye-tracking cameras can collect detailed contextual information, including your bodily orientation and where your gaze is focused. That helps the earpieces pinpoint an \u201cacoustic spotlight\u201d to amplify. And, of course, an AR system can mix its own soundscape with the amplified audio.<\/p>\n

Facebook could also complement the system with its other AR initiatives. LiveMaps<\/a>, for instance, aims to create rich maps full of information about people\u2019s surroundings. If LiveMaps detects that someone is wearing glasses in a restaurant, the glasses could do something like automatically flag and cancel clinking silverware sounds.<\/p>\n

Facebook released a photo of a prototype in-ear monitor on a dummy head as well as a picture of team members wearing headphones that look more like Valve\u2019s off-ear Index speakers<\/a>. The Oculus Quest VR headset uses directed speakers, which are a convenient one-size-fits-all solution for projecting sound into an environment, but leak sound to bystanders \u2014 while in-ear monitors may be better for changing overall sound levels while wearing AR glasses in public. \u201cWhat form factor to use to solve a problem ultimately depends on the application,\u201d says FRL Research audio lead Ravish Mehra.<\/p>\n

Other companies have experimented with aural augmentation \u2014 most prominently Bose, which revealed audio-only AR glasses<\/a> in 2018. Bose\u2019s glasses used spatial sensors to detect where wearers were looking and offer features like virtual street tours. The company failed to make them commercially viable, though, and it abandoned the project<\/a> this June.<\/p>\n

Unlike in previous years, Facebook isn\u2019t holding conferences where it can show off experimental tech in 2020, so we haven\u2019t heard its audio in person. FRL Research offered reporters a demonstration of its filtering system on a remote call. And any spatial audio improvements could soon be applied to VR headsets, including a rumored Oculus Quest update<\/a> around this month\u2019s Facebook Connect conference. \u201cThe work that we are doing and the work that we presented today definitely has applications for our VR line of devices,\u201d says Mehra.<\/p>\n

\n <\/p>\n

<\/p>\n\"Facebook<\/p>\n<\/picture>\n

<\/span><\/p>\n

<\/span><\/p>\n<\/figure>\n

A full application of these capabilities could be years away. We don\u2019t know much about Facebook\u2019s roadmap for consumer AR glasses, even if we\u2019re slowly learning more about what the company thinks they ought to do. A couple of pictures show plastic frames, but they\u2019re designed to hold the microphone array, not work as full-fledged glasses. Facebook has previously said<\/a> it\u2019s built multiple prototype variants, including a recent combined AR \/ VR headset design<\/a> that looks like a pair of sunglasses.<\/p>\n

To sell a real commercial product, Facebook will need to allay concerns that go beyond the technical. That includes convincing people that \u201cperceptual superpowers\u201d don\u2019t pose a privacy threat.<\/p>\n

Without any limitations, sufficiently powerful microphones and filtering tech could let people imperceptibly eavesdrop on conversations across a public space. AI audio analysis presents even weirder and more alarming possibilities, like the ability to flag specific voices or conversation keywords in a crowded room. (It could also theoretically perform more innocuous, helpful tasks like real-time translation<\/a>.) And, of course, the glasses would be recording your own conversations as well.<\/p>\n

There\u2019s no indication that FRL Research is considering deliberate surveillance. But many people already fear \u2014 probably incorrectly<\/a> \u2014 that Facebook\u2019s apps are recording them to target ads. So the mere possibility may alarm consumers. FRL Research says that captured audio is currently encrypted and stored on servers that a small number of researchers can access.<\/p>\n

Abrash says that Facebook is attempting to build privacy into all aspects of its glasses design, echoing a refrain about its overall AR \/ VR plans<\/a>. Among other things, the system could ask for permission from someone else\u2019s glasses before amplifying a conversation, or it could have a limited range. \u201cRather than thinking of this as a magic flashlight we can point at anybody and hear what they\u2019re saying, think about it more as people being able to participate in the conversation they\u2019re in anyway,\u201d he told reporters.<\/p>\n

And for now, Abrash says, the tiny microphone arrays aren\u2019t powerful enough to conduct long-range spying. \u201cIf you ever see anyone walking with glasses that are two feet wide,\u201d he cautioned jokingly, \u201cyou should be suspicious.\u201d<\/p>\n<\/p>\n

Published at Thu, 03 Sep 2020 16:52:30 +0000<\/p>\n","protected":false},"excerpt":{"rendered":"

Facebook wants its AR glasses to give wearers superhearing A new Facebook experiment shows how…<\/p>\n","protected":false},"author":12,"featured_media":2543,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[61],"tags":[],"class_list":["post-2542","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-augmented-virtual-reality"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/techclot.com\/wp-content\/uploads\/2020\/09\/FRL_Research_Audio___Demo_2__2_.jpg?fit=1200%2C800&ssl=1","jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p3orZX-F0","jetpack-related-posts":[],"_links":{"self":[{"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/posts\/2542","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/comments?post=2542"}],"version-history":[{"count":0,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/posts\/2542\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/media\/2543"}],"wp:attachment":[{"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/media?parent=2542"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/categories?post=2542"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/techclot.com\/index.php\/wp-json\/wp\/v2\/tags?post=2542"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}