According to an experiment, Facebook is trying to introduce sound in their AR glasses. Michael Abrash, Chief scientist and his team at FRL Research, published details about what they call “perceptual superpowers”. It is an augmented reality system that figures out what users are trying to hear, then amplify it and reduces background noise. Coupled with spatial audio features, the device generates the aural equivalent of a hologram overlay in the AR glasses.
The augmented reality glasses can perform this by using contextual clues to measure which sounds are necessary, picking up audio with microphones and serving those sounds through a noise-canceling earpiece.
According to Facebook, the “perceptual superpowers” pitch is similar to hearing aids’ function, which magnifies sound and reduces background noise.
Facebook published a photo of a model in-ear monitor on a mannequin head and a picture of team members using headphones. The Oculus Quest VR headset employs designated speakers, a handy one-size-fits-all solution for propelling sound into an environment. “What form factor to use to solve a problem ultimately depends on the application. The work that we are doing and the work that we presented today definitely has applications for our VR line of devices” – FRL Research audio lead Ravish Mehra.
There is also no evidence that Facebook’s FRL Research is considering deliberate surveillance. FRL Research states that the audio being captured is encrypted with limited access.
According to Abrash Facebook is trying to create privacy into all aspects of its AR glasses design. The system could ask for approval from someone else’s AR glasses before magnifying the sound, or it may have limited range. “Rather than thinking of this as a magic flashlight we can point at anybody and hear what they’re saying, think about it more as people being able to participate in the conversation they’re in anyway,” – Abrash.
For now, according to Abrash, the small microphone arrays are not powerful enough to capture long-range sound. “If you ever see anyone walking with glasses that are two feet wide,” he cautioned jokingly, “you should be suspicious,” – Abrash.
