Tech giant Microsoft has developed a novel Artificial Intelligence-based technology that helps blind children and young people better understand their immediate social environment, interact with peers and gain confidence more easily.
For children born blind, social interaction can be particularly challenging as they have difficulty aiming their voices at the person they’re talking to. As a result, they struggle with engaging and making friends with their peers, which can be frustrating for the child and for their support network of family members and teachers who want to help.
Called PeopleLens, the project helps blind children to make friends; and for teachers and parents, it’s a way for these children and young people to develop competence and confidence in social interaction.
PeopleLens is a head-worn device that reads aloud in spatialised audio the names of known individuals when the learner looks at them. It means the sound is coming from the direction of the person, assisting the learner in understanding both the relative position and distance of their peers.
“The PeopleLens helps learners build a People Map, a mental map of those around them needed to effectively signal communicative intent,” Cecily Morrison, Principal Research Manager, at Microsoft’s Research Lab in Cambridge, along with other authors, wrote in a blog post.
“The technology, in turn, indicates to the learner’s peers when the peers have been seen and can interact – a replacement for the eye contact that usually initiates the interaction between people,” Morrison added.
PeopleLens is a sophisticated AI prototype system that works on augmented reality glasses tethered to a phone. The system can recognize only those faces that have been registered in the system. To register, people must take several photographs of themselves with the phone attached to the PeopleLens.
The system then receives images and processes them with computer vision algorithms to continuously locate, identify, track, and capture the gaze directions of people in the vicinity.
Further, the system also employs a series of sounds to assist the wearer in placing people in the surrounding space: a percussive bump indicates when their gaze has crossed a person up to 10 meters away. The bump is followed by the person’s name if the person is registered in the system, is within 4 meters of the wearer, and both the person’s ears can be detected.
The functions of the PeopleLens system include experiencing features such as recognizing a person in front of the wearer, attention notifications from the direction of those who look at the wearer, the ability to follow someone, and an orientation guide to help wearers find people and faces.
While PeopleLens is not yet commercially available, Microsoft is recruiting learners in the UK aged 5 to 11 who have the support of a teacher to explore the technology as part of a multistage research study. For the study, led by the University of Bristol, learners will be asked to use the PeopleLens for a three-month period beginning in September 2022.
