‘Don’t make me say it twice!’

FAU researchers want to improve hearing in humanoid robots

In science fiction films, they are already taking over the world from humans – in the reality of the 21st century, they could at least come as far as being useful helpers: humanoid robots are expected to conquer more and more aspects of daily life and to take over tasks for humans. In the EARS project, FAU researchers work together with colleagues from Be’er Sheva, Paris, London, Berlin and Grenoble to teach robots to hear – even in loud environments. The researchers are using a small humanoid robot named NAO built by the French company Aldebaran Robotics as a test platform and developing new algorithms that will allow it to better understand acoustic signals. The EU is supporting the project with 3.52 million euros over the course of 3 years.

They have eyes and ears and can, of course, see and hear: cameras and microphones on head and limbs make humanoid robots astonishingly perceptive already today. However, in order to be able to use our artificial friends in public places, for instance in a hotel lobby, there are still a few technical challenges to overcome so that proper communication between human and machine is ensured.

Even though automatic voice recognition is already quite advanced, it usually only works perfectly in quiet environments. In public, though, sources of acoustic interference often prevent the humanoid helpers from understanding their handlers. In large spaces, speech may be lost to reverberations, the person may be too far away, or voices, machines and street noises interfere with the signal.

This is why researchers led by Prof. Dr. Walter Kellermann at FAU’s Department of Multimedia Communications and Signal Processing, which co-ordinates the EU project, are researching better methods of freeing the acoustic signals from interferences and recognising the speech signals they contain. The robot will, for example, record its own speech when it talks, and the volume will usually be louder than that of the human voice it is supposed to recognise. This problem can be solved with acoustic echo compensation. The robot can use this to filter out its own speech. Little NAO will also learn to localise and identify individual speakers and their words in several languages through its internal microphones..

Even if the project is mostly concerned with acoustic signals, the researchers are also examining how the robot’s motor functions can be combined with visual and auditive information. The robot’s facial recognition must likewise be optimised for it to be able to locate a human even when they aren’t speaking – for instance to locate who spoke to it and respond with gestures. For this reason, the researchers redesigned the robot’s head in the course of the project, allowing it to hear and see better, and thus also to understand and react better.

Prof. Kellermann sees great potential for the possible uses of such a robot: ‘Sure, a robot won’t be able to replace a nurse or concierge any time soon. But even with abilities such as understanding simple commands or answering simple questions, it can, for example, provide information in a hotel lobby or shopping centre. There would also be other sensible uses such as household companions for elderly people.’ However, the most important thing to Prof. Kellermann is that the algorithms the researchers developed for the robot NAO can also be used in many other fields.

‘Of course the project is basically about perfecting a commercial product. But the greater aim is to keep improving the competitive economic situation of the EU in this field of research and to expand the market for humanoid robots,’ says Kellermann.

The EARS project (Embodied Audition for RobotS) has a duration of three years. The EU funding of 3.52 million euros enables the financing of twelve junior researchers at six research institutions. The funding was granted to the FAU researchers as leaders of a consortium with the Ben-Gurion University of the Negev in Be’er Sheva (Israel), the Imperial College in London, Humboldt Universität Berlin, INRIA Grenoble and the industry partner Aldebaran, S.A. from Paris, in a competition with about 150 entries from different European research institutions.

Further information:

Dr. Heinrich Löllmann
Phone: +49 (0)9131 85 20173
loellmann@lnt.de