What the Listener can do
The Listener is a speaking-time display for meetings and group conversations. The current prototype in the black housing listens with four microphones, recognizes voices and the direction a voice comes from, and shows who speaks how much.
Take the survey (German, about 3 minutes)The benefits at a glance
Speaking time becomes visible
Everyone sees how speaking time is distributed. Nobody has to point out that some people say a lot or a little.
More balance in conversation
Dominant speakers notice by themselves, and quieter people find room more easily. That helps in meetings as much as in private groups.
Simply put it on the table
No headset, no clip-on microphone: the device stands in the middle and listens to the whole group.
Voice recognition in the device
A local neural network recognizes the voices inside the Listener itself, not in a cloud.
The features
Four microphones
Four microphones in a square pick up the sound around the device. From the tiny differences in arrival time between the microphones, the direction of the speaker can be calculated.
Direction of the speaker
The Listener determines on which side of the table someone is speaking, so speaking time can also be assigned to a seat.
In first tests in a quiet room, with four tested directions, the displayed direction was at most about 10 degrees off.
Recognizing speakers
A neural network compares each utterance with the trained voice profiles and assigns the speaking time to the right person. It runs locally in the device and needs only a few tenths of a second per utterance.
Noise suppression
The Listener learns steady sounds such as fans, air conditioning or traffic noise as background and removes them, so they do not count as speech.
Detecting background music
Music usually has a steady beat, speech does not. The Listener analyzes the sound for regular rhythm and so recognizes background music. Later, it should not count as speaking time, for example at a regulars' table in a pub.
At the moment, the beat is detected and displayed. Hiding the music is not switched on yet (see status of development).
How the neural network and the direction finding work in detail is explained on the page Technology.
What the operation looks like
The prototype currently runs on a development board (ESP32-P4 with touch display). Housing and design are not final. The display has two views: Speaking time and Live.
Speaking time
Shows how long the measurement has run, how much of it was speech, and how speaking time is split between people: as a ring chart and as a table with contributions, average length, share and time.
The Listener also names who has the largest share and compares it with the “fair” share. People who have spoken very little are marked. “Reset times” starts a new round.
Live
Shows who is speaking right now and how they were recognized, the seating plan with direction, and the level of the four microphones.
Here you can also enroll voices, calibrate the device (with a voice from the front it learns where the front is) and relearn the noise profile so background sound is filtered out.
The screenshots are photos of the display; the names are made up.
Status of development
The Listener is a prototype. “Works” here means: the function works in the prototype. This way you can judge what already functions.
| Function | Status |
|---|---|
| Recognizing speakers (local neural network) | works |
| Four microphones and direction of the speaker | works first tests in a quiet room |
| Noise suppression | works |
| Detecting background music (beat analysis) | in testing The beat is detected and displayed; hiding is not active yet. Tests in a real environment are still to come. |
| Operation and display of speaking time | works on the display of the development board and through a web interface in the Wi-Fi |
Not a surveillance device
The Listener is meant to be a mirror, not a supervisor. Voice recognition runs inside the device itself, not in a cloud. The Listener does not analyze content. The plan is that, in normal operation, no conversations are stored or sent to third parties. For testing, the prototype has a recording function (5 seconds) that is triggered by hand. Voice profiles are only created after deliberate enrollment. Anyone using the Listener should inform everyone present beforehand. More in the privacy policy (German).
Frequently asked questions
Why four microphones instead of two?
With two microphones, front and back are hard to tell apart. With four microphones, the Listener can determine the direction all around the device.
Does the Listener work with background music?
That is the goal, and we are working on it. So far the Listener detects the beat of music and displays it. Hiding it is not switched on yet, and whether it works reliably in a real environment has not been shown yet.
Does the Listener recognize what is said?
No. It measures who speaks for how long. It does not analyze the content of conversations.
Is the Listener available yet?
No, so far there is a prototype. The survey helps us find out which features matter and what a device may cost.
Which features matter to you?
Your answers help us decide what the Listener should be able to do and what it may cost.
Take the survey (German)