Live from the lab: this page refreshes itself every hour with the latest numbers from our training computer. Last update: November 1, 2026 at 9:45 AM (Pacific time). Training runs around the clock, so check back to watch it move.
CymaticBridge
The future of robot hearing
Bio-inspired cochlea model • Trained listening network • Fold-by-fold validation
EAR LABStage 1
Connecting
An experiment in robot hearing: can a machine that listens through a model of the human ear learn to hear like we do?
Model scores
Clean-audio accuracy, mean of 5 folds x 2 seeds
Experiment progress
0%Overall completion
Each finished training run, in order
Cochlear
transductionCARFAC turns sound into the firing pattern of the inner ear
transductionCARFAC turns sound into the firing pattern of the inner ear
Architecture overview
From sound to answer, the path every clip takes
Sound inESC-10 clips, levelled, at –
Cochlea or melCARFAC ear model, or a mel spectrogram (control)
Ear features10 ms frames, 5 ms frames, or + pitch (SAI)
Listening networkSame small network for every ear; 10 sound classes
Noise test
Accuracy as pink noise gets louder (clean → 0 dB)
Appears when the noise test finishes.
Fold-by-fold validation
Test accuracy per fold, average of 2 seeds; best in each row glows
Noise test
Trained on clean audio, tested on fold 5 with pink noise; in brackets the change from that ear's clean score
One dog bark, both ears
Latest log
Cybernetic learning pipeline // Stage 2
Two listening models trained without labels on ~110 h of sound and speech: one hears through the cochlea model, one through mel. Then the test. Every number here is live from the PC.
Ear pass
Every clip turned into ear pictures, both ears
Training clips finished over time (one point a minute)
Machine
Live load, refreshed every minute
Training
Masked-autoencoder pre-training, same model, steps and seed for both ears; lower loss means it rebuilds hidden sound better
Run 1 · the test
Run 1 (17 to 18 October). Frozen encoders, one linear layer on top: ESC-50, ESC-10 and pink noise; untrained copies show what training added
Runs 1 to 5: what I learned, what runs next
Every Stage 2 training run and test is logged here: each run's result, the quick screens that explain Run 1, and Run 15's result, the latest
Run 1 verdict
Mel won: 82.7% against 69.9% on ESC-50, and it stayed ahead in noise. Both models learned (untrained copies score 26 to 33%), so the training worked; the cochlea input was the weak link.
Why the cochlea lost
Quick screens scored eleven ways of reading the ear in about 15 minutes. Run 1's cochlea input squashed the nerve signal with a log curve that buries quiet detail. With the ear's own cube-root compression the same signal already ties mel, and beats it in noise.
Closer to the real nerve
The real auditory nerve has three kinds of fibre: very sensitive, medium and hard to excite. CARFAC can model all three. Together with a first "brainstem" step (how fast each band's loudness flickers), this was the best input of all.
| Quick screen (simple scorer, not a trained model) | ESC-50 | Pink 20 dB | Pink 10 dB | Pink 0 dB |
|---|---|---|---|---|
| Mel, as in Run 1 | 53.7% | 37.8% | 25.5% | 6.2% |
| Cochlea, as in Run 1 | 50.1% | 36.8% | 24.0% | 9.8% |
| Nerve, cube-root compression | 54.5% | 44.8% | 32.0% | 7.2% |
| Three fibre types | 56.9% | 46.5% | 35.0% | 9.0% |
| Mel + cube-root + brainstem (fair control) | 58.9% | 52.2% | 42.0% | 12.2% |
| Three fibres + brainstem | 62.4% | 56.2% | 39.5% | 11.5% |
Run 2 resultMel won again: 82.1% against 58.2% on ESC-50 (ESC-10: 93.3% against 84.0%). Both started level untrained (about 36%), but the fibre model's training loss began near 47 against mel's 1: short clips were padded with a "silence" value the sensitive fibres never produce, and the model spent its training rebuilding that padding.
Run 3 resultThe padding fix worked: the three-fibre ear jumped from 58.2% to 76.7% on ESC-50 (ESC-10: 92.3%), beating Run 1's cochlea (69.9%) and closing most of the gap. Mel still leads: 82.6% (ESC-10 93.5%), and it stays ahead in noise (pink 0 dB: 29.5% against 19.5%). The gap went from 24 points to 6.
Run 4 resultFine timing did not survive full training. The fibres-plus-timing ear scored 72.3% on ESC-50 (ESC-10: 89.3%), below Run 3's fibres alone (76.7%) and mel (82.6%). Training was the same as Run 3 (60,000 steps, same model), so the four timing channels cost about 4 points, even though the short screen had shown them helping. In noise it held level with Run 3 (pink 0 dB: 20.8% against 19.5%); mel stays ahead there (29.5%). Untrained, it started at 40.0%. Lesson: the short screens do not predict full runs well enough. Later reading suggests why timing did not help here: in MIT's 2024 study, fine nerve timing mattered most for recognising voices and locating sounds, and far less for identifying everyday sounds like these.
Run 5 resultNoise-proof training lifted both ears in heavy noise, but mel far more. Both trained on every clip with a noisy twin, learning to rebuild the clean sound. At pink 0 dB the ear rose from 19.5% (Run 3) to 29.5%, while mel jumped from 29.5% to 56.3%, so the gap in heavy noise grew from 10 points to 27. On clean sound both stayed about level: ear 75.9% on ESC-50 (ESC-10: 89.8%) against mel 81.9% (ESC-10: 93.3%). Noise is still mel's strongest ground, and this kind of noise training does not favour the ear. Voice tests: level again (emotion, CREMA-D: ear 59.6% against mel 59.9%; RAVDESS: 58.5% against 58.8%; who is speaking: 98.9% against 99.2%).
Run 6 resultKeeping the nerve's fine timing closed the gap. Runs 1 to 5 averaged the ear's signal into 10 ms steps; Run 6 ran a model cochlea live on the graphics card at 4,410 readings a second, and a small learned timing reader passed that detail to Run 3's AI. Mel got the identical reader. On everyday sounds the ear rose from 75.9% (Run 5) to 82.7% on ESC-50, level with mel's 82.5% for the first time (the 0.2-point difference is within the spread between test folds, so it is a tie, not a win). ESC-10: ear 94.8% against mel 93.2%. Voices, my real goal, now lean to the ear: emotion, RAVDESS 60.7% against 56.8% (about 4 points, larger than the fold-to-fold spread); CREMA-D 62.0% against 60.5%; who is speaking 99.7% against 99.3%. Noise stays even: pink 0 dB 27.5% against 26.8%, pink 20 dB 66.0% against 67.5%. Finished 21 October at 17:57.
Run 7 result: the midbrain, first tryI added a model of the inferior colliculus, the midbrain hub whose cells are tuned to how fast sound rises and falls (rhythm, syllables, roughness, voice pitch): eight modulation bands from 3 to 384 times a second, written from the published research (Dau 1997, Carney's midbrain models), with mel getting the identical stage. The ear still edged mel on most tests, but both AIs scored a little below Run 6. Everyday sounds: ear 82.2% against mel 81.6% on ESC-50 (ESC-10: 93.3% against 94.0%). Emotion: RAVDESS 59.0% against 56.8%; CREMA-D 59.3% against 59.2%, now a tie. Who is speaking: 99.5% against 99.1%. In heavy noise mel pulled ahead (pink 0 dB: 23.0% against 26.8%). Untrained, the ear started well ahead (52.2% against 49.2% on ESC-50; 52.5% against 43.2% on clean fold 5), so its built-in shape helps from the start. Why it slipped: checking the code showed Run 7 replaced Run 6's learned timing reader with the fixed midbrain bands, so it changed two things at once and lost the part that won Run 6. Finished 22 October at 10:41.
Run 8 result: stacking did not helpRun 8 kept everything from Run 6 (the live cochlea and the learned timing reader) and added Run 7's eight midbrain bands beside it, with mel getting the identical treatment. Mel won on everyday sounds: 81.8% against 80.0% on ESC-50 (ESC-10: 95.5% against 94.0%), and in very heavy noise (pink 0 dB: 27.0% against 23.0%). The ear won on emotion in voices (RAVDESS: 57.4% against 55.1%) and in light noise (pink 20 dB: 64.3% against 62.5%). CREMA-D was about even (59.3% against 59.9%) and who is speaking a tie (99.2% against 99.1%). Training clearly worked (untrained copies scored about 49 to 51% on ESC-50). The lesson: stacking the midbrain on the timing reader left the ear below Run 6 (82.7% on ESC-50, 60.7% on RAVDESS), so the fixed midbrain bands are not helping yet. Under the rule I set before the results, Run 9 goes ahead without them, and the midbrain returns in Run 10 as a stage that learns. Finished 23 October at 15:46.
Run 9 result: the ear wins again, with its best score yetRun 9 put a model outer and middle ear in front of Run 6's live cochlea (the ear canal's boost around 2.7 kHz, the concha's near 5 kHz, the middle ear's best band from about 0.8 to 4 kHz) and added three cell types of the cochlear nucleus, the brainstem's first stop: bushy cells (sharper timing), stellate cells (a steady spectrum for noise) and octopus cells (the starts of sounds). Mel got the identical outer ear and the same three formulas. The ear won on everyday sounds: 83.0% against 81.2% on ESC-50, my ear's best so far (Run 6: 82.7%), and 93.5% against 93.2% on ESC-10. It won on emotion in voices (RAVDESS: 59.7% against 55.9%; CREMA-D: 62.1% against 61.2%), on who is speaking (99.5% against 99.0%), and in light and medium noise (pink 20 dB: 70.2% against 68.0%; pink 10 dB: 52.5% against 50.5%). Mel kept a small edge only in very heavy noise (pink 0 dB: 25.5% against 24.2%). Noise is where the new stages showed most: in light noise my ear rose from 64.3% in Run 8 to 70.2%. Training clearly worked (untrained copies scored 37.1% and 44.5% on ESC-50). Mel's training overflowed the graphics card's memory and was restarted once at 4:29 am on 24 October with the same batch in smaller chunks (identical maths). Finished 24 October at 9:05.
Run 10 result: spoiled by a leak, so no verdictBoth AIs scored lower than in Run 9: my ear 76.4% on ESC-50 (Run 9: 83.0%) and mel 76.4% (Run 9: 81.2%). Voices and noise fell too (emotion, RAVDESS: ear 57.6% against mel 55.3%; CREMA-D: 58.3% against 58.1%; who is speaking: 98.7% against 98.5%; pink 0 dB: 17.3% against 19.5%). The cause was my mistake, not the midbrain. The AIs learn by rebuilding pieces of sound I hide from them, 80% of every clip. The new midbrain listened to the whole clip, a quarter of a second at a time, before those pieces were hidden, so every piece the AI could see carried clues about its hidden neighbours. The AIs copied the clues instead of learning to hear: their training error fell to less than half of Run 9's while their test scores dropped. Both sides had the same leak, so the near-tie says nothing about my ear against mel, and nothing about the midbrain. Run 11 repeats Run 10 with the leak fixed.
Run 11 result: Run 10 done right, and the ear winsWith the leak fixed, the midbrain only heard the parts of each clip the AI was allowed to see (a 45-minute check first confirmed it: training error at step 5,000 was 0.136, the same as Run 9). My ear beat mel on nearly every test: everyday sounds 82.9% against 82.0% on ESC-50 (ESC-10: a tie at 93.2%); emotion in voices, RAVDESS 60.6% against 58.5% and CREMA-D 62.1% against 61.6%; who is speaking 99.5% against 99.0%; and in noise at every level, even the heaviest, where mel had led in Run 9 (pink 20 dB: 68.5% against 66.8%; pink 10 dB: 51.2% against 49.0%; pink 0 dB: 24.0% against 23.0%). What the learned midbrain did: my ear stayed about where Run 9 left it (83.0% on everyday sounds; RAVDESS rose from 59.7% to 60.6%), while mel gained more (81.2% to 82.0%; RAVDESS 55.9% to 58.5%). So the midbrain mostly gave mel the rhythm processing my ear's own stages already provide, and my lead narrowed but held. Finished 26 October at 04:17.
Run 12 result: hearing in noise, a mixed resultRun 11's whole ear plus two of a real ear's defences against noise, both given to mel too: the middle-ear reflex (loud sounds turn the low frequencies down, by up to 15 dB) and an automatic gain on every channel, modelled on the feedback nerves to the cochlea (each channel turns itself down by its own running level, so steady background noise fades). My ear set its best score yet on everyday sounds, 83.1% against mel's 82.9% on ESC-50 (ESC-10: 95.0% against 93.0%), and its lead in the heaviest noise grew: pink 0 dB 25.3% against 22.3% (Run 11: 24.0% against 23.0%). It still led at pink 20 dB (69.0% against 67.0%) and just led at pink 10 dB (49.8% against 49.5%). But the noise stage cost my ear on emotion in voices: RAVDESS fell from 60.6% to 57.3% (mel 56.4%), and CREMA-D went to mel by a hair (61.8% against 61.6%); who is speaking: 99.4% against 99.2%. And mel gained more from the same stage on everyday sounds (82.0% to 82.9%), so the gap there narrowed. My reading: an automatic gain that turns steady sound down also flattens the slow loudness changes that carry emotion in a voice. Finished 26 October at 23:47.
Run 13 result: two ears, a split decisionEach training sound came from somewhere around a real measured head, mostly in rooms with echoes, and the superior olive compared the two ears (mel got two ears and the loudness difference, but no fine timing). On the usual tests, with the sound straight ahead, it was nearly a tie: everyday sounds 80.9% against mel's 80.8% on ESC-50 (ESC-10: 93.0% against 93.2%). My ear led on voices: emotion, RAVDESS 61.2% against 58.5% (my ear's best yet) and CREMA-D 61.8% against 61.1%; who is speaking 99.1% against 98.5%. Mel led in noise this time: pink 10 dB 53.0% against 51.2%, pink 0 dB 30.5% against 27.0% (my ear still led at pink 20 dB, 68.8% against 67.0%). Both scored about 2 points lower on everyday sounds than in Run 12, the price of learning from sounds that came from every direction and bounced around rooms. On the new two-ear tests, through four heads the AI had never heard, my ear pointed to the right one of 12 directions 94.8% of the time in the open (3.7 degrees off on average) against mel's 94.0% (4.5 degrees), and 65.5% against 62.3% in a room. The clearest win: picking out a sound in a crowd. When the sound came from the side instead of the same place as the crowd, my ear lost nothing (24.8% to 24.2%) while mel fell from 22.2% to 16.2%. One surprise: before any training, my ear's timing map pointed to the right direction more often (98.0% in the open, 77.6% in a room) than after it. The way the AIs learn, rebuilding hidden pieces of sound, does not reward knowing where a sound is, so training wore some of it away. Finished 28 October at 15:52.
Run 15 result: the best everyday-sound score yet, and the ear is ahead by a noseThree jobs at once in noisy rooms (where a sound is, at less than half Run 14's weight; what it is; who is speaking) gave both AIs their best everyday-sound scores so far: 84.7% for my ear against 84.1% for mel (Run 14: 75.1% and 73.0%; Run 13: 80.9% and 80.8%). Direction stayed nearly perfect: 99.8% in the open and 96.0% in rooms for my ear, 100% and 95.9% for mel. Both named the speaker almost every time (100% and 99.9%). My ear led on emotion in voices (59.6% against 59.0% on CREMA-D, 53.9% against 53.3% on RAVDESS) and on picking a sound out of a crowd in the same place (32.3% against 30.5%, up from 18.0% in Run 14); mel led in the heaviest noise (30.5% against 28.0% at 0 dB) and on a crowd coming from the side (27.5% against 22.5%). The gaps are small, and emotion is still below Run 13 (RAVDESS 61.2%), even though neither AI is ever trained on it. The lesson: several jobs together help, as long as none takes over. Run 16 keeps these three jobs and adds the thalamus and the cortex.
Run 14 result: direction solved, everything else droppedWith one extra question during training (which of 12 directions did this sound come from?), both AIs learned direction almost perfectly: my ear found the right direction 100% of the time in the open and 99.1% of the time in rooms (Run 13: 94.8% and 65.5%), mel 100% and 98.6%. But the question pulled both AIs away from everything else. Everyday sounds fell to 75.1% for my ear against 73.0% for mel (Run 13: 80.9% and 80.8%), emotion in voices to 56.5% against 53.2% (RAVDESS; Run 13: 61.2% and 58.5%), and picking a sound out of a crowd got worse. My ear still led mel on sounds, emotion and speakers. The lesson: a job that is easy and weighted too heavily takes over what the AI learns. Run 15 keeps the question, at less than half the weight.
The build plan from hereI now build up the pathway in the order a real ear does. Run 9 (done, 24 October; the ear won): the outer ear and canal (the boost around 3 kHz) and the middle ear at the front, then the cochlear nucleus, the brainstem's first stop, which splits the nerve into streams: bushy cells sharpen timing, stellate cells keep a noise-proof picture of the spectrum, octopus cells mark onsets. Run 10 (24 October): a midbrain that learns, fed from the cochlear nucleus as in a real brain; a leak in my code spoiled it, so Run 11 (done, 26 October; the ear won) repeated it with the leak fixed. Run 12 (done, 27 October; mixed): noise, with the middle-ear reflex and an automatic gain modelled on the feedback nerves that turn down background noise; it helped in the heaviest noise and cost some emotion. Run 13 (done, 28 October; a split decision): two ears in virtual rooms, through the head and ear shapes of real people, with the superior olive comparing the two ears; my ear led on voices, on where a sound is and on picking a sound out of a crowd, mel led in heavy noise. Run 14 (done, 30 October): Run 13 plus a direction question during training; direction became nearly perfect, but sounds and voices dropped. Run 15 (done, 31 October; the ear ahead by a nose): many jobs at once in noisy rooms, with the direction question at less than half the weight, plus naming the everyday sound (200 labels) and who is speaking (251 voices); everyday sounds reached 84.7%, the best yet, and direction stayed nearly perfect. Now, Run 16 (from 31 October, with the same three jobs): the thalamus and the cortex, the last two stages on the map. The cortex becomes three tiers that listen over longer and longer stretches (about 0.3 s, 0.9 s, then the whole 5 s), as measured in the human brain, and the thalamus becomes a gate the cortex steers through feedback. New tests for words in noise and for rhythm and music come along the way. After the cortex, the plan (1 November) goes beyond the main pathway, one stage per run: Run 17, a true olivocochlear feedback loop with the middle-ear reflex, so the brainstem turns the cochlea down in steady noise, the one place mel still leads; Run 18, attention from the cortex back down to the thalamus, midbrain and ear, so it can follow one voice in a crowd; Run 19, the belt and parabelt around the core and the two streams, “what” (voices and words) and “where/how” (place and motion); Run 20, the amygdala’s fast alarm route, the startle reflex and surprise detection; Run 21, beat and timing; Run 22, memory for sounds heard before; Run 23, the superior colliculus, where sound meets sight.
Teaching machines to hear the way people do
Machine hearing today starts from a picture of sound. I start from a model of the human ear.
About CymaticBridge
The goal
The goal is robots that hear the way people do: that catch happiness, sadness or anger in a voice, and hear a violinist playing Mozart as music rather than as a stream of numbers. In the long run, I want that machine to listen to music and then create it, using my virtual studio, the Hearing Machine (under How hearing works). Because it can make unlimited audio where I know exactly how every sound was made, I can teach the AI to listen to music: first to judge mixes, later to write its own patches.
Why I start with the ear
Today's listening AI starts from a mel spectrogram, a picture of sound that keeps only rough loudness per frequency band. It then learns patterns from huge amounts of data. It can label a sound without perceiving it the way we do.
CymaticBridge starts from the other end: give the machine a human-shaped ear first (cochlea, auditory nerve, later brainstem and cortex-like processing), then let it learn. If the ear is right, the machine should come to share our perception, including our quirks and illusions.
How I test it
I trained two of my own AI listening models from scratch: one hears through a cochlea model, the other through a standard spectrogram, and everything else is identical. Neither started from a pre-trained AI. Both followed the same published, open design and learned from about 110 hours of everyday sound and speech, without ever being told what any sound was. Then I test both on how well they recognize 50 kinds of sound, in quiet and in noise.
The first milestone is practical: a cochlea front end must beat mel on a standard listening test, fairly. Until it does, a human-shaped ear is only an idea.
Where this could lead
This is early research, and none of these exist yet. But a machine that hears the way people do would be useful far beyond one experiment. These are the directions I am working towards.
Robots and assistants that hear feelings
A home robot that hears worry in an older person's voice, or a care assistant that notices a child is upset, because it takes in tone the way a person does, not only the words. My emotion tests are the first step: the ear model already edges mel at hearing emotion in actors' voices.
Safety and support
Crisis lines, telehealth and emergency calls where AI helps a human listener spot distress in a voice, early, as a second pair of ears.
Better hearing for people
A working model of the human ear can simulate hearing loss. It could help design and test hearing aids and cochlear implants against how people actually hear, not just how loud things are.
Music
AI that hears music as music: judging whether a mix sounds right to human ears, and one day helping make music people feel. The Hearing Machine studio on this site is where that work starts.
Nature and places
Machines that hear birdsong, animal calls and the sound of a place the way we do, for wildlife monitoring and conservation.
Understanding our own hearing
If a machine with a human-shaped ear starts to share our perceptions, even our sound illusions, it becomes a way to test ideas about how human hearing and the auditory brain work. I plan to keep the core research open.
Live from the lab
Checking on the PC…
Hearing voices
Naming a dog or the rain is one thing. My goal is a machine that hears people: the anger, joy or sadness in a voice, and who is speaking. So every run now also takes three voice tests, with actors and speakers it has never heard.
| The first results arrive later today. |
Lab notebook
Every run is a fair race: my ear AI against the standard mel AI, same network, same training, same test. Here is every result so far, and what each one taught me.
| Loading the results… |
20 Oct · New: voice testsMy goal is a machine that hears emotion in a voice, so I added three voice tests: emotion in 7,442 clips of 91 actors (CREMA-D), emotion in 1,440 clips of 24 actors (RAVDESS), and telling 40 people apart by voice (LibriSpeech). Every run takes them from now on. First results, Run 3: on emotion my ear matched or beat mel for the first time in any test (RAVDESS 59.6% against 55.9%; CREMA-D 59.5% against 59.3%), and both told 40 voices apart about 99% of the time. Full table in "Hearing voices" above.
22 Oct · The plan from hereI will build the hearing pathway in the order nature does. Next comes the outer and middle ear, then the cochlear nucleus, the brain's first stop, which splits the nerve signal into streams for timing, a noise-proof spectrum and the starts of sounds. After that: a midbrain that learns, the feedback that turns down background noise, and finally two ears that can tell where a sound is. I'll also add tests for words in noise and for rhythm and music.
1 Nov · The plan past the cortexOnce Run 16 finishes the thalamus and cortex, every stage on the main path of hearing will be in my model except one small relay. So the plan now reaches past it, one stage per run. Run 17: a real feedback loop from the brainstem back to the cochlea, which turns steady noise down (heavy noise is the one place mel still wins). Run 18: attention, the cortex telling the lower stages what to listen for, the trick that lets you follow one voice in a crowd. Run 19: the cortex around the core, and its two streams for what a sound is and where it is. Run 20: the brain’s fast alarm route, the startle reflex and surprise at the unexpected. Run 21: beat and timing. Run 22: memory for sounds heard before. Run 23: the place where sound meets sight. Each one gets a test that shows whether it helps. The Living Ear Map on the Home tab shows the whole plan.
31 Oct · Run 15 result, Run 16 nextGiving both AIs three jobs at once worked: everyday sounds reached 84.7% for my ear and 84.1% for mel, the best either has scored, about 10 points up on Run 14. Direction stayed nearly perfect (my ear 96.0% in rooms, mel 95.9%) and both named the speaker almost every time. My ear led on emotion in voices and on a crowd in the same place, mel in the heaviest noise and on a crowd from the side; every gap is under 5 points. Emotion, which neither AI is trained on, is still a few points below Run 13, so that is what I watch next. Run 16, the thalamus and the cortex, keeps the same three jobs; its check started as soon as Run 15 finished at 8:13 this morning, and it starts on its own when the check passes.
30 Oct · Run 14 result, Run 15 started, Run 16 lined upDirection is solved: with the extra question, my ear found the right one of 12 directions every time in the open and 99.1% of the time in rooms, and mel nearly as often. But everything else slipped for both AIs: everyday sounds 75.1% against 73.0% (about 6 points down on Run 13) and emotion in voices 56.5% against 53.2%. The question was easy and weighted too heavily, so it took over. By a rule I wrote before the results, Run 15 cut its weight to less than half and started on its own at 3:06 am after its check passed, with two new jobs: naming the everyday sound and who is speaking. Run 16, the thalamus and the cortex, is built and tested and starts by itself when Run 15 is done; on the Ear Map both now say Run 16.
28 Oct · Run 14 started: keep the sense of whereRun 14 is Run 13 again with one small extra question during training: which of 12 directions did this sound come from? I place every training sound myself, so the answer is known, and mel gets exactly the same question. A first sign from a 45-minute check: after only 1,000 of its 60,000 steps, my ear already named the right direction 97.3% of the time in the open and 83.3% in a room, more than fully trained Run 13 managed (94.8% and 65.5%). Training takes about a day and a half; results around early morning, 30 October.
28 Oct · Run 13 resultA split decision. With two ears, my ear led on voices (emotion, RAVDESS 61.2%, its best yet, against 58.5%), on where a sound is (94.8% of directions right in the open, against 94.0%) and above all on picking a sound out of a crowd when it comes from the side (it lost nothing; mel lost 6 points). Mel led in heavy noise (30.5% against 27.0% at pink 0 dB), and everyday sounds were a tie (80.9% against 80.8%). Surprise: untrained, my ear's timing map found directions even better (98.0%); training wore some of that away. Next: keep it.
27 Oct · Run 13 started: two earsEvery run so far heard with one ear. Run 13 puts each training sound somewhere around a real measured head (61 people and dummy heads from the SADIE II and CIPIC collections; four more heads are kept back so the tests listen through ears the AI has never heard), mostly inside simulated rooms with echoes. Each ear gets Run 11's whole front end, and a new stage, the superior olive, compares the two ears: their timing (to 45 millionths of a second) and their loudness. Mel gets two ears and the loudness difference, but no fine timing. A first sign from a quick check on 300 sounds: even untrained, the timing map alone points to the right one of 12 directions 88% of the time in the open, against 67% for mel's untrained copy. Training takes about a day and a half.
27 Oct · Run 12 resultMixed. My ear's best score yet on everyday sounds (83.1% against mel's 82.9%) and a bigger lead in the heaviest noise (25.3% against 22.3%), but emotion in voices dropped (RAVDESS 57.3%, from 60.6% in Run 11) and mel gained more from the same noise stage. An automatic gain that tames steady noise also seems to flatten the slow loudness swings that carry feeling in a voice. Next up: two ears.
26 Oct · Run 12 startedHearing in noise. My ear gets the middle-ear reflex, which turns down loud low sounds, and an automatic gain on every channel, modelled on the feedback nerves that let a real ear turn down steady background noise. The mel AI gets both too, so the race stays fair.
26 Oct · Run 11 resultWith the leak fixed, my ear beat mel on nearly every test: 82.9% against 82.0% on everyday sounds, 60.6% against 58.5% on emotion in voices (RAVDESS), and ahead in noise at every level, even the heaviest, where mel had led before. The learned midbrain helped the mel AI more than my ear, which already had similar processing in its own stages, so my lead narrowed but held.
25 Oct · Run 11 startedRun 10 again, done right. The midbrain now only hears the pieces of sound the AI is allowed to see. A short check runs first to confirm the leak is gone before the full run.
25 Oct · Run 10 result: a leakBoth AIs scored about 6 points lower than in Run 9 (my ear 76.4%, mel 76.4% on everyday sounds), and the reason was a mistake in my code. The AIs learn by filling in pieces of sound I hide from them. The new midbrain heard the whole clip before the pieces were hidden, so it passed the AI clues about the hidden parts, and the AI copied them instead of learning. Run 10 tells us nothing about the midbrain, so Run 11 repeats it with the leak fixed.
24 Oct · Run 10 startedA midbrain that learns. Run 9's ear stays exactly as it was, and a model inferior colliculus now sits on top of the cochlear nucleus, as it does in a real brain. Instead of the fixed rhythm bands that did not help in Runs 7 and 8, it learns 8 rhythm detectors of its own, each listening to a quarter of a second across five neighbouring places. The mel AI gets the same midbrain. Results due early on 25 October.
24 Oct · Run 9 resultThe ear won, with its best score yet: 83.0% on everyday sounds against the mel AI's 81.2%, ahead of Run 6's 82.7%. It also won on emotion in voices (59.7% against 55.9% on RAVDESS), on who is speaking, and in light and medium noise, where the new stages helped most: in light noise it rose from 64.3% in Run 8 to 70.2%. The mel AI kept a small edge only in very heavy noise. Building the front of the ear the way nature does, outer ear, middle ear, then the cochlear nucleus, paid off. Next, Run 10: a midbrain that learns, fed from the cochlear nucleus.
23 Oct · Run 9 startedThe outer ear, the middle ear and the cochlear nucleus, the first stop in the brainstem, where the nerve signal splits into three streams: sharper timing, a steadier spectrum for noise, and the starts of sounds. The mel AI gets the same outer ear and the same three formulas.
23 Oct · Run 8 resultStacking didn't help. I gave my ear back the timing reader that won Run 6 and added the midbrain's rhythm bands on top. The mel AI won on everyday sounds (81.8% against 80.0%) and in very heavy noise, while my ear won on emotion in voices (57.4% against 55.1%, RAVDESS) and in light noise. Both were a little below Run 6, so the midbrain bands, as written, aren't adding anything yet. Run 9 goes back to the front of the ear instead, and the midbrain returns later as a stage that learns.
22 Oct · Run 7 resultI added a model midbrain, and it didn't help yet. My ear still edged the mel AI on everyday sounds (82.2% against 81.6%) and on emotion (59.0% against 56.8%, RAVDESS), but both scored a little below Run 6, and in heavy noise mel pulled ahead. When I checked the code I found why: the midbrain had replaced the timing reader that won Run 6 instead of joining it. One good sign: before any training, my ear started well ahead of mel. Run 8 puts the timing reader back and adds the midbrain on top.
21 Oct · Run 6 resultMy biggest step yet. Keeping the nerve's fine timing lifted my ear from 75.9% to 82.7% on everyday sounds, level with the mel AI (82.5%) for the first time. Call it a tie: the difference is smaller than the normal wobble between test rounds. On voices my ear now leads: emotion 60.7% against 56.8% (RAVDESS) and 62.0% against 60.5% (CREMA-D), and who is speaking 99.7% against 99.3%. Noise is still even. The mel AI got the same timing reader and training, so the race stays fair.
20 Oct · Run 5 resultI trained both AIs on lessons buried in noise. Both got better in heavy noise, but the mel AI far more: it jumped from 29.5% to 56.3%, while my ear went from 19.5% to 29.5%. On clean sounds nothing changed much (75.9% against 81.9%). So noise training alone doesn't favour the ear; noise stays mel's strongest ground. On voices the two stayed level: emotion 59.6% against 59.9% (CREMA-D) and 58.5% against 58.8% (RAVDESS). Run 6 tests the nerve's fine timing, which research ties to voices.
20 Oct · Run 4 resultI added the nerve's fine timing, and the ear got worse: 72.3%, down from 76.7%. Short test runs had shown timing helping, so the lesson is that short tests don't predict full runs. Later reading suggests why: fine timing matters most for recognising voices and locating sounds, not for naming everyday sounds. I'll test it on voices next.
19 Oct · Run 3 resultI found a bug: short clips were padded with a "silence" the ear's nerve fibres never produce. Fixing it lifted the ear from 58.2% to 76.7%, against mel's 82.6%. The gap shrank from 24 points to 6.
18 Oct · Run 2 resultThree types of nerve fibre, like a real ear, against a mel picture given the same treatment: mel 82.1%, ear 58.2%. The ear's training started oddly high, which led me to the padding bug.
18 Oct · Run 1 resultThe first big race, on about 110 hours of sound: mel 82.7%, cochlea ear 69.9%. Quick follow-up tests pointed at how loudness was being compressed as the main problem.
Mid-October · The first testsFour ways of hearing compared on ten everyday sounds (Test 1 and Stage 1 tabs). Mel came out slightly ahead on clean sound, and both struggled in heavy noise.
The Living Ear Map
Every stage of hearing, ear to brain, and how far my robot's hearing has come. It grows as I build: each stage turns green when my model has it.
built: in my model now · Run 16: being built now · Run 17 and on: planned, in build order · later: still to come. Dashed lines are feedback running back down.
Explore
Contact
Questions, ideas or want to work together? Get in touch with Ryan Hatch.
Everything in this site was designed and built by me.
Neural map
What the ear sends the model, one 5 s clip
time → 5 s