Note
The voice that read a whole book: the audio companion to the third article
A synthetic voice read a whole novel — 16 chapters, 1 hour 58 minutes, a median of 2.15 % character errors per chapter. What broke along the way was not the voice but everything around it: gluing sentences, trimming breaths, one letter. Here you can hear it, before and after.
The third article of the series is out on ana-yurt.com, and it is about the voice: speech synthesis, the computer reading text out loud. A synthetic voice read a whole book — Şamil Alâdin’s novel «Merdiven», sixteen chapters, 1 hour 58 minutes of sound. This page is the article’s audio companion: what the article tells in words, you can hear here.
Why a book and not a handful of good sentences? Fourteen test sentences say nothing about how a voice behaves for ten minutes straight: whether the timbre drifts by the end of a chapter, what the machine does with dialogue, with names, with lines one or two words long, how it breathes. You can’t hear any of that in a fifteen-second clip. In a book you hear all of it.
The verdict surprised me. The voice held: the timbre barely changed from the first minute to the last, and the pace stayed even. The faults were not in the model but in the text and audio handling around it — how short sentences get glued together, how breaths get trimmed, how one letter gets rewritten into another. I had written that part myself, which made it good news: it could be fixed without retraining anything. Below you can hear those places before the fix and after it.
Everything on this page is synthesis; there is no human narrator here. «Merdiven» is read in Sevil’s voice; the second book and the voice-sample example are in my own.
The whole book
The opening of chapter one of «Merdiven» is in the companion to the first article. Here is chapter ten instead, and almost all of it is dialogue: short lines, questions, exclamations, the speaker changing.
«Merdiven», chapter X: dialogue
The whole novel is in the ana-yurt.com library, in a reader that lights up each word as it is spoken: ana-yurt.com/kutuphane/3444/read.
I checked every chapter on its own: I read the synthesis back with our speech recogniser and counted the characters that differ from the text. A single average for the whole book would be the wrong number — one bad line spoils the average and says nothing about the reading. Chapter by chapter the picture is even: median 2.15 %, best chapter 1.5 %, worst 2.7 %, and quality slides neither toward the end of the book nor on the long chapters.
What broke, and how it sounds
Gluing short sentences
At first I glued short sentences onto their neighbours: the model had learned from recordings no shorter than three seconds, so a two-word line seemed safer not handed to it alone. I never checked that assumption — and on the glued pieces the model started dropping whole sentences. The glued pieces made up a quarter of the chapter’s text and produced 43 % of its errors.
Glued into one request
Each sentence on its own
So I finally tested the assumption and voiced the chapter with no gluing at all. Short sentences really do come out worse — about two and a half times as many errors, though in absolute terms that is about two percentage points. But that penalty is smaller than the price of gluing, which was losing whole pieces of text. I switched gluing off.
The trim that ate the first word
Synthesis sometimes opens a phrase with an intake of breath, like a live narrator, so I wrote a trim that cut everything before the first voiced sound. But plenty of words begin with a voiceless consonant, and in 111 sentences out of 147 the trim cut into the first word. The new trim goes by loudness instead: a consonant is quiet, but still much louder than the silence in front of it. Both clips below are the same rendering; the only difference is where the start was cut.
Old trim
New trim
«nice» that came out as «nizhe»
In the first article I told the story of our word «nice» — no relation to the English one: it means “how many” and sounds roughly like nee-djeh — being read as «nizhe», a Russian word meaning “lower”. The rule that rewrites our letters for the model was quietly turning dj into zh, so the model was literally handed a Russian word. To fix it I compared five ways of writing that sound. On the fourteen test sentences the two best were indistinguishable; only the whole chapter, 195 sentences, showed that one of them, the Tatar letter җ, fixed the word and damaged everything else: 27 % more character errors. Plain dj fixed the word and broke nothing.
dj rewritten as zh
Plain dj
Whose voice it is
The sample decides
The model I fine-tuned learns the language, not the timbre: it takes the timbre from a short sample, a few seconds of someone’s recording along with its exact text. I first tried my own voice on single sentences and heard long, misplaced pauses in the middle of them. It wasn’t the model, it was the sample: 0.77 seconds of silence before the words and 1.77 seconds after. The model takes the pace from the sample as well, and it was spreading that extra time through the phrase as pauses. I trimmed the silence off the edges, and the median total of pauses inside a sentence dropped from 930 to 85 milliseconds. In a blind comparison the voice with the trimmed sample couldn’t be told apart from the old one: 7 to 6.
Sample with silence at its edges
The same sample, silence trimmed
The second book
With the sample sorted out, I decided to try my own voice on a whole book. That is how the second book in our library — Yunus Qandım’s «Küreş meydanını ot basmaz», 28 chapters, 6 hours 29 minutes of sound — came to be read in my voice. I didn’t pick it by chance: its author, Yunus Qandım, is my uncle. It was read by the current version of the model; «Merdiven» by an earlier one.
«Küreş meydanını ot basmaz», the opening of chapter one
You can listen to all of it too: ana-yurt.com/kutuphane/3516/read.
The voice turned out to be the most reliable part of the whole chain. Nearly everything that broke was around it: the gluing, the trim, the letters, the timings. Some of it the instruments found, and the ear never would have. Some of it the ear found, and the instruments couldn’t even score it. And the whole book found what neither fourteen sentences nor one chapter would have.