Where the data comes from: the audio companion to the fourth article

The fourth article of the series is about data. The speech recogniser from the second article and the voice from the third had to learn from something, and our language had almost no ready-made data. The article tells where it came from and how many traps it held. Here are four examples you can hear or see.

Two recognisers agreed

We had an audiobook several hours long, and I never found the printed text of that edition. So our recogniser transcribed the recording, and then a second one, built in a completely different way, listened to it too. Only the pieces where the two transcripts almost matched went into the corpus. Two models that make different mistakes rarely make the same one.

A piece of the audiobook

The old run, same place

Menimki öldi, men cenki kettim, saq selamet qaytıp, başıñız sağ olsun, başıñız sağ olsun, başıñız sağ olsun… and so on, about 50 times in a row

Now, both recognisers

Aydı! Yahşı! Aç olmadı, balam! Toqluqnıñ zararı yoq. Tek içmesin! – dep qoştı Arslanov.

This is not proof that the text is right: two wrong models can, in principle, agree. That is why we check the doubtful pieces by ear.

The reference text was wrong

To tell whether the recogniser got better, you need reference texts. For live speech they used to be made by the machine itself, the old version of the recogniser. I took the pieces where the old and new models disagreed and listened to them blind, without knowing which transcript was whose. The reference texts turned out to be wrong in about 15 % of words. Here are three short lines from our portal, recorded on an ordinary microphone. In each one the reference was wrong, and the new model wrote down what I heard.

Three lines from the portal

1. Reference

Aleyküm selâm, sebzeler qidıram. Qaçtan qıyar sized?

1. By ear

Aleykom selâm, sebzelerni qıdıram, qaçtan qıyar sizde?

2. Reference

Daa ek kilo pomidor berimiz.

2. By ear

Daa eki kilo pomidor beriñiz?

3. Reference

Çoq sağ oluñız! Qarlı künler sizgede! qarlı künler means “snowy days”

3. By ear

Çoq sağ oluñız, hayırlı künler sizge de. hayırlı künler means “good days”

As long as the reference was wrong, a new model that disagreed with the old one automatically looked worse. In fact it was right noticeably more often.

One text, two synthetic voices

When you have few recordings of your own, you want to make them yourself. A synthetic recording comes with a perfect text: we chose the text before the machine read it. I added synthetic speech in several voices to the recogniser’s training. Below is just an illustration of what that means: the same two sentences, in Sevil’s voice and in mine. These recordings were not part of the training.

On an exam of two audiobooks the synthetic speech helped: word errors went down to 15.7 % from 17.0 %. But on recordings closer to real life, the model trained with it looked worse, and it never went into use.

Ours or Turkish

Once our language model answered me «Ayt, men dinlemeye hazırım». In Crimean Tatar it should be «Ayt, men diñlemege azırım», “Go on, I’m ready to listen”. Almost the same words, but a Turkish form. An ordinary language filter does not see this, so I had to build my own detectors. They look for forms that Crimean Tatar does not have.

Ayt, men diñlemege azırım.

The machines do the rough work here. In the doubtful places, the last word stays with a native speaker’s ear.

Read the full story

• The full article on ana-yurt.com (in Russian) — link to follow