Презентация Speech Recognition and Synthesis. Waveform Synthesis (in Concatenative TTS) онлайн
На нашем сайте вы можете скачать и просмотреть онлайн доклад-презентацию на тему Speech Recognition and Synthesis. Waveform Synthesis (in Concatenative TTS) абсолютно бесплатно. Урок-презентация на эту тему содержит всего 66 слайдов. Все материалы созданы в программе PowerPoint и имеют формат ppt или же pptx. Материалы и темы для презентаций взяты из открытых источников и загружены их авторами, за качество и достоверность информации в них администрация сайта не отвечает, все права принадлежат их создателям. Если вы нашли то, что искали, отблагодарите авторов - поделитесь ссылкой в социальных сетях, а наш сайт добавьте в закладки.
Презентации » Физика » Speech Recognition and Synthesis. Waveform Synthesis (in Concatenative TTS)
Оцените!
Оцените презентацию от 1 до 5 баллов!
- Тип файла:ppt / pptx (powerpoint)
- Всего слайдов:66 слайдов
- Для класса:1,2,3,4,5,6,7,8,9,10,11
- Размер файла:2.46 MB
- Просмотров:78
- Скачиваний:0
- Автор:неизвестен
Слайды и текст к этой презентации:
№6 слайд
![Diphone TTS architecture](/documents_6/4c94d2745087cbe375c23fefabef4202/img5.jpg)
Содержание слайда: Diphone TTS architecture
Training:
Choose units (kinds of diphones)
Record 1 speaker saying 1 example of each diphone
Mark the boundaries of each diphones,
cut each diphone out and create a diphone database
Synthesizing an utterance,
grab relevant sequence of diphones from database
Concatenate the diphones, doing slight signal processing at boundaries
use signal processing to change the prosody (F0, energy, duration) of selected sequence of diphones
№8 слайд
![Diphones mid-phone is more](/documents_6/4c94d2745087cbe375c23fefabef4202/img7.jpg)
Содержание слайда: Diphones
mid-phone is more stable than edge
Need O(phone2) number of units
Some combinations don’t exist (hopefully)
ATT (Olive et al. 1998) system had 43 phones
1849 possible diphones
Phonotactics ([h] only occurs before vowels), don’t need to keep diphones across silence
Only 1172 actual diphones
May include stress, consonant clusters
So could have more
Lots of phonetic knowledge in design
Database relatively small (by today’s standards)
Around 8 megabytes for English (16 KHz 16 bit)
№10 слайд
![Designing a diphone inventory](/documents_6/4c94d2745087cbe375c23fefabef4202/img9.jpg)
Содержание слайда: Designing a diphone inventory:
Nonsense words
Build set of carrier words:
pau t aa b aa b aa pau
pau t aa m aa m aa pau
pau t aa m iy m aa pau
pau t aa m iy m aa pau
pau t aa m ih m aa pau
Advantages:
Easy to get all diphones
Likely to be pronounced consistently
No lexical interference
Disadvantages:
(possibly) bigger database
Speaker becomes bored
№11 слайд
![Designing a diphone inventory](/documents_6/4c94d2745087cbe375c23fefabef4202/img10.jpg)
Содержание слайда: Designing a diphone inventory:
Natural words
Greedily select sentences/words:
Quebecois arguments
Brouhaha abstractions
Arkansas arranging
Advantages:
Will be pronounced naturally
Easier for speaker to pronounce
Smaller database? (505 pairs vs. 1345 words)
Disadvantages:
May not be pronounced correctly
№12 слайд
![Making recordings consistent](/documents_6/4c94d2745087cbe375c23fefabef4202/img11.jpg)
Содержание слайда: Making recordings consistent:
Diiphone should come from mid-word
Help ensure full articulation
Performed consistently
Constant pitch (monotone), power, duration
Use (synthesized) prompts:
Helps avoid pronunciation problems
Keeps speaker consistent
Used for alignment in labeling
№13 слайд
![Building diphone schemata](/documents_6/4c94d2745087cbe375c23fefabef4202/img12.jpg)
Содержание слайда: Building diphone schemata
Find list of phones in language:
Plus interesting allophones
Stress, tons, clusters, onset/coda, etc
Foreign (rare) phones.
Build carriers for:
Consonant-vowel, vowel-consonant
Vowel-vowel, consonant-consonant
Silence-phone, phone-silence
Other special cases
Check the output:
List all diphones and justify missing ones
Every diphone list has mistakes
№15 слайд
![Labeling Diphones Run a](/documents_6/4c94d2745087cbe375c23fefabef4202/img14.jpg)
Содержание слайда: Labeling Diphones
Run a speech recognizer in forced alignment mode
Forced alignment:
A trained ASR system
A wavefile
A word transcription of the wavefile
Returns an alignment of the phones in the words to the wavefile.
Much easier than phonetic labeling:
The words are defined
The phone sequence is generally defined
They are clearly articulated
But sometimes speaker still pronounces wrong, so need to check.
Phone boundaries less important
+- 10 ms is okay
Midphone boundaries important
Where is the stable part
Can it be automatically found?
№16 слайд
![Diphone auto-alignment Given](/documents_6/4c94d2745087cbe375c23fefabef4202/img15.jpg)
Содержание слайда: Diphone auto-alignment
Given
synthesized prompts
Human speech of same prompts
Do a dynamic time warping alignment of the two
Using Euclidean distance
Works very well 95%+
Errors are typically large (easy to fix)
Maybe even automatically detected
Malfrere and Dutoit (1997)
№18 слайд
![Finding diphone boundaries](/documents_6/4c94d2745087cbe375c23fefabef4202/img17.jpg)
Содержание слайда: Finding diphone boundaries
Stable part in phones
For stops: one third in
For phone-silence: one quarter in
For other diphones: 50% in
In time alignment case:
Given explicit known diphone boundaries in prompt in the label file
Use dynamic time warping to find same stable point in new speech
Optimal coupling
Taylor and Isard 1991, Conkie and Isard 1996
Instead of precutting the diphones
Wait until we are about to concatenate the diphones together
Then take the 2 complete (uncut diphones)
Find optimal join points by measuring cepstral distance at potential join points, pick best
№21 слайд
![Concatenating diphones](/documents_6/4c94d2745087cbe375c23fefabef4202/img20.jpg)
Содержание слайда: Concatenating diphones: junctures
If waveforms are very different, will perceive a click at the junctures
So need to window them
Also if both diphones are voiced
Need to join them pitch-synchronously
That means we need to know where each pitch period begins, so we can paste at the same place in each pitch period.
Pitch marking or epoch detection: mark where each pitch pulse or epoch occurs
Finding the Instant of Glottal Closure (IGC)
(note difference from pitch tracking)
№23 слайд
![Epoch-labeling](/documents_6/4c94d2745087cbe375c23fefabef4202/img22.jpg)
Содержание слайда: Epoch-labeling: Electroglottograph (EGG)
Also called laryngograph or Lx
Device that straps on speaker’s neck near the larynx
Sends small high frequency current through adam’s apple
Human tissue conducts well; air not as well
Transducer detects how open the glottis is (I.e. amount of air between folds) by measuring impedence.
№33 слайд
![Overlap and Add OLA Hanning](/documents_6/4c94d2745087cbe375c23fefabef4202/img32.jpg)
Содержание слайда: Overlap and Add (OLA)
Hanning windows of length 2N used to multiply the analysis signal
Resulting windowed signals are added
Analysis windows, spaced 2N
Synthesis windows, spaced N
Time compression is uniform with factor of 2
Pitch periodicity somewhat lost around 4th window
№37 слайд
![Summary Diphone Synthesis](/documents_6/4c94d2745087cbe375c23fefabef4202/img36.jpg)
Содержание слайда: Summary: Diphone Synthesis
Well-understood, mature technology
Augmentations
Stress
Onset/coda
Demi-syllables
Problems:
Signal processing still necessary for modifying durations
Source data is still not natural
Units are just not large enough; can’t handle word-specific effects, etc
№39 слайд
![Unit Selection Synthesis](/documents_6/4c94d2745087cbe375c23fefabef4202/img38.jpg)
Содержание слайда: Unit Selection Synthesis
Generalization of the diphone intuition
Larger units
From diphones to sentences
Many many copies of each unit
10 hours of speech instead of 1500 diphones (a few minutes of speech)
Little or no signal processing applied to each unit
Unlike diphones
№40 слайд
![Why Unit Selection Synthesis](/documents_6/4c94d2745087cbe375c23fefabef4202/img39.jpg)
Содержание слайда: Why Unit Selection Synthesis
Natural data solves problems with diphones
Diphone databases are carefully designed but:
Speaker makes errors
Speaker doesn’t speak intended dialect
Require database design to be right
If it’s automatic
Labeled with what the speaker actually said
Coarticulation, schwas, flaps are natural
“There’s no data like more data”
Lots of copies of each unit mean you can choose just the right one for the context
Larger units mean you can capture wider effects
№41 слайд
![Unit Selection Intuition](/documents_6/4c94d2745087cbe375c23fefabef4202/img40.jpg)
Содержание слайда: Unit Selection Intuition
Given a big database
For each segment (diphone) that we want to synthesize
Find the unit in the database that is the best to synthesize this target segment
What does “best” mean?
“Target cost”: Closest match to the target description, in terms of
Phonetic context
F0, stress, phrase position
“Join cost”: Best join with neighboring units
Matching formants + other spectral characteristics
Matching energy
Matching F0
№42 слайд
![Targets and Target Costs A](/documents_6/4c94d2745087cbe375c23fefabef4202/img41.jpg)
Содержание слайда: Targets and Target Costs
A measure of how well a particular unit in the database matches the internal representation produced by the prior stages
Features, costs, and weights
Examples:
/ih-t/ from stressed syllable, phrase internal, high F0, content word
/n-t/ from unstressed syllable, phrase final, low F0, content word
/dh-ax/ from unstressed syllable, phrase initial, high F0, from function word “the”
№44 слайд
![How to set target cost](/documents_6/4c94d2745087cbe375c23fefabef4202/img43.jpg)
Содержание слайда: How to set target cost weights (1)
What you REALLY want as a target cost is the perceivable acoustic difference between two units
But we can’t use this, since the target is NOT ACOUSTIC yet, we haven’t synthesized it!
We have to use features that we get from the TTS upper levels (phones, prosody)
But we DO have lots of acoustic units in the database.
We could use the acoustic distance between these to help set the WEIGHTS on the acoustic features.
№45 слайд
![How to set target cost](/documents_6/4c94d2745087cbe375c23fefabef4202/img44.jpg)
Содержание слайда: How to set target cost weights (2)
Clever Hunt and Black (1996) idea:
Hold out some utterances from the database
Now synthesize one of these utterances
Compute all the phonetic, prosodic, duration features
Now for a given unit in the output
For each possible unit that we COULD have used in its place
We can compute its acoustic distance from the TRUE ACTUAL HUMAN utterance.
This acoustic distance can tell us how to weight the phonetic/prosodic/duration features
№46 слайд
![How to set target cost](/documents_6/4c94d2745087cbe375c23fefabef4202/img45.jpg)
Содержание слайда: How to set target cost weights (3)
Hunt and Black (1996)
Database and target units labeled with:
phone context, prosodic context, etc.
Need an acoustic similarity between units too
Acoustic similarity based on perceptual features
MFCC (spectral features) (to be defined next week)
F0 (normalized)
Duration penalty
№51 слайд
![Join costs The join cost can](/documents_6/4c94d2745087cbe375c23fefabef4202/img50.jpg)
Содержание слайда: Join costs
The join cost can be used for more than just part of search
Can use the join cost for optimal coupling (Isard and Taylor 1991, Conkie 1996), i.e., finding the best place to join the two units.
Vary edges within a small amount to find best place for join
This allows different joins with different units
Thus labeling of database (or diphones) need not be so accurate
№53 слайд
![Improvements Taylor and Black](/documents_6/4c94d2745087cbe375c23fefabef4202/img52.jpg)
Содержание слайда: Improvements
Taylor and Black 1999: Phonological Structure Matching
Label whole database as trees:
Words/phrases, syllables, phones
For target utterance:
Label it as tree
Top-down, find subtrees that cover target
Recurse if no subtree found
Produces list of target subtrees:
Explicitly longer units than other techniques
Selects on:
Phonetic/metrical structure
Only indirectly on prosody
No acoustic cost
№56 слайд
![Database creation Good](/documents_6/4c94d2745087cbe375c23fefabef4202/img55.jpg)
Содержание слайда: Database creation (1)
Good speaker
Professional speakers are always better:
Consistent style and articulation
Although these databases are carefully labeled
Ideally (according to AT&T experiments):
Record 20 professional speakers (small amounts of data)
Build simple synthesis examples
Get many (200?) people to listen and score them
Take best voices
Correlates for human preferences:
High power in unvoiced speech
High power in higher frequencies
Larger pitch range
№57 слайд
![Database creation Good](/documents_6/4c94d2745087cbe375c23fefabef4202/img56.jpg)
Содержание слайда: Database creation (2)
Good recording conditions
Good script
Application dependent helps
Good word coverage
News data synthesizes as news data
News data is bad for dialog.
Good phonetic coverage, especially wrt context
Low ambiguity
Easy to read
Annotate at phone level, with stress, word information, phrase breaks
№59 слайд
![Practical System Issues Size](/documents_6/4c94d2745087cbe375c23fefabef4202/img58.jpg)
Содержание слайда: Practical System Issues
Size of typical system (Rhetorical rVoice):
~300M
Speed:
For each diphone, average of 1000 units to choose from, so:
1000 target costs
1000x1000 join costs
Each join cost, say 30x30 float point calculations
10-15 diphones per second
10 billion floating point calculations per second
But commercial systems must run ~50x faster than real time
Heavy pruning essential: 1000 units -> 25 units
№60 слайд
![Unit Selection Summary](/documents_6/4c94d2745087cbe375c23fefabef4202/img59.jpg)
Содержание слайда: Unit Selection Summary
Advantages
Quality is far superior to diphones
Natural prosody selection sounds better
Disadvantages:
Quality can be very bad in places
HCI problem: mix of very good and very bad is quite annoying
Synthesis is computationally expensive
Can’t synthesize everything you want:
Diphone technique can move emphasis
Unit selection gives good (but possibly incorrect) result
№61 слайд
![Recap Joining Units F](/documents_6/4c94d2745087cbe375c23fefabef4202/img60.jpg)
Содержание слайда: Recap: Joining Units (+F0 + duration)
unit selection, just like diphone, need to join the units
Pitch-synchronously
For diphone synthesis, need to modify F0 and duration
For unit selection, in principle also need to modify F0 and duration of selection units
But in practice, if unit-selection database is big enough (commercial systems)
no prosodic modifications (selected targets may already be close to desired prosody)
№63 слайд
![Evaluation of TTS](/documents_6/4c94d2745087cbe375c23fefabef4202/img62.jpg)
Содержание слайда: Evaluation of TTS
Intelligibility Tests
Diagnostic Rhyme Test (DRT)
Humans do listening identification choice between two words differing by a single phonetic feature
Voicing, nasality, sustenation, sibilation
96 rhyming pairs
Veal/feel, meat/beat, vee/bee, zee/thee, etc
Subject hears “veal”, chooses either “veal or “feel”
Subject also hears “feel”, chooses either “veal” or “feel”
% of right answers is intelligibility score.
Overall Quality Tests
Have listeners rate space on a scale from 1 (bad) to 5 (excellent) (Mean Opinion Score)
AB Tests (prefer A, prefer B) (preference tests)
№64 слайд
![Recent stuff Problems with](/documents_6/4c94d2745087cbe375c23fefabef4202/img63.jpg)
Содержание слайда: Recent stuff
Problems with Unit Selection Synthesis
Can’t modify signal
(mixing modified and unmodified sounds bad)
But database often doesn’t have exactly what you want
Solution: HMM (Hidden Markov Model) Synthesis
Won the last TTS bakeoff.
Sounds unnatural to researchers
But naïve subjects preferred it
Has the potential to improve on both diphone and unit selection.
Скачать все slide презентации Speech Recognition and Synthesis. Waveform Synthesis (in Concatenative TTS) одним архивом:
Похожие презентации
-
Alessandro Volta (1745-1827)
-
Fast and Simple Physics using Sequential Impulses
-
Alessandro Volta
-
Heat flow and the first law of thermodynamics. Lecture 6
-
Methods and technical means for using the energy of waves
-
Wire vibration and ways to prevent it
-
General terms of transmission lines performance and simulation
-
Static Equilibrium and Friction
-
CLP4NET. IAEA Cyber Learning Platform For Nuclear Education and Training
-
Superconductivity in science and in engineering