Zero-shot Multi-Lingual and Multi-Speaker TTS


Authors: Cheng Gong, Xin Wang, Erica Cooper, Dan Wells,
Longbiao Wang, Jianwu Dang, Korin Richmond, Junichi Yamagishi

College of Intelligence and Computing, Tianjin University, China

National Institute of Informatics, Japan

Centre for Speech Technology Research, University of Edinburgh, United Kingdom

Zero-shot TTS for LibriSpeech test-clean

All speakers are unseen during training

Sentence 1 (61-70968-0028)

The head and chief of the riot the nottingham apprentice with clenched fists threatened montfichet

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)

Sentence 2 (2830-3980-0038)

Grace be to you and peace from god the father and from our lord jesus chris t

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)

Sentence 3 (6829-68769-0022)

We have heard something of your story said kenneth and are interested in it

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)

Sentence 4 (8455-210777-0027)

when this captain should have taken himself and his vessel back to england i would retire to a small farm which i possessed at the farthest side of the island and there in seclusion would i end my days

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)

Sentence 5 (8555-284449-0017)

When first they entered the throne room they tried to be as haughty and scornful as ever but the blues who were assembled there all laughed at them and jeered them for there was not a single person in all the blue country who loved the princesses the least little bit

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)

Sentence 6 (260-123286-0029)

Those huge creatures attacked each other with the greatest animosity

GT

VALL-E-X

  HierSpeech++(a)

  HierSpeech++(b)

StyleTTS 2

ZMM-TTS2x(a)

ZMM-TTS2x(b)