Title: UnifySpeech: A Unified Framework for Zero-shot Text-to-Speech and  Voice Conversion. Abstract: The unlabeled speech contains rich speaker style information, which can improve the few-shots modeling capability. This paper proposes UnifySpeech to make use of large amounts of unlabeled data for model training and boost the performance of text-to-speech (TTS) and voice conversion (VC) simultaneously. UnifySpeech brings TTS and VC into a unified framework for the first time. The model is based on the assumption that speech can be decoupled into three independent components: content information, speaker information, prosody information. Both TTS and VC can be regarded as mining these three parts of information from the input and completing the reconstruction of speech. For TTS, the speech content information is derived from the text, while in VC it's derived from the source speech, so all the remaining units are shared except for the speech content extraction module in the two pipelines. We applied vector quantization and loss optimization to bridge the gap between the content domains of TTS and VC. Objective evaluation shows UnifySpeech gets higer speaker similarity and pitch prediction accuracy, indicating the improvements of the style modeling ability. Subjective evaluation shows speech generated by UnifySpeech obtains high mean opinion score (mos) that the audio is as natural as human voice.