https://github.com/k2-fsa/OmniVoice

需求

文字生成音频

python 3.11

conda create -n omniVoice-env python=3.11
conda activate omniVoice-env
conda install pip
python -m pip -V 这个查看pip命令是哪里的，默认不装pip，

torch

我是1650 只能装老版本

pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu121
2.4.1 版本太低了，omniVoice 用不了，换一个
pip install torch==2.6.0 torchaudio==2.6.0 torchvision==0.21.0 --index-url https://download.pytorch.org/whl/cu126
这个ok了

查看安装版本

(omniVoice-env) ➥ omniVoice-env $ python -c "import torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available()); print('cuDNN:', torch.backends.cudnn.is_available()); print('Version:', torch.backends.cudnn.version() if torch.backends.cudnn.is_available() else 'N/A')"
PyTorch: 2.4.1+cu121
CUDA: True
cuDNN: True
Version: 90100

cudnn 跳过

1650版本
安装完torch是带cudnn的，现看看是什么版本

pip show nvidia-cudnn-cu12

conda install -c conda-forge cudnn=8.9.7

runtime 跳过

1650版本

pip install onnxruntime-gpu==1.19.2

omnivoice

pip install omnivoice

从hf上下载了个模型, 查看缓存目录

(omniVoice-env) ➥ omniVoice-env $ python -c "from huggingface_hub import constants; print(constants.HF_HOME)"
C:\Users\Reciter\.cache\huggingface

设置的变量查看

echo $env:HF_HOME
我这里没有显示，就放c盘吧 我就现不折腾了

代码

测试可以用，参考10秒里面的内容很重要，这个我特意裁剪了个带哈哈哈的，注意要没有背景环境音的参考音频。

from omnivoice import OmniVoice
import soundfile as sf
import torch

model = OmniVoice.from_pretrained(
    "k2-fsa/OmniVoice",
    device_map="cuda:0",
    dtype=torch.float16
)
# Apple Silicon users: use device_map="mps" instead
# Intel Arc GPU users: use device_map="xpu" instead

audio = model.generate(
    language="zh",
    text="这是一个测试的文本。哈哈哈哈哈哈，好了好了，现不说了，Callback！",
    ref_audio="douyin_voice_9s.wav",
    ref_text="笑死我了，哈哈哈。这个配音，哈哈哈。你们赶紧用一下吧，哈哈哈。这个配音名字叫做真人博客女。",
) # audio is a list of `np.ndarray` with shape (T,) at 24 kHz.

# If you don't want to input `ref_text` manually, you can directly omit the `ref_text`.
# The model will use Whisper ASR to auto-transcribe it. To use a local copy (or
# a different Whisper model), pass `asr_model_name="..."` to `from_pretrained`.
# To control which device Whisper is loaded on (e.g. another GPU in multi-GPU
# setups, or the CPU), pass `asr_device="cuda:1"` (or `"cpu"`).

sf.write("out.wav", audio[0], 24000)

ApplioV3.6.3

这个音色转换的也不错，这个是音频转音频的。需要找音色模型
E:\aizzz-env-dev\rvc-env\ApplioV3.6.3