Enhance README with features and usage examples
Expanded README with detailed features, usage instructions, and FAQs.
This commit is contained in:
88
README.md
88
README.md
@@ -1,23 +1,95 @@
|
|||||||
Simple installation:
|
# LuxTTS
|
||||||
|
LuxTTS is an lightweight zipvoice based text-to-speech model designed for high quality voice cloning and realistic generation at speeds exceeding 150x realtime.
|
||||||
|
|
||||||
|
https://github.com/user-attachments/assets/a3b57152-8d97-43ce-bd99-26dc9a145c29
|
||||||
|
|
||||||
|
|
||||||
|
### The main features are
|
||||||
|
- Voice cloning: SOTA voice cloning on par with models 10x larger.
|
||||||
|
- Clarity: Clear 48khz speech generation unlike most TTS models which are limited to 24khz.
|
||||||
|
- Speed: Reaches speeds of 150x realtime on a single GPU and faster then realtime on CPU's as well.
|
||||||
|
- Efficiency: Fits within 1gb vram meaning it can fit in any local gpu.
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
#### Simple installation:
|
||||||
```
|
```
|
||||||
git clone https://github.com/ysharma3501/LuxTTS.git
|
git clone https://github.com/ysharma3501/LuxTTS.git
|
||||||
cd LuxTTS
|
cd LuxTTS
|
||||||
pip install -r requirements.txt
|
pip install -r requirements.txt
|
||||||
```
|
```
|
||||||
|
|
||||||
Usage:
|
#### Load model:
|
||||||
```python
|
```python
|
||||||
from zipvoice.luxtts import LuxTTS
|
from zipvoice.luxtts import LuxTTS
|
||||||
lux_tts = LuxTTS('YatharthS/LuxTTS', device='cuda') ## change device to cpu for cpu usage
|
lux_tts = LuxTTS('YatharthS/LuxTTS', device='cuda', threads=2) ## change device to cpu for cpu usage
|
||||||
```
|
```
|
||||||
|
|
||||||
Infer:
|
#### Simple inference
|
||||||
```python
|
```python
|
||||||
text = "Hey, what's up loser? I think you should shut up? You have NO dignity either way, ugh!"
|
from IPython.display import Audio
|
||||||
prompt_audio = '/kaggle/input/voices/ElevenLabs_2025-11-02T22_31_30_Jessica_pre_sp100_s35_sb80_v3.mp3'
|
|
||||||
|
|
||||||
encoded_prompt = lux_tts.encode_prompt(prompt_audio, rms=0.001)
|
text = "Hey, what's up? I'm feeling really great if you ask me honestly!"
|
||||||
final_wav = lux_tts.generate_speech(text, encoded_prompt, num_steps=4, t_shift=0.9)
|
prompt_audio = 'audio_file.wav'
|
||||||
|
|
||||||
|
## encode audio(takes 10s to init because of librosa first time)
|
||||||
|
encoded_prompt = lux_tts.encode_prompt(prompt_audio, rms=rms)
|
||||||
|
|
||||||
|
## generate speech
|
||||||
|
final_wav = lux_tts.generate_speech(text, encoded_prompt, num_steps=num_steps)
|
||||||
|
|
||||||
|
## display speech
|
||||||
display(Audio(final_wav, rate=48000))
|
display(Audio(final_wav, rate=48000))
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Inference with sampling params:
|
||||||
|
```python
|
||||||
|
from IPython.display import Audio
|
||||||
|
|
||||||
|
text = "Hey, what's up? I'm feeling really great if you ask me honestly!"
|
||||||
|
prompt_audio = 'audio_file.wav'
|
||||||
|
|
||||||
|
rms = 0.01 ## higher makes it sound louder(0.01 or so recommended)
|
||||||
|
t_shift = 0.9 ## sampling param, higher can sound better but worse WER
|
||||||
|
num_steps = 4 ## sampling param, higher sounds better but takes longer(3-4 is best for efficiency)
|
||||||
|
speed = 1.0 ## sampling param, controls speed of audio(lower=faster)
|
||||||
|
return_smooth = False ## sampling param, makes it sound smoother possibly but less cleaner
|
||||||
|
|
||||||
|
## encode audio(takes 10s to init because of librosa first time)
|
||||||
|
encoded_prompt = lux_tts.encode_prompt(prompt_audio, rms=rms)
|
||||||
|
|
||||||
|
## generate speech
|
||||||
|
final_wav = lux_tts.generate_speech(text, encoded_prompt, num_steps=num_steps, t_shift=t_shift, speed=speed, return_smooth=return_smooth)
|
||||||
|
|
||||||
|
## display speech
|
||||||
|
display(Audio(final_wav, rate=48000))
|
||||||
|
```
|
||||||
|
## Tips
|
||||||
|
- Please use at minimum a 3 second audio file for voice cloning.
|
||||||
|
- You can use return_smooth = True if you hear metallic sounds.
|
||||||
|
- Lower t_shift for less possible pronunciation errors but worse quality and vice versa.
|
||||||
|
|
||||||
|
|
||||||
|
## Info
|
||||||
|
|
||||||
|
Q: How is this different from ZipVoice?
|
||||||
|
|
||||||
|
A: LuxTTS uses the same architecture but distilled to 4 steps with an improved sampling technique. It also uses a custom 48khz vocoder instead of the default 24khz version.
|
||||||
|
|
||||||
|
Q: Can it be even faster?
|
||||||
|
|
||||||
|
A: Yes, currently it uses float32. Float16 should be significantly faster(almost 2x).
|
||||||
|
|
||||||
|
## Roadmap
|
||||||
|
|
||||||
|
- [x] Release model and code
|
||||||
|
- [ ] Huggingface spaces demo
|
||||||
|
- [ ] Release code for float16 inference
|
||||||
|
|
||||||
|
## Final Notes
|
||||||
|
|
||||||
|
This project is licensed under the Apache-2.0 license. See LICENSE for details.
|
||||||
|
|
||||||
|
Stars/Likes would be appreciated, thank you.
|
||||||
|
|
||||||
|
Email: yatharthsharma350@gmail.com
|
||||||
|
|||||||
Reference in New Issue
Block a user