mirror of
https://github.com/storytold/storyteller-ml.git
synced 2026-10-13 23:53:02 +00:00
Brandon Thomas
65 lines
1.8 KiB
Markdown
65 lines
1.8 KiB
Markdown
tts-core
|
|
========
|
|
The core TTS models that power FakeYou.
|
|
|
|
I'm in the progress of pulling these out of the service monorepo so we can iterate faster.
|
|
|
|
**This code sucks and doesn't run yet**.
|
|
|
|
1. Please do not think this is indicative of the kind of code I work on or maintain.
|
|
This was a hermetic module that the FakeYou service called out to, and I want to improve it now.
|
|
The rest of the frontend and backend are in a good state.
|
|
|
|
2. I'll spend some time getting this cleaned up and working before getting any help with it.
|
|
|
|
How This Works
|
|
--------------
|
|
We have a Rust monolith that controls the user interface, account system, and all the database CRUD operations.
|
|
It doesn't do any ML work itself or have any attached GPUs.
|
|
|
|
There are a series of worker pods ("jobs") that pull from work queues and run inference, then upload the results.
|
|
|
|
|
|
|
|
The jobs are written in Rust and either shell out to Python code or call an in-container Python server that attempts
|
|
to LRU cache models in memory.
|
|
|
|
|
|
|
|
|
|
|
|
TODO
|
|
----
|
|
|
|
Core cleanup
|
|
|
|
- [ ] Cleanup Tacotron code
|
|
- [ ] Integrate WaveGlow and HifiGan more cleanly
|
|
- [ ] Define the HTTP server interface
|
|
- [ ] Define the model checking interface
|
|
- [ ] Clean up in-memory LRU caching
|
|
- [ ] There are two copies of Nemo. Why? Clean this up.
|
|
- [ ] Deployability
|
|
- [ ] Clean up requirements.txt and upgrade packages
|
|
- [ ] Clean up Dockerfile
|
|
|
|
Make the models sound better
|
|
|
|
- [ ] Arpabet
|
|
- [ ] G2P
|
|
- [ ] Arpabet custom dictionary
|
|
- [ ] User-supplied Arpabet strings in braces
|
|
|
|
- [ ] Word break down 888 -> eight hundred eighty eight
|
|
|
|
- [ ] Support I18N
|
|
- [ ] Accent character support
|
|
|
|
Introduce new models
|
|
|
|
- [ ] TalkNet
|
|
- [ ] GlowTTS + MelGan (vocoder)
|
|
- [ ] Make these work under the same venv
|
|
|
|
Lots more I haven't covered yet...
|