mirror of
https://github.com/storytold/storyteller-ml.git
synced 2026-10-16 00:13:07 +00:00
Introduce code from Rust monorepo. This is a mess
This commit is contained in:
Brandon Thomas
@@ -0,0 +1,20 @@
|
||||
Dataset Creation Tool Based on CTC-Segmentation
|
||||
===============================================
|
||||
|
||||
This tool provides functionality to align long audio files and the corresponding transcripts into shorter fragments
|
||||
that are suitable for an Automatic Speech Recognition (ASR) model training.
|
||||
|
||||
More details could be found in `NeMo/tutorials/tools/CTC_Segmentation_Tutorial.ipynb <https://github.com/NVIDIA/NeMo/blob/main/tutorials/tools/CTC_Segmentation_Tutorial.ipynb>`__ (can be executed with `Google's Colab <https://colab.research.google.com/notebooks/intro.ipynb>`_).
|
||||
|
||||
The tool is based on the `CTC-Segmentation <https://github.com/lumaku/ctc-segmentation>`__ package and
|
||||
`CTC-Segmentation of Large Corpora for German End-to-end Speech Recognition
|
||||
<https://arxiv.org/abs/2007.09127>`__ :cite:`tools-kurzinger2020ctc`
|
||||
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
.. bibliography:: tools_all.bib
|
||||
:style: plain
|
||||
:labelprefix: TOOLS
|
||||
:keyprefix: tools-
|
||||
@@ -0,0 +1,14 @@
|
||||
Tools
|
||||
=====
|
||||
|
||||
NeMo provides a set of tools useful for developing Automatic Speech Recognitions (ASR) and Text-to-Speech (TTS) models: \
|
||||
`https://github.com/NVIDIA/NeMo/tree/main/tools <https://github.com/NVIDIA/NeMo/tree/main/tools>`__ .
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
ctc_segmentation
|
||||
speech_data_explorer
|
||||
text_processing_deployment
|
||||
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
Speech Data Explorer
|
||||
====================
|
||||
|
||||
.. note::
|
||||
|
||||
The tool could be found under `NeMo/tools/speech_data_explorer <https://github.com/NVIDIA/NeMo/tree/main/tools/speech_data_explorer>`__.
|
||||
|
||||
`Dash <https://plotly.com/dash/>`__-based tool for interactive exploration of ASR/TTS datasets.
|
||||
|
||||
Features:
|
||||
|
||||
* dataset's statistics (alphabet, vocabulary, duration-based histograms)
|
||||
* navigation across dataset (sorting, filtering)
|
||||
* inspection of individual utterances (waveform, spectrogram, audio player)
|
||||
* errors' analysis (Word Error Rate, Character Error Rate, Word Match Rate, Mean Word Accuracy, diff)
|
||||
|
||||
Please make sure that requirements are installed. Then run:
|
||||
|
||||
.. code::
|
||||
|
||||
python data_explorer.py path_to_manifest.json
|
||||
|
||||
|
||||
JSON manifest file should contain the following fields:
|
||||
|
||||
* `audio_filepath` (path to audio file)
|
||||
* `duration` (duration of the audio file in seconds)
|
||||
* `text` (reference transcript)
|
||||
|
||||
Errors' analysis requires "pred_text" (ASR transcript) for all utterances.
|
||||
|
||||
Any additional field will be parsed and displayed in 'Samples' tab.
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
NeMo Text Processing Deployment
|
||||
===============================================
|
||||
|
||||
This tool deploys :doc:`NeMo Inverse Text Normalization (ITN) <../nemo_text_processing/inverse_text_normalization>` and :doc:`NeMo Text Normalization (TN) <../nemo_text_processing/text_normalization>` for production :cite:`tools-itn_deploy-zhang2021nemo`.
|
||||
It uses `Sparrowhawk <https://github.com/google/sparrowhawk>`_ :cite:`tools-itn_deploy-sparrowhawk` -- an open-source version of Google Kestrel :cite:`tools-itn_deploy-ebden2015kestrel`.
|
||||
|
||||
Requirements
|
||||
------------------------
|
||||
|
||||
:doc:`nemo_text_processing <../nemo_text_processing/intro>` package
|
||||
|
||||
|
||||
Usage
|
||||
------------
|
||||
|
||||
Starts docker container with production backend with plugged in grammars. This is entry point script.
|
||||
|
||||
Arguments:
|
||||
^^^^^^^^^
|
||||
* ``GRAMMARS`` - ``tn_grammars`` or ``itn_grammars`` to export either TN or ITN grammars from :doc:`nemo_text_processing <../nemo_text_processing/intro>`.
|
||||
* ``INPUT_CASE`` - ``cased`` or ``lower_cased`` (lower_cased is supported only in TN grammars).
|
||||
* ``MODE`` - choose ``test`` to run test on the grammars inside the container.
|
||||
|
||||
For example:
|
||||
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# to export ITN grammars
|
||||
bash export_grammar.sh --GRAMMARS=itn_grammars
|
||||
|
||||
# to export and test TN grammars
|
||||
bash export_grammar.sh --GRAMMARS=itn_grammars --INPUT_CASE=cased --MODE=test
|
||||
|
||||
This script runs the following steps in sequence:
|
||||
|
||||
Exports grammar `ClassifyFst` and `VerbalizeFst` from :doc:`nemo_text_processing <../nemo_text_processing/intro>` to `OUTPUT_DIR/classify/tokenize_and_classify.far` and `OUTPUT_DIR/verbalize/verbalize.far` respectively.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
python pynini_export.py <--output_dir OUTPUT_DIR> <--grammars GRAMMARS> <--input_case INPUT_CASE>
|
||||
|
||||
Builds C++ production backend docker
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
bash docker/build.sh
|
||||
|
||||
|
||||
Plugs in grammars into production backend by mounting grammar directory `classify/` and `verbalize/` with sparrowhawk grammar directory inside docker. Returns docker prompt
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# to launch container with the exported grammars
|
||||
bash docker/launch.sh
|
||||
|
||||
# to launch container with the exported grammars and run tests on TN grammars
|
||||
bash docker/launch.sh test_tn_grammars
|
||||
|
||||
# to launch container with the exported grammars and run tests on ITN grammars
|
||||
bash docker/launch.sh test_itn_grammars
|
||||
|
||||
|
||||
Runs TN or ITN in docker container:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo "two dollars fifty" | ../../src/bin/normalizer_main --config=sparrowhawk_configuration.ascii_proto
|
||||
|
||||
This returns $2.50 for ITN.
|
||||
|
||||
References
|
||||
----------
|
||||
|
||||
.. bibliography:: tools_all.bib
|
||||
:style: plain
|
||||
:labelprefix: TOOLS-ITN_DEPLOY
|
||||
:keyprefix: tools-itn_deploy-
|
||||
@@ -0,0 +1,49 @@
|
||||
@inproceedings{kurzinger2020ctc,
|
||||
title={CTC-segmentation of large corpora for german end-to-end speech recognition},
|
||||
author={K{\"u}rzinger, Ludwig and Winkelbauer, Dominik and Li, Lujun and Watzel, Tobias and Rigoll, Gerhard},
|
||||
booktitle={International Conference on Speech and Computer},
|
||||
pages={267--278},
|
||||
year={2020},
|
||||
organization={Springer}
|
||||
}
|
||||
|
||||
@article{ebden2015kestrel,
|
||||
title={The Kestrel TTS text normalization system},
|
||||
author={Ebden, Peter and Sproat, Richard},
|
||||
journal={Natural Language Engineering},
|
||||
volume={21},
|
||||
number={3},
|
||||
pages={333},
|
||||
year={2015},
|
||||
publisher={Cambridge University Press}
|
||||
}
|
||||
|
||||
@article{sproat2016rnn,
|
||||
title={RNN approaches to text normalization: A challenge},
|
||||
author={Sproat, Richard and Jaitly, Navdeep},
|
||||
journal={arXiv preprint arXiv:1611.00068},
|
||||
year={2016}
|
||||
}
|
||||
|
||||
@book{taylor2009text,
|
||||
title={Text-to-speech synthesis},
|
||||
author={Taylor, Paul},
|
||||
year={2009},
|
||||
publisher={Cambridge university press}
|
||||
}
|
||||
|
||||
@misc{zhang2021nemo,
|
||||
title={NeMo Inverse Text Normalization: From Development To Production},
|
||||
author={Yang Zhang and Evelina Bakhturina and Kyle Gorman and Boris Ginsburg},
|
||||
year={2021},
|
||||
eprint={2104.05055},
|
||||
archivePrefix={arXiv},
|
||||
primaryClass={cs.CL}
|
||||
}
|
||||
|
||||
@inproceedings{sparrowhawk,
|
||||
title = {TTS for Low Resource Languages: A Bangla Synthesizer},
|
||||
author = {Alexander Gutkin and Linne Ha and Martin Jansche and Knot Pipatsrisawat and Richard Sproat},
|
||||
booktitle = {10th Language Resources and Evaluation Conference},
|
||||
year = {2016},
|
||||
}
|
||||
Reference in New Issue
Block a user