mirror of
https://github.com/storytold/FineTrainers-Conditioning.git
synced 2026-10-09 00:09:45 +00:00
Sayak Paul
eeb4dd7afa
* refactor docs for easier info parsing * refactor readme * anonym paths * updates * remove notes. * add a toc. * minor typos. * move cog.md -> cogvideox.md * add a note about the model-specific docs in training readme. * add memory usage for CogVideoX. Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com> * change to 5b from 2b for CogVideoX. Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com> * more appropriate names. * add headers to the model docs. * fix adapter name * minor * updates * fix cog training command example --------- Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
9 lines
658 B
Markdown
9 lines
658 B
Markdown
To lower memory requirements during training:
|
|
|
|
- Use a DeepSpeed config to launch training (refer to [`accelerate_configs/deepspeed.yaml`](./accelerate_configs/deepspeed.yaml) as an example).
|
|
- Pass `--precompute_conditions` when launching training.
|
|
- Pass `--gradient_checkpointing` when launching training.
|
|
- Pass `--use_8bit_bnb` when launching training. Note that this is only applicable to Adam and AdamW optimizers.
|
|
- Do not perform validation/testing. This saves a significant amount of memory, which can be used to focus solely on training if you're on smaller VRAM GPUs.
|
|
|
|
We will continue to add more features that help to reduce memory consumption. |