storytold/FineTrainers-Conditioning
2
0
Fork 0
mirror of https://github.com/storytold/FineTrainers-Conditioning.git synced 2026-10-09 00:09:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

[docs] refactor docs for easier info parsing (#175)

Browse Source 
* refactor docs for easier info parsing

* refactor readme

* anonym paths

* updates

* remove notes.

* add a toc.

* minor typos.

* move cog.md -> cogvideox.md

* add a note about the model-specific docs in training readme.

* add memory usage for CogVideoX.

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* change to 5b from 2b for CogVideoX.

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>

* more appropriate names.

* add headers to the model docs.

* fix adapter name

* minor

* updates

* fix cog training command example

---------

Co-authored-by: a-r-r-o-w <contact.aryanvs@gmail.com>
This commit is contained in:
Sayak Paul
2025-01-05 20:50:30 +05:30
committed by GitHub
parent b3d0ba8e12
commit eeb4dd7afa
7 changed files with 535 additions and 545 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/training/optimization.md
+9
View File
@@ -0,0 +1,9 @@
To lower memory requirements during training:
- Use a DeepSpeed config to launch training (refer to [`accelerate_configs/deepspeed.yaml`](./accelerate_configs/deepspeed.yaml) as an example).
- Pass `--precompute_conditions` when launching training.
- Pass `--gradient_checkpointing` when launching training.
- Pass `--use_8bit_bnb` when launching training. Note that this is only applicable to Adam and AdamW optimizers.
- Do not perform validation/testing. This saves a significant amount of memory, which can be used to focus solely on training if you're on smaller VRAM GPUs.
We will continue to add more features that help to reduce memory consumption.