storytold/FineTrainers-Conditioning
2
0
Fork 0
mirror of https://github.com/storytold/FineTrainers-Conditioning.git synced 2026-10-18 07:33:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Full Finetuning for LTX possibily extended to other models. (#192)

Browse Source 
* Full Finetuning for LTX possibily extended to other models.

* Change name of the flag

* Used disable grad for component on lora fine tuning enabled

* Suggestions Addressed
Renamed to SFT
Added 2 other models.
Testing required.

* Switching to Full FineTuning

* Run linter.

* parse subfolder when needed.

* tackle saving and loading hooks.

* tackle validation.

* fix subfolder bug.

* remove __class__.

* refactor

* remove unnecessary changes

* handle saving of final model weights correctly

* remove unnecessary changes

* LTX uses a default frame rate of 24 FPS
We need to modify the output validation framerate to match that value.
Add Framerate args.
Add Update video output and inference frame rate

* There was a results_args mapping that needed to be modified.

* update

* update README

* Update README.md

* update docs

* add training configuration in cogvideox

---------

Co-authored-by: Sayak Paul <spsayakpaul@gmail.com>
Co-authored-by: Aryan <aryan@huggingface.co>
Co-authored-by: Aryan <contact.aryanvs@gmail.com>
This commit is contained in:
ArEnSc
2025-01-13 13:25:39 -05:00
committed by GitHub
parent a3898075d1
commit e5df80cc36
17 changed files with 365 additions and 131 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/training/cogvideox.md
+29
View File
@@ -2,6 +2,8 @@
## Training
For LoRA training, specify `--training_type lora`. For full finetuning, specify `--training_type full-finetune`.
```bash
#!/bin/bash
export WANDB_MODE="offline"
@@ -84,6 +86,8 @@ echo -ne "-------------------- Finished executing script --------------------\n\
## Memory Usage
### LoRA
LoRA with rank 128, batch size 1, gradient checkpointing, optimizer adamw, `49x480x720` resolutions, **with precomputation**:
```
@@ -109,6 +113,31 @@ Training configuration: {
| after validation end | 11.145 | 28.324 |
| after training end | 11.144 | 11.592 |
### Full finetuning
```
Training configuration: {
"trainable parameters": 5570283072,
"total samples": 1,
"train epochs": 2,
"train steps": 2,
"batches per device": 1,
"total batches observed per epoch": 1,
"train batch size": 1,
"gradient accumulation steps": 1
}
```
| stage | memory_allocated | max_memory_reserved |
|:-----------------------------:|:-----------------:|:-------------------:|
| after precomputing conditions | 8.880 | 8.941 |
| after precomputing latents | 9.300 | 12.441 |
| before training start | 10.376 | 10.387 |
| after epoch 1 | 31.160 | 52.939 |
| before validation start | 31.161 | 52.939 |
| after validation end | 31.161 | 52.939 |
| after training end | 31.160 | 34.295 |
## Supported checkpoints
CogVideoX has multiple checkpoints as one can note [here](https://huggingface.co/collections/THUDM/cogvideo-66c08e62f1685a3ade464cce). The following checkpoints were tested with `finetrainers` and are known to be working: