storytold/FineTrainers-Conditioning
2
0
Fork 0
mirror of https://github.com/storytold/FineTrainers-Conditioning.git synced 2026-10-09 00:09:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

deepspeed note

Browse Source
This commit is contained in:
sayakpaul
2024-12-26 08:12:41 +05:30
parent 1d5e0566fb
commit 1594c1dade
1 changed files with 1 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+1
View File
@@ -385,6 +385,7 @@ If you would like to use a custom dataset, refer to the dataset preparation guid
> [!NOTE]
> To lower memory requirements:
> - Use a DeepSpeed config to launch training (refer to [`accelerate_configs/deepspeed.yaml`](./accelerate_configs/deepspeed.yaml) as an example).
> - Pass `--precompute_conditions` when launching training.
> - Pass `--gradient_checkpointing` when launching training.
> - Do not perform validation/testing. This saves a significant amount of memory, which can be used to focus solely on training if you're on smaller VRAM GPUs.