storytold/FineTrainers-Conditioning
2
0
Fork 0
mirror of https://github.com/storytold/FineTrainers-Conditioning.git synced 2026-10-09 00:09:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

Used disable grad for component on lora fine tuning enabled

Browse Source
This commit is contained in:
CrossProduct
2025-01-07 20:41:12 +00:00
parent f0db0ccb00
commit 4cd5a8e53b
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
finetrainers/trainer.py
+2 -1
View File
@@ -390,7 +390,8 @@ class Trainer:
self._enable_grad_for_components(components=[self.transformer])
else:
logger.info("Lora Fine Tuning Enabled")
self._disable_grad_for_components(components=[self.transformer])
# For mixed precision training we cast all non-trainable weights (vae, text_encoder and transformer) to half-precision
# as these weights are only used for inference, keeping weights in full precision is not required.
weight_dtype = self._get_training_dtype(accelerator=self.state.accelerator)