storytold/FineTrainers-Conditioning
2
0
Fork 0
mirror of https://github.com/storytold/FineTrainers-Conditioning.git synced 2026-10-18 07:33:20 +00:00
Code Issues Packages Projects Releases Wiki Activity

Used disable grad for component on lora fine tuning enabled

Browse Source
This commit is contained in:
CrossProduct
2025-01-07 20:41:12 +00:00
parent f0db0ccb00
commit 4cd5a8e53b
1 changed files with 2 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
finetrainers/trainer.py
+1
View File
@@ -390,6 +390,7 @@ class Trainer:
self._enable_grad_for_components(components=[self.transformer]) self._enable_grad_for_components(components=[self.transformer])
else: else:
logger.info("Lora Fine Tuning Enabled") logger.info("Lora Fine Tuning Enabled")
self._disable_grad_for_components(components=[self.transformer])
# For mixed precision training we cast all non-trainable weights (vae, text_encoder and transformer) to half-precision # For mixed precision training we cast all non-trainable weights (vae, text_encoder and transformer) to half-precision
# as these weights are only used for inference, keeping weights in full precision is not required. # as these weights are only used for inference, keeping weights in full precision is not required.