Skip to content

Commit 6bf24ac

Browse files
DOC Update LoRA dev guides wrt to unloading (#2871)
1 parent f4763a0 commit 6bf24ac

File tree

1 file changed

+4
-4
lines changed
  • docs/source/developer_guides

1 file changed

+4
-4
lines changed

docs/source/developer_guides/lora.md

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -511,10 +511,10 @@ from peft import PeftModel
511511
base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-v0.1")
512512
peft_model_id = "alignment-handbook/zephyr-7b-sft-lora"
513513
model = PeftModel.from_pretrained(base_model, peft_model_id)
514-
model.merge_and_unload()
514+
model = model.merge_and_unload()
515515
```
516516

517-
If you need to keep a copy of the weights so you can unmerge the adapter later or delete and load different ones, you should use the [`~LoraModel.merge_adapter`] function instead. Now you have the option to use [`~LoraModel.unmerge_adapter`] to return the base model.
517+
It is important to assign the returned model to a variable and use it, [`~LoraModel.merge_and_unload`] is not an in-place operation. If you need to keep a copy of the weights so you can unmerge the adapter later or delete and load different ones, you should use the [`~LoraModel.merge_adapter`] function instead. Now you have the option to use [`~LoraModel.unmerge_adapter`] to return the base model.
518518

519519
```py
520520
from transformers import AutoModelForCausalLM
@@ -603,11 +603,11 @@ model.load_adapter("alignment-handbook/zephyr-7b-dpo-lora", adapter_name="dpo")
603603
model.set_adapter("dpo")
604604
```
605605

606-
To return the base model, you could use [`~LoraModel.unload`] to unload all of the LoRA modules or [`~LoraModel.delete_adapter`] to delete the adapter entirely.
606+
To return the base model, you could use [`~LoraModel.unload`] to unload all of the LoRA modules or [`~LoraModel.delete_adapter`] to delete the adapter entirely. [`~LoraModel.unload`] is not an in-place operation, remember to assign the returned model to a variable and use it.
607607

608608
```py
609609
# unload adapter
610-
model.unload()
610+
model = model.unload()
611611

612612
# delete adapter
613613
model.delete_adapter("dpo")

0 commit comments

Comments
 (0)