-
Notifications
You must be signed in to change notification settings - Fork 17
Description
Hello,
I am trying to train the VQGAN on custom data. I believe I have everything setup correctly. I am running into the error below.
Has anyone seen this before?
Traceback (most recent call last):
File "main.py", line 752, in
trainer.fit(model, data)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 553, in fit
self._run(model)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 918, in _run
self._dispatch()
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 986, in _dispatch
self.accelerator.start_training(self)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 92, in start_training
self.training_type_plugin.start_training(trainer)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/training_type_plugin.py", line 161, in start_training
self._results = trainer.run_stage()
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 996, in run_stage
return self._run_train()
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1031, in _run_train
self._run_sanity_check(self.lightning_module)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py", line 1115, in _run_sanity_check
self._evaluation_loop.run()
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 111, in run
self.advance(*args, **kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 110, in advance
dl_outputs = self.epoch_loop.run(
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/loops/base.py", line 111, in run
self.advance(*args, **kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 110, in advance
output = self.evaluation_step(batch, batch_idx, dataloader_idx)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 154, in evaluation_step
output = self.trainer.accelerator.validation_step(step_kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/accelerators/accelerator.py", line 211, in validation_step
return self.training_type_plugin.validation_step(*step_kwargs.values())
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/plugins/training_type/ddp.py", line 386, in validation_step
return self.model(*args, **kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 963, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
return forward_call(*input, **kwargs)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/overrides/base.py", line 93, in forward
output = self.module.validation_step(*inputs, **kwargs)
File "/media/exx/Samsung8TB_2/sattwik/diffusion/latentdiffusion/stable-diffusion/src/taming-transformers/taming/models/vqgan.py", line 117, in validation_step
self.log_dict(log_dict_ae)
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 507, in log_dict
self.log(
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/core/lightning.py", line 446, in log
results.log(
File "/home/exx/.conda/envs/ldm/lib/python3.8/site-packages/pytorch_lightning/trainer/connectors/logger_connector/result.py", line 464, in log
raise MisconfigurationException(
pytorch_lightning.utilities.exceptions.MisconfigurationException: You called self.log(val/rec_loss, ...) twice in validation_step with different arguments. This is not allowed