kl项

[NieSYsc20]

Hi. I noticed that both KL_LOSS_COEF=0.001 and KL_COEF=0.001 are set in the training script. Does this mean that the KL term is being applied in both the reward function and the loss function?

Laser/scripts/training/laser-d-1.5b/laser-d-1.5b-l2048.sh

Line 36 in c9e7c4e

KL_LOSS_COEF=0.001

Laser/scripts/training/laser-d-1.5b/laser-d-1.5b-l2048.sh

Line 42 in c9e7c4e

KL_COEF=0.001

[VPeterV]

Hi, thanks for your question!

In VERL, the default setting for GRPO disables the KL term in the reward function. As a result, the kl_coef parameter has no effect under this setting.