Skip to content

Commit cb76056

Browse files
set stop_gradient (#5178)
1 parent 26582d4 commit cb76056

File tree

1 file changed

+2
-0
lines changed
  • PaddleNLP/paddlenlp/transformers/transformer

1 file changed

+2
-0
lines changed

PaddleNLP/paddlenlp/transformers/transformer/modeling.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -276,8 +276,10 @@ def forward(self, src_word, trg_word):
276276
src_slf_attn_bias = paddle.cast(
277277
src_word == self.bos_id,
278278
dtype=paddle.get_default_dtype()).unsqueeze([1, 2]) * -1e9
279+
src_slf_attn_bias.stop_gradient = True
279280
trg_slf_attn_bias = self.transformer.generate_square_subsequent_mask(
280281
trg_max_len)
282+
trg_slf_attn_bias.stop_gradient = True
281283
trg_src_attn_bias = src_slf_attn_bias
282284
src_pos = paddle.cast(
283285
src_word != self.bos_id, dtype="int64") * paddle.arange(

0 commit comments

Comments
 (0)