Skip to content

Commit 828a27f

Browse files
authored
Fix gpt-oss rope warning (#40550)
* fix * fix print * rm * real fix * fix * style
1 parent 74a2421 commit 828a27f

File tree

1 file changed

+8
-1
lines changed

1 file changed

+8
-1
lines changed

src/transformers/models/gpt_oss/configuration_gpt_oss.py

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,14 @@ def __init__(
6262
initializer_range: float = 0.02,
6363
max_position_embeddings=131072,
6464
rms_norm_eps: float = 1e-5,
65-
rope_scaling={"rope_type": "yarn", "factor": 32.0, "beta_fast": 32.0, "beta_slow": 1.0, "truncate": False},
65+
rope_scaling={
66+
"rope_type": "yarn",
67+
"factor": 32.0,
68+
"beta_fast": 32.0,
69+
"beta_slow": 1.0,
70+
"truncate": False,
71+
"original_max_position_embeddings": 4096,
72+
},
6673
attention_dropout: float = 0.0,
6774
num_experts_per_tok=4,
6875
router_aux_loss_coef: float = 0.9,

0 commit comments

Comments
 (0)