Support intern-s1

[RunningLeon]

Support internlm/Intern-S1

[CISC]

The Python Type-Check CI needs to be resolved.

[RunningLeon]

The Python Type-Check CI needs to be resolved.

@CISC hi, could you tell how to fix this error? Seems not reasonable to me

/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:3219:23 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:3220:13 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:3220:49 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:32[21](https://github.com/ggml-org/llama.cpp/actions/runs/16612224904/job/46997396567?pr=14875#step:5:22):23 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:3[22](https://github.com/ggml-org/llama.cpp/actions/runs/16612224904/job/46997396567?pr=14875#step:5:23)2:13 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
/home/runner/work/llama.cpp/llama.cpp/convert_hf_to_gguf.py:3222:49 - error: Object of type "None" is not subscriptable (reportOptionalSubscript)
Error: Object of type "None" is not subscriptable (reportOptionalSubscript)
6 errors, 0 warnings, 14 informations
Error: 6 errors

[CISC]

The Python Type-Check CI needs to be resolved.

@CISC hi, could you tell how to fix this error? Seems not reasonable to me

Running pyright locally helps, the line numbers are wrong for some reason, this is the actual erroneous codeblock:

llama.cpp/convert_hf_to_gguf.py

Lines 3002 to 3005 in 5eba3e3

	if isinstance(self.hparams_vision['image_size'], list):
	self.hparams_vision['image_size'] = self.hparams_vision['image_size'][0]
	if isinstance(self.hparams_vision['patch_size'], list):
	self.hparams_vision['patch_size'] = self.hparams_vision['patch_size'][0]

[CISC]

This does not work because Intern-S1 requires custom code, you must handle that here and not call base class get_vocab_base.

The Intern-S1 tokenizer looks like it's fairly special, so I think it requires custom handling to work as intended, not just using AutoTokenizer.

[RunningLeon]

@CISC Hi, thanks for your reminder. Indeed, the intern-s1 tokenizer is special. It bases on Qwen3 bpe tokenizer, and expands with three spm tokenizer models. It uses some regex patterns to match to which sub vocab to use when tokenizing. Don't know how to implement it in llama.cpp. Do you have any suggestion for this special case? THX

[CISC]

No easy feat I'm afraid, SMILES especially, you will have to add a special case for it to do the sub-vocab matching and implement tokenizer in llama-vocab.cpp.

[RunningLeon]

Agree. After consideration, the sub-vocab feat would be not added in this PR.