Inference chat_completion uses deprecated conversational task

[dpsommer]

Describe the bug

The InferenceClient and AsyncInferenceClient both hardcode the conversational task when creating the TaskProviderHelper. Since the conversational task is now deprecated, many models use the more general text-generation task, which causes chat_completion inference to fail.

AsyncInferenceClient call to get_provider_helper:

huggingface_hub/src/huggingface_hub/inference/_generated/_async_client.py

Line 926 in caeaeeb

provider_helper = get_provider_helper(self.provider, task="conversational", model=payload_model)

Reproduction

# Only slightly modified from the Chat Completion example:
# https://huggingface.co/docs/inference-providers/en/tasks/chat-completion#code-snippet-example-for-conversational-llms
import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    api_key=os.environ["HUGGINGFACEHUB_API_TOKEN"],
)

completion = client.chat.completions.create(
    model="SanjiWatsuki/Kunoichi-DPO-v2-7B",
    messages=[
        {
            "role": "user",
            "content": "How are you today?"
        }
    ],
)

print(completion.choices[0].message)

Logs

Traceback (most recent call last):
  File "/home/user/dev/python/test/repro.py", line 8, in <module>
    completion = client.chat.completions.create(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/venv/lib/python3.12/site-packages/huggingface_hub/inference/_client.py", line 912, in chat_completion
    request_parameters = provider_helper.prepare_request(
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/venv/lib/python3.12/site-packages/huggingface_hub/inference/_providers/_common.py", line 92, in prepare_request
    provider_mapping_info = self._prepare_mapping_info(model)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.pyenv/versions/venv/lib/python3.12/site-packages/huggingface_hub/inference/_providers/_common.py", line 170, in _prepare_mapping_info
    raise ValueError(
ValueError: Model SanjiWatsuki/Kunoichi-DPO-v2-7B is not supported for task conversational and provider featherless-ai. Supported task: text-generation.

System info

- huggingface-hub version: 0.35.3
- Python version: 3.12.8

[Wauplin]

Since the conversational task is now deprecated

What do you mean by that?

"conversational" tag is added to any text-generation or image-text-to-text model that has a chat template. So it's not deprecated and working fine

[dpsommer]

What do you mean by that?

"conversational" tag is added to any text-generation or image-text-to-text model that has a chat template. So it's not deprecated and working fine

It's my understanding that the Conversational task has been removed in favour of using Text Generation, reference: https://discuss.huggingface.co/t/conversational-task-is-deprecated/93008/3

[dpsommer]

I see what you're saying - the inference providers still use the conversational task if a chat_template is defined in the tokenizer config. I had assumed that provider and model tasks were the same.

To the point made in that thread, a lot of text generation models can still be used for conversational purposes without being instruct models; is the alternative to just use text_generation instead?

[Wauplin]

I had assumed that provider and model tasks were the same.

They are, except for that one particular case :)

To the point made in that thread, a lot of text generation models can still be used for conversational purposes without being instruct models; is the alternative to just use text_generation instead?

Yes if no chat template is provided, you must use text_generation and prompt the model correctly to make it generate a chat response. That highly depends on the base model.

[dpsommer]

@Wauplin That makes sense. Appreciate the help!