Skip to content

[docs] Create page on inference servers with transformers backend #39550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 31 commits into from
Jul 22, 2025
Merged

[docs] Create page on inference servers with transformers backend #39550

merged 31 commits into from
Jul 22, 2025

Conversation

zucchini-nlp
Copy link
Member

@zucchini-nlp zucchini-nlp commented Jul 21, 2025 •
edited
Loading

What does this PR do?

As per title, I added the basic info about existing inference engines so feel free to add more examples/tips etc. This PR creates a space where we can host docs on all third-party servers and we can submit PRs in vLLM/SGLang/TGI pointing to this page

@HuggingFaceDocBuilderDev
Copy link

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

hmellor
hmellor reviewed Jul 21, 2025
View reviewed changes
zucchini-nlp and others added 2 commits July 21, 2025 19:27
@zucchini-nlp @hmellor
Update docs/source/en/_toctree.yml
59c2d5e 
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@zucchini-nlp
update
bfa5fba
zucchini-nlp
zucchini-nlp commented Jul 21, 2025
View reviewed changes
docs/source/en/_toctree.yml Outdated
Comment on lines 106 to 109
- local: serving
title: Serving
- local: transformers_as_backend
title: Transformers as a Unified Modeling Backend
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we can put docs here, I see Agents and Tools are deprecated. Though I can't say it belong in LLMs, it is more about generative models overall

@stevhliu or should we rename the LLMs section to smth like Text Generation models or simlar?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its ok to leave the LLM section as is and have serving.md and transformers_as_backend.md hang out in the overall Inference section

stevhliu
stevhliu reviewed Jul 21, 2025
View reviewed changes
Copy link
Member

@stevhliu stevhliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks!

docs/source/en/_toctree.yml Outdated
Comment on lines 106 to 109
- local: serving
title: Serving
- local: transformers_as_backend
title: Transformers as a Unified Modeling Backend
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think its ok to leave the LLM section as is and have serving.md and transformers_as_backend.md hang out in the overall Inference section

zucchini-nlp and others added 20 commits July 22, 2025 09:16
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
15b1ce8 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/_toctree.yml
0f48ecc 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
a04e981 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
836d217 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
1909288 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
9b45483 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
9be800a 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
33e01e5 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
b8e7401 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
6a38ba6 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
37b13ac 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
bdddf56 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
5e305df 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
70acaac 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
bcd171e 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
228396e 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
d1e3b97 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
99fd3ec 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
031ead8 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
0b056a0 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
zucchini-nlp and others added 6 commits July 22, 2025 09:24
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
f53a804 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
2bd29a4 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
fd0a96d 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Update docs/source/en/transformers_as_backend.md
581e4dc 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp @stevhliu
Apply suggestions from code review
6895d06 
Co-authored-by: Steven Liu <59462357+stevhliu@users.noreply.github.com>
@zucchini-nlp
apply last suggestions
756c5b2
ArthurZucker
ArthurZucker approved these changes Jul 22, 2025
View reviewed changes
Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 🤗

@zucchini-nlp zucchini-nlp merged commit 1806583 into huggingface:main Jul 22, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hmellor hmellor hmellor left review comments

@stevhliu stevhliu stevhliu left review comments

@ArthurZucker ArthurZucker ArthurZucker approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@zucchini-nlp @HuggingFaceDocBuilderDev @hmellor @ArthurZucker @stevhliu