You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
`inference.py`: Python example of sending inference requests to inference server using OpenAI API, make sure to install OpenAI API in your environment.
3
-
4
-
`inference.sh`: Bash example of sending inference requests to inference server, supports JSON mode
5
-
6
-
`logits.py`: Python example of getting logits from hosted model.
2
+
-[`inference`](inference): Examples for sending inference requests
3
+
-[`llm/chat_completions.py`](inference/llm/chat_completions.py): Python example of sending chat completion requests to OpenAI compatible server
4
+
-[`llm/completions.py`](inference/llm/completions.py): Python example of sending completion requests to OpenAI compatible server
5
+
-[`llm/completions.sh`](inference/llm/completions.sh): Bash example of sending completion requests to OpenAI compatible server, supports JSON mode
6
+
-[`vlm/vision_completions.py`](inference/vlm/vision_completions.py): Python example of sending chat completion requests with image attached to prompt to OpenAI compatible server for vision language models
7
+
-[`logits`](logits): Example for logits generation
8
+
-[`logits.py`](logits/logits.py): Python example of getting logits from hosted model.
0 commit comments