Skip to content

Commit 40b2e29

Browse files
jhpiedrahitaobaskaryan
jhpiedrahitao
and
baskaryan
authored
community[minor]: Sambanova llm integration (#20955)
- **Description:** Added [Sambanova systems](https://sambanova.ai/) integration, including sambaverse and sambastudio LLMs - **Dependencies:** sseclient-py (optional) --------- Co-authored-by: Bagatur <22008038+baskaryan@users.noreply.github.com> Co-authored-by: Bagatur <baskaryan@gmail.com>
1 parent 955cf18 commit 40b2e29

File tree

3 files changed

+1105
-0
lines changed

3 files changed

+1105
-0
lines changed

‎docs/docs/integrations/llms/sambanova.ipynb‎

Lines changed: 212 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,212 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Sambanova\n",
8+
"\n",
9+
"**[Sambanova](https://sambanova.ai/)'s** [Sambaverse](https://sambaverse.sambanova.ai/) and [Sambastudio](https://sambanova.ai/technology/full-stack-ai-platform) are platforms for running your own open source models\n",
10+
"\n",
11+
"This example goes over how to use LangChain to interact with Sambanova models"
12+
]
13+
},
14+
{
15+
"cell_type": "markdown",
16+
"metadata": {},
17+
"source": [
18+
"## Sambaverse"
19+
]
20+
},
21+
{
22+
"cell_type": "markdown",
23+
"metadata": {},
24+
"source": [
25+
"**Sambaverse** allows you to interact with multiple Open source models you can se the list of available models an interact with then in the [playground](https://sambaverse.sambanova.ai/playground)"
26+
]
27+
},
28+
{
29+
"cell_type": "markdown",
30+
"metadata": {},
31+
"source": [
32+
"An API key is required to access to Sambaverse models get one creating an account in [sambaverse.sambanova.ai](https://sambaverse.sambanova.ai/)\n",
33+
"\n",
34+
"The [sseclient-py](https://pypi.org/project/sseclient-py/) package is required to run streaming predictions "
35+
]
36+
},
37+
{
38+
"cell_type": "code",
39+
"execution_count": null,
40+
"metadata": {},
41+
"outputs": [],
42+
"source": [
43+
"%pip install --quiet sseclient-py==1.8.0"
44+
]
45+
},
46+
{
47+
"cell_type": "markdown",
48+
"metadata": {},
49+
"source": [
50+
"Register your API Key environment variable:"
51+
]
52+
},
53+
{
54+
"cell_type": "code",
55+
"execution_count": 4,
56+
"metadata": {},
57+
"outputs": [],
58+
"source": [
59+
"import os\n",
60+
"\n",
61+
"sambaverse_api_key = \"<Your sambaverse API key>\"\n",
62+
"\n",
63+
"# Set the environment variables\n",
64+
"os.environ[\"SAMBAVERSE_API_KEY\"] = sambaverse_api_key"
65+
]
66+
},
67+
{
68+
"cell_type": "markdown",
69+
"metadata": {},
70+
"source": [
71+
"Call Sambaverse models directly from langchain!"
72+
]
73+
},
74+
{
75+
"cell_type": "code",
76+
"execution_count": null,
77+
"metadata": {},
78+
"outputs": [],
79+
"source": [
80+
"from langchain_community.llms.sambanova import Sambaverse\n",
81+
"\n",
82+
"llm = Sambaverse(\n",
83+
" sambaverse_model_name=\"Meta/llama-2-7b-chat-hf\",\n",
84+
" streaming=False,\n",
85+
" model_kwargs={\n",
86+
" \"do_sample\": True,\n",
87+
" \"max_tokens_to_generate\": 1000,\n",
88+
" \"temperature\": 0.01,\n",
89+
" \"process_prompt\": True,\n",
90+
" \"select_expert\": \"llama-2-7b-chat-hf\",\n",
91+
" # \"repetition_penalty\": {\"type\": \"float\", \"value\": \"1\"},\n",
92+
" # \"top_k\": {\"type\": \"int\", \"value\": \"50\"},\n",
93+
" # \"top_p\": {\"type\": \"float\", \"value\": \"1\"}\n",
94+
" },\n",
95+
")\n",
96+
"\n",
97+
"print(llm.invoke(\"Why should I use open source models?\"))"
98+
]
99+
},
100+
{
101+
"cell_type": "markdown",
102+
"metadata": {},
103+
"source": [
104+
"## SambaStudio"
105+
]
106+
},
107+
{
108+
"cell_type": "markdown",
109+
"metadata": {},
110+
"source": [
111+
"**SambaStudio** allows you to Train, run batch inference jous, and deploy online inference endpoints to run your own fine tunned open source models"
112+
]
113+
},
114+
{
115+
"cell_type": "markdown",
116+
"metadata": {},
117+
"source": [
118+
"A SambaStudio environment is required to deploy a model. Get more information in [sambanova.ai/products/enterprise-ai-platform-sambanova-suite](https://sambanova.ai/products/enterprise-ai-platform-sambanova-suite)\n",
119+
"\n",
120+
"The [sseclient-py](https://pypi.org/project/sseclient-py/) package is required to run streaming predictions "
121+
]
122+
},
123+
{
124+
"cell_type": "code",
125+
"execution_count": null,
126+
"metadata": {},
127+
"outputs": [],
128+
"source": [
129+
"%pip install --quiet sseclient-py==1.8.0"
130+
]
131+
},
132+
{
133+
"cell_type": "markdown",
134+
"metadata": {},
135+
"source": [
136+
"Register your environment variables:"
137+
]
138+
},
139+
{
140+
"cell_type": "code",
141+
"execution_count": null,
142+
"metadata": {},
143+
"outputs": [],
144+
"source": [
145+
"import os\n",
146+
"\n",
147+
"sambastudio_base_url = \"<Your SambaStudio environment URL>\"\n",
148+
"sambastudio_project_id = \"<Your SambaStudio project id>\"\n",
149+
"sambastudio_endpoint_id = \"<Your SambaStudio endpoint id>\"\n",
150+
"sambastudio_api_key = \"<Your SambaStudio endpoint API key>\"\n",
151+
"\n",
152+
"# Set the environment variables\n",
153+
"os.environ[\"SAMBASTUDIO_BASE_URL\"] = sambastudio_base_url\n",
154+
"os.environ[\"SAMBASTUDIO_PROJECT_ID\"] = sambastudio_project_id\n",
155+
"os.environ[\"SAMBASTUDIO_ENDPOINT_ID\"] = sambastudio_endpoint_id\n",
156+
"os.environ[\"SAMBASTUDIO_API_KEY\"] = sambastudio_api_key"
157+
]
158+
},
159+
{
160+
"cell_type": "markdown",
161+
"metadata": {},
162+
"source": [
163+
"Call SambaStudio models directly from langchain!"
164+
]
165+
},
166+
{
167+
"cell_type": "code",
168+
"execution_count": null,
169+
"metadata": {},
170+
"outputs": [],
171+
"source": [
172+
"from langchain_community.llms.sambanova import SambaStudio\n",
173+
"\n",
174+
"llm = SambaStudio(\n",
175+
" streaming=False,\n",
176+
" model_kwargs={\n",
177+
" \"do_sample\": True,\n",
178+
" \"max_tokens_to_generate\": 1000,\n",
179+
" \"temperature\": 0.01,\n",
180+
" # \"repetition_penalty\": {\"type\": \"float\", \"value\": \"1\"},\n",
181+
" # \"top_k\": {\"type\": \"int\", \"value\": \"50\"},\n",
182+
" # \"top_logprobs\": {\"type\": \"int\", \"value\": \"0\"},\n",
183+
" # \"top_p\": {\"type\": \"float\", \"value\": \"1\"}\n",
184+
" },\n",
185+
")\n",
186+
"\n",
187+
"print(llm.invoke(\"Why should I use open source models?\"))"
188+
]
189+
}
190+
],
191+
"metadata": {
192+
"kernelspec": {
193+
"display_name": "Python 3 (ipykernel)",
194+
"language": "python",
195+
"name": "python3"
196+
},
197+
"language_info": {
198+
"codemirror_mode": {
199+
"name": "ipython",
200+
"version": 3
201+
},
202+
"file_extension": ".py",
203+
"mimetype": "text/x-python",
204+
"name": "python",
205+
"nbconvert_exporter": "python",
206+
"pygments_lexer": "ipython3",
207+
"version": "3.9.1"
208+
}
209+
},
210+
"nbformat": 4,
211+
"nbformat_minor": 4
212+
}

0 commit comments

Comments
 (0)