LORAX_USE_GLOBAL_HF_TOKEN is not applied at the first time of calling adapter from huggingface private hub #541
Description
System Info
ghcr.io/predibase/lorax:24cb494
Information
- Docker
- The CLI directly
Tasks
- An officially supported command
- My own modifications
Reproduction
- I've set
HUGGING_FACE_HUB_TOKENas environment when running docker container - Also set
LORAX_USE_GLOBAL_HF_TOKEN=1as environment - I'm pulling adapter from huggingface private hub.
curl http://XXXX/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "PRIVATE_HF_HUB_PATH",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant who gives detailed informations"
},
{
"role": "user",
"content": "Hello!"
}
]
}'
I've got the error below.
{"error":"Request failed during generation: Server error: No valid adapter config file found: tried None and None","error_type":"generation"}%
Reason why this happens
Expected behavior
I should return the result correctly even though I don't send api_token on request