Internal Server Error

[shobeg]

After successfully starting the docker for the basic rag and opening the webpage, the page shows an Internal Server error. And, the web console shows a 500 error trying to get /content/converse/?__theme=light.

[mrsaso]

Hi, what is the output when you check the chain-server logs? What is the specified error is it giving for the 500 Internal Server?
In terminal, run $ docker logs chain-server

[yxchia98]

Facing the same issue as well, seems like something to do with the rag-playground container on the frontend.
Chain-server is running fine, here's my logs after running docker logs chain-server:

/usr/local/lib/python3.10/dist-packages/pymilvus/client/__init__.py:6: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
  from pkg_resources import DistributionNotFound, get_distribution
INFO:     Started server process [1]
INFO:     Waiting for application startup.
INFO:faiss.loader:Loading faiss with AVX2 support.
INFO:faiss.loader:Successfully loaded faiss with AVX2 support.
INFO:RAG.src.chain_server.utils:Using nvidia-ai-endpoints as model engine and nvidia/nv-embedqa-e5-v5 and model for embeddings
INFO:RAG.src.chain_server.utils:Using embedding model nvidia/nv-embedqa-e5-v5 hosted at api catalog
INFO:RAG.src.chain_server.utils:Using milvus collection: multimodal_rag
INFO:RAG.src.chain_server.utils:Using milvus collection: multimodal_rag
INFO:RAG.src.chain_server.utils:Vector store created and saved.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8081 (Press CTRL+C to quit)

And here's my logs for rag-playground after running docker logs rag-playground:

[1] Jul 08 03:41:34 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/gradio-messaging/en "HTTP/1.1 200 OK"
[1] Jul 08 03:41:36 [   INFO] - frontend - Loading application configuration.
[1] Jul 08 03:41:36 [   INFO] - frontend - Configuration: 
model-name: meta/llama3-8b-instruct
server-port: '8081'
server-url: http://chain-server

[1] Jul 08 03:41:36 [   INFO] - frontend - Instantiating the API Server.
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://checkip.amazonaws.com/ "HTTP/1.1 200 "
[1] Jul 08 03:41:36 [  ERROR] - frontend.chat_client - Failed to connect /documents endpoint of chain-server. Error details: HTTPConnectionPool(host='chain-server', port=8081): Max retries exceeded with url: /documents (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7b3156737ca0>: Failed to establish a new connection: [Errno 111] Connection refused')).
[1] Jul 08 03:41:36 [WARNING] - matplotlib - mkdir -p failed for path /.config/matplotlib: [Errno 13] Permission denied: '/.config'
[1] Jul 08 03:41:36 [WARNING] - matplotlib - Matplotlib created a temporary cache directory at /tmp/matplotlib-me1b8lqh because there was an issue with the default path (/.config/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
/usr/local/lib/python3.10/dist-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.43.0, however version 4.44.1 is available, please upgrade. 
--------
  warnings.warn(
[1] Jul 08 03:41:36 [   INFO] - httpx - HTTP Request: GET https://api.gradio.app/pkg-version "HTTP/1.1 200 OK"
/usr/local/lib/python3.10/dist-packages/gradio/analytics.py:106: UserWarning: IMPORTANT: You are using gradio version 4.43.0, however version 4.44.1 is available, please upgrade. 
--------
  warnings.warn(
[1] Jul 08 03:41:36 [   INFO] - matplotlib.font_manager - generated new fontManager
[1] Jul 08 03:41:37 [   INFO] - frontend - Starting the API Server.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Started server process [1]
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Waiting for application startup.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Application startup complete.
[1] Jul 08 03:41:37 [   INFO] - uvicorn.error - Uvicorn running on http://0.0.0.0:8090 (Press CTRL+C to quit)
[1] Jul 08 03:42:05 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /kb HTTP/1.1" 200
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/css/7636246223312442.css HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /_next/static/chunks/webpack-5146130448d8adf7.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/chunks/main-92011a1a7f336a6f.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /_next/static/WuNGAl0x4o1D5HqLxhHMt/_buildManifest.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /_next/static/chunks/pages/kb-cf0d102293dc0a74.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:2741 - "GET /_next/static/chunks/pages/_app-f55c3b932a623280.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:7582 - "GET /_next/static/chunks/78-a36dca5d49fafb86.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:21050 - "GET /_next/static/chunks/framework-7a7e500878b44665.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:33675 - "GET /_next/static/WuNGAl0x4o1D5HqLxhHMt/_ssgManifest.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /_next/static/chunks/pages/converse-61880f01babd873a.js HTTP/1.1" 304
[1] Jul 08 03:42:06 [   INFO] - uvicorn.access - 132.237.184.249:32867 - "GET /content/kb/?__theme=light HTTP/1.1" 500
[1] Jul 08 03:42:06 [  ERROR] - uvicorn.error - Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 460, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 187, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 165, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/gradio/route_utils.py", line 760, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 715, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 735, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 288, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 76, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 62, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 51, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 73, in app
    response = await f(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 301, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 214, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 39, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 967, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 427, in main
    gradio_api_info = api_info(False)
  File "/usr/local/lib/python3.10/dist-packages/gradio/routes.py", line 456, in api_info
    app.api_info = app.get_blocks().get_api_info()
  File "/usr/local/lib/python3.10/dist-packages/gradio/blocks.py", line 2782, in get_api_info
    python_type = client_utils.json_schema_to_python_type(info)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 893, in json_schema_to_python_type
    type_ = _json_schema_to_python_type(schema, schema.get("$defs"))
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 938, in _json_schema_to_python_type
    elements = _json_schema_to_python_type(items, defs)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 908, in _json_schema_to_python_type
    return _json_schema_to_python_type(defs[schema["$ref"].split("/")[-1]], defs)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 947, in _json_schema_to_python_type
    des = [
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 948, in <listcomp>
    f"{n}: {_json_schema_to_python_type(v, defs)}{get_desc(v)}"
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 955, in _json_schema_to_python_type
    f"str, {_json_schema_to_python_type(schema['additionalProperties'], defs)}"
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 901, in _json_schema_to_python_type
    type_ = get_type(schema)
  File "/usr/local/lib/python3.10/dist-packages/gradio_client/utils.py", line 863, in get_type
    if "const" in schema:
TypeError: argument of type 'bool' is not iterable
[1] Jul 08 03:42:07 [   INFO] - uvicorn.access - 132.237.184.249:59656 - "GET /favicon.ico HTTP/1.1" 304

Any fixes to this?

[mrsaso]

@yxchia98 What architecture are you running on? x86 or aarch64/Arm?

I had run this on an aarch64 system, and unfortunately the rag-playground container only supports x86. I made modifications to remove rag-playground deployment and simply used the FastAPI to interface with the RAG. Not sure if may be the issue you are having.

[tom5610]

hey @mrsaso & @yxchia98, I encountered exact the same error and it was due to some conflicts between pydantic and gradio. I resolved the issue with pinning pydantic v2.10.6 in requirements.txt under "rag-playground' module. (refer github issue in gradio - gradio-app/gradio#10662)

...
pydantic==2.10.6
...

In addition, I couldn't invoke 'meta/llama3-70b-instruct' and encountered an error -

... Function id 'a88f115a-4a47-4381-ad62-ca25dc33dc1b' version 'null': Specified function in account ...

To resolve the issue with using different LLM (e.g. deepseek-r1). @mrsaso, any hints on the error? Thanks.