Ollama integration with Open WebUI not working

[mikanygTC]

Describe the bug

I am unable to get the CommunityToolkit.Aspire.Hosting.Ollama (v9.7.0) working with Aspire 9.4.

When I follow this guide https://learn.microsoft.com/en-us/dotnet/aspire/community-toolkit/ollama?tabs=dotnet-cli%2Cdocker#open-webui-support to add Ollama with Open WebUI to the Aspire Starter App in Visual Studio (v17.14.11) and run the app, then after all containers have started and downloaded the model, then I can open Open Web UI, but any prompts towards Ollama fails with the error: "Unexpected token 'd', "data: {"id"... is not valid JSON".

When I look at the browser console I see many entries with failing web socket conncetions.

Regression

No response

Steps to reproduce

1. Create a new Aspire Starter App in Visual Studio (v17.14.11)
2. Update all nuget packages versions to latest (Aspire 9.4)
3. Update apphost project SDK version to aspire 9.4.0
4. Add code snippet to app host
My AppHost looks like this. 


var builder = DistributedApplication.CreateBuilder(args);

var cache = builder.AddRedis("cache");

var ollama = builder.AddOllama("ollama")
                    .WithLifetime(ContainerLifetime.Persistent)
                    .WithDataVolume()
                    .WithGPUSupport(OllamaGpuVendor.Nvidia);

ollama.WithOpenWebUI(container =>
{
    container.WithLifetime(ContainerLifetime.Persistent)
        .WithDataVolume();
});

var llamaModel = ollama.AddModel("tinyllama");

var apiService = builder.AddProject<Projects.AspireApp1_ApiService>("apiservice")
    .WithReference(llamaModel)
    .WithHttpHealthCheck("/health");

builder.AddProject<Projects.AspireApp1_Web>("webfrontend")
    .WithExternalHttpEndpoints()
    .WithHttpHealthCheck("/health")
    .WithReference(cache)
    .WaitFor(cache)
    .WithReference(apiService)
    .WaitFor(apiService);

builder.Build().Run();

5. Run the app, wait for open web ui to be ready
6. open Web UI and write a chat prompt

Expected behavior

That the code setup works out of the box as described in the documentation code samples.

Screenshots

IDE and version

VS 2022

IDE version

17.14.11

Nuget packages

CommunityToolkit.Aspire.Hosting.Ollama" Version="9.7.0"

Additional context

Help us help you

No, just wanted to report this

[aaronpowell]

I'm unable to replicate this myself - would it be possible to capture the HAR from the chrome devtools network tab for inspection?

[github-actions]

We have noticed this issue has not been updated within 21 days. If there is no action on this issue in the next 14 days, we will automatically close it. You can use /stale-extend to extend the window.