Asynchronous Structured

[oliverbj]

Hi.

Does Neuron AI have an option to call the structured endpoint asynchronous? Currently, I can only see that option available for the chat endpoint.

[ilvalerione]

What's your real case? Do you need to parallelize tasks like https://docs.neuron-ai.dev/getting-started/asynchronous-processing#basic-async-implementation?

[oliverbj]

Yes, that was the idea to parallelize structured output tasks :)

[ilvalerione]

It's quite hard with the current structure. I put this idea in the backlog, but frankly I don't know if it will be addressed. The other option is someone else looking into it and create a PR.