Can't properly produce structured response with array

[acgonzales]

Hello,

Just checking to see if others have experience or fix this issue.

I need my agent to respond with an structured output likeso:

// ParentData.php

use NeuronAI\StructuredOutput\Validation\Rules\ArrayOf;

final class ParentData
{
    /**
     * @var ItemData[]
     */
    #[ArrayOf(ItemData::class)]
    public array $items;
}

// ItemData.php

use NeuronAI\StructuredOutput\SchemaProperty;

final class ItemData
{
    #[SchemaProperty(description: 'The description of the item.', required: true)]
    public string $description;

    #[SchemaProperty(description: 'The priority of the item.', required: true)]
    public int $priority;
}

I'm met with this error:

NeuronAI\Exceptions\AgentException

The LLM wasn't able to generate a structured response for the class App\Data\ParentData.

Already tried variations like using FQCNs or using array<ItemData>.

Tried stronger models like Gemini 2.5 Pro and Claude 4 Sonnet but still throws the same exception.

Don't get me wrong, but I did not experience a problem like this with Prism using the same structure and even weaker models like Gemini 2.0 Flash.

Am I doing something wrong?

[ilvalerione]

Hi @acgonzales two things can help:

1. What version of Neuron are you using? If you can try with the latest, because fixes are released almost every day.

2. If you can connect Inspector to your application we can see the chain of errors to better understand what's going on behind the scenes, and eventually identify the bug.

[ilvalerione]

Ok, I just wanted to be sure, because I tried a generation with your classes and FQCN, and it worked. This is the right syntax anyway.

So the problem could be elsewhere. In the latest version 1.14.29 the underlying error of failed structured output generation will be propagated in the exception. If you can udate the composer dependencies and retry the generation you should see a more clear indication of the problem. With Inspector we can also see the schema and model response details. Let me know eventually.

[acgonzales]

Updated the package to 1.14.29 and tried to generate. Got a more clearer exception.

Impossible to generate a structured response for the class App\Neuron\Agents\Outputs\ParentData:
- items must be an array of App\Neuron\Agents\Outputs\ItemData

As for Inspector, I created a free account and added the ingestion key in .env but with 5-10 generation tries. No events are logged in my dashboard. Do I need to do anything else?

-- EDIT --

I was initially testing the generation via an Laravel artisan command, no Inspector events. But when I tried to execute it on an HTTP controller, the error and transactions are now created. And this is what it gives me with regards to the structured output error:

Client error: `POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent` resulted in a `400 Bad Request` response: 
{ "error": { "code": 400, "message": "Invalid JSON payload received. Unknown name \"$ref\" at 'generation_config.response_schema.properties[0].value.items': Cannot find field.\nInvalid JSON payload received. Unknown name \"definitions\" at 'generation_config.response_schema': Cannot find field.", "status": "INVALID_ARGUMENT", "details": [ { " @type": "type.googleapis.com/google.rpc.BadRequest", "fieldViolations": [ { "field": "generation_config.response_schema.properties[0].value.items", "description": "Invalid JSON payload received. Unknown name \"$ref\" at 'generation_config.response_schema.properties[0].value.items': Cannot find field." }, { "field": "generation_config.response_schema", "description": "Invalid JSON payload received. Unknown name \"definitions\" at 'generation_config.response_schema': Cannot find field." } ] } ] } }

Maybe the package is sending an invalid request format to the Gemini API?

Anyways, tried it again with Claude 3.5 and seems like its working now.

[ilvalerione]

ok, it needs further investigation for Gemini. Thank you for the report.

[ilvalerione]

@acgonzales The latest 1.15.0 also support nested output structures for Gemini. Thank you for reporting it here.

[acgonzales]

Awesome work, thanks @ilvalerione!

[zliebersbach]

I was getting the same problem, I needed to use a fully-qualified namespace in my docblock for it to work!

Before:

    /**
     * @var DocumentSplitChunk[]
     */

After:

    /**
     * @var \App\Splitters\DocumentSplitChunk[]
     */

[oliverbj]

@ilvalerione I still encounter this problem. I'm running version 2.2 and using gpt-5 with structured output support through Azure OpenAI.

Error:

"insights must be an array of App\Services\AI\Schemas\UserSalesAgent\Insight"

I have defined a schema:

class Schema
{
    #[SchemaProperty(description: 'Sales staff code', required: true)]
    #[NotBlank]
    public string $staff_code;


    #[SchemaProperty(description: 'Array of generated sales insights for the sales staff member', required: true)]
    /**
     * @var \App\Services\AI\Schemas\UserSalesAgent\Insight[]
     */
    #[ArrayOf(Insight::class)]
    public array $insights;
}

And the Insights:

class Insight
{
    #[SchemaProperty(description: 'The category of the insight (e.g. declining_client, upsell_opportunity, quote_followup)', required: true)]
    public string $type;

    #[SchemaProperty(description: 'Description of the insight. Format the description to be clear and concise. Give it some prose in a proffesional tone. Avoid abbrevations.', required: true)]
    public string $summary;

    #[SchemaProperty(description: 'Clear recommended next step for the staff member. Format the action to be clear and concise. Give it some prose in a proffesional tone. Avoid abbrevations.', required: true)]
    public string $action;

    #[SchemaProperty(description: 'Optional contextual references for drill-down in the widget (client code, quote number, shipment number)', required: false)]
    /**
     * @var  \App\Services\AI\Schemas\UserSalesAgent\ContextReference[]
     */
    #[ArrayOf(ContextReference::class)]
    public array $context_references;
}

My agent is using the schema:

protected function provider(): AIProviderInterface
{
    return new AzureOpenAI(
        key: config('ai.azure_openai.key'),
        endpoint: config('ai.azure_openai.endpoint'),
        model: config('ai.azure_openai.model'),
        version: config('ai.azure_openai.version')
    );
}

protected function getOutputClass(): string
{
    return Schema::class;
}

Looking at this through Inspector, everything seems to be outputted correctly by the model:

validate schema returns the right format (each sub item within insights is also structured correctly and the JSON is able to be parsed:

{
  "staff_code":  "PP0",
  "insights": 
  [
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...}, // 4 items
    {...} // 4 items
  ]
}

[oliverbj]

Weirdly enough, it works if I remove the nested ArrayOf I had going on inside the Insight class:

Commenting this out, and it generates without an issue.

  /**
   * @var  \App\Services\AI\Schemas\UserSalesAgent\ContextReference[]
   */
  /*
  #[SchemaProperty(description: 'Optional contextual references for drill-down in the widget (client code, quote number, shipment number)', required: false)]
  #[ArrayOf(ContextReference::class)]
  public array $context_references;*/

Can't we provide nested ArrayOf schemas?

[ilvalerione]

Hi @oliverbj, It's supposed to work. You've found a bug I believe. I'll go deeper into it to test this scenario. I'll update here for some news.

[oliverbj]

Appreciate it! It's a great package overall, really enjoy working with it.

[oliverbj]

Perhaps a follow up to this one as well:

I try to use the #[Count(exact: 5)] validator, but that fails:

#[SchemaProperty(description: 'Array of generated sales insights', required: true)]
/**
 * @var \App\Services\AI\Schemas\UserSalesAgent\Insight[]
 */
#[ArrayOf(Insight::class)]
#[Count(min: 5, max: 8)]
public array $insights;

My Agent generated a bit more items:

{
"insights": 
  [
  {...},
  {...},
  {...},
  {...},
  {...},
  {...},
  {...},
  {...}
  ]
}

Violations:

[
"insights must be exactly 5 items long"
]

I'm wondering if this is just a limitation of the LLM (GPT-5 in this case) and it's ability to follow the rules set, or if it's linked to the usage of Count on an ArrayOf property.

[ilvalerione]

@oliverbj take a look at this test case I created: https://github.com/inspector-apm/neuron-ai/blob/2.x/tests/ValidationTest.php#L159

It should represent your use case and it seems to work.

The problem could be in the ContextReference class, or ultimately in the response from the LLM.

[ilvalerione]

You mentioned "exact" rule, but in your example you use #[Count(min: 5, max: 8)].