Agentic Workflow with Human In The Loop

[ilvalerione]

Hi everyone! After a lot of effort we have finally managed to ship the beta release of our implementation for Agentic Workflow.

What makes this implementation special is the native support for Human In The Loop pattern. This means your agentic system can literally pause mid-process, ask for human input, wait for feedback, and then continue exactly where it left off – even if that's hours or days later.

https://docs.neuron-ai.dev/workflow/getting-started

We will continuing to improve the documentation also based on your feedback.

I hope it could become the solution PHP deserve to implement enterprise-grade agentic software. Feel free to send us your feedback, share this repository with other PHP developers, create experiments, open issues, PRs! We can't be more happy to help you build great things!

[acgonzales]

I've been playing with it for a while and all I can say is its been working well so far. One thing though: it would be nice if we can add type-safety to the state or support custom data classes to use as the workflow's state..

[thomasdv]

I managed to get by using flags in the state to skip critical statements before the interrupt call, not clean but I'm satisfied with it.

My problem is that I need to interrupt from within a tool (Do you really want me to write this data to the database? Yes/No). The generated data is saved into state and interrupted. When the node is resumed, I can see my data in the state, but there is no proper way to get the user feedback to check if the anwer was Yes or No.

And if I let it execute, the agent and tool will just run again, repeat the whole process and generate new data before the interrupt call.

Maybe the problem is in my design, but I have not found a better way so far.

[ilvalerione]

Hi, I didn't followed the whole discussion since I'm out of work, but you should get the user feedback as the return value of the interrupt method: https://docs.neuron-ai.dev/workflow/human-in-the-loop#interruption

Isn't it your case?

[acgonzales]

We can get the feedback via the return value of interrupt. It is the data passed in the $workflow->resume() call. You can maybe share a snippet of your code and lets see where it went wrong.

[thomasdv]

Sure, I was refering to the following logic (as that was my first impression of how it should be used based on the docs). Simplified example:

class MyNode extends Node
{
    public function run(WorkflowState $state): WorkflowState {
    
      $response = MyAgent::make()->structured(
          new UserMessage("Set article ID 123 as active.")
      );
    
      if($response->needsConfirmation){
        $feedback = $this->interrupt($response->question);

        if($feedback == "Yes"){
          // save data from $response to database
        }
      }

      // ...
      
      return $state;
    }
}

But that is obviously flawed logic, because after resuming a new response will be created by the agent and new data will be saved to database.

Since I can only read $feedback value after that agent call there is no way to call the agent only if needed.

My solution was saving $response to $state, redirecting flow to dedicated InterruptionNode where the interrupt call is made. So when resumed, no data is changed and response can be handled. What I dislike about this solution is that now InterruptionNode is responsible for saving data into database which it should not be.

Is there a better way?

[ilvalerione]

You could save the confirmation feedback into the state so on the next node execution you can verify if the agent should run again or not.

Something like:

class MyNode extends Node
{
    public function run(WorkflowState $state): WorkflowState 
    {
        if (!$state->get('needsConfirmation')) {
            return $state;
        } 
        
        $response = MyAgent::make()->structured(
            new UserMessage("Set article ID 123 as active.")
        );
        
        $state->set('needsConfirmation', $response->needsConfirmation);

        if($response->needsConfirmation){
            $feedback = $this->interrupt($response->question);

            if($feedback == "Yes"){
                // save data from $response to database
            }
        }

        // ...

        return $state;
    }
}

Just to suggest the idea to use the state to remember what you need to do on the next execution.