[Bug] Agent Infinite Loop on Simple Conversation Tasks

[Clearner1]

🐛 Bug Report: Agent Infinite Loop on Simple Conversation Tasks

Description

When sending a simple conversation message (e.g., "Hello"), the Agent falls into an infinite loop, repeatedly processing the same task until the token limit is reached.

Steps to Reproduce

1. Start the Bytebot system (docker-compose -f docker/docker-compose.proxy.yml up -d)
2. In the UI, send a simple message: "Hello"
3. Observe the Agent logs

Expected Behavior

• The Agent should reply "Hello" and complete the task
• The task status should transition from RUNNING to COMPLETED
• No infinite loop should occur

Actual Behavior

• The Agent enters an infinite loop, repeatedly processing the same task
• Message count increases from 1 to 18+
• Token usage keeps growing: 2319 → 6013 → ...
• Task status remains RUNNING and never completes

Error Logs

2025-09-16 17:27:40 [AgentProcessor] Processing iteration for task ID: ec4e4c92-0660-49db-9cdb-0a7ea8b79592
2025-09-16 17:27:40 [AgentProcessor] Sending 1 messages to LLM for processing
2025-09-16 17:27:41 [AgentProcessor] Received 1 content blocks from LLM
2025-09-16 17:27:41 [AgentProcessor] Token usage: 2319/128000 (2%)

2025-09-16 17:27:41 [AgentProcessor] Processing iteration for task ID: ec4e4c92-0660-49db-9cdb-0a7ea8b79592
2025-09-16 17:27:41 [AgentProcessor] Sending 2 messages to LLM for processing
2025-09-16 17:27:42 [AgentProcessor] Received 1 content blocks from LLM
2025-09-16 17:27:42 [AgentProcessor] Token usage: 2334/128000 (2%)

[loop continues...]

2025-09-16 17:31:02 [AgentProcessor] Sending 18 messages to LLM for processing
2025-09-16 17:31:02 [AgentProcessor] Token usage: 6013/128000 (5%)

Environment

• OS: Windows 11 + WSL2 Ubuntu

• Docker: Docker Compose

• Model: Local vLLM (Qwen2.5-VL)

• Task Type: IMMEDIATE

• Task Control: ASSISTANT

• Model Config:

  model: "openai//home/user/00model"
  api_base: "http://host.docker.internal:8000/v1"

Root Cause Analysis

Source code analysis shows this is a design flaw:

In packages/bytebot-agent/src/agent/agent.processor.ts lines 368–387:

// The task only completes when Agent explicitly calls the set_task_status tool
if (setTaskStatusToolUseBlock) {
  switch (setTaskStatusToolUseBlock.input.status) {
    case 'completed':
      await this.tasksService.update(taskId, {
        status: TaskStatus.COMPLETED,
        completedAt: new Date(),
      });
  }
}

// But a new iteration is always scheduled!
if (this.isProcessing) {
  setImmediate(() => this.runIteration(taskId));
}

Problem: For simple conversation tasks, the Agent never calls set_task_status, so tasks never complete.

Suggested Fixes

Option 1: Smart Task Completion Detection

// Add completion detection in runIteration
const shouldContinue = await this.shouldContinueProcessing(taskId, responseBlocks);

if (this.isProcessing && shouldContinue) {
  setImmediate(() => this.runIteration(taskId));
} else {
  // Auto-complete simple conversation tasks
  await this.tasksService.update(taskId, {
    status: TaskStatus.COMPLETED,
    completedAt: new Date(),
  });
}

Option 2: Different Logic by Task Type

Use separate completion logic for conversation vs. action tasks.

Option 3: Safety Net

Add a maximum iteration limit to prevent infinite loops:

private readonly MAX_ITERATIONS = 50;
private iterationCounts = new Map<string, number>();

Impact

• Severity: High
• Effect: Resource waste, system unusable, token exhaustion
• User Experience: Cannot use basic conversation functionality

Reproducibility

100% – happens every time a simple conversation message is sent

[Clearner1]

WeChat_20250916174702.mp4