FEATURE REQUEST: Enhance Classification Prompt for Stricter DevRel Query Handling

[smokeyScraper]

Is your feature request related to a problem?

• Yes, it is related to a problem

Describe the feature you'd like

🌟 Feature Description

This feature focuses on a critical enhancement of the initial Classification Prompt. The goal is to refine the prompt's logic to be much stricter and more precise, ensuring that the DevRel agent is only triggered for queries that are explicitly its responsibility. This involves improving the instructions the LLM uses to decide whether a message requires agent intervention.

Additionally, this task includes a secondary objective to review all other existing prompts (agent_prompt, summarization_prompt, etc.) for clarity, efficiency, and alignment with the latest system architecture.

🔍 Problem Statement

Currently, the classification prompt may be too broad, leading to the DevRel agent responding to general conversations or messages where it's not needed. This can create several issues:

1. Unnecessary Invocations: The agent wastes resources processing irrelevant messages.
2. User Annoyance: An AI assistant that injects itself into every conversation can be perceived as intrusive or unhelpful.
3. Lack of Precision: The agent may misinterpret context and provide answers when it should remain silent, degrading the quality of the user experience.

The core problem is a lack of strict activation boundaries for the agent.

🎯 Expected Outcome

1. Modified Classification Prompt: The Classification Prompt will be rewritten to include new, strict rules. The agent should only be activated if one of the following conditions is met:
   • The user explicitly tags or mentions the DevRel AI bot (e.g., @Devr.AI, can you help?).
   • The user's query is a direct question about the project, community, contribution process, or a technical issue clearly related to the repository (e.g., "How do I set up the development environment?", "I'm getting a build error.").
   • The prompt will explicitly instruct the model to ignore general chatter, greetings, and conversations between other users unless the bot is tagged.
2. Reduced False Positives: After implementation, there should be a measurable decrease in the number of times the agent responds to irrelevant conversations.
3. Improved Prompt Suite: A brief review of all other system prompts will be conducted, with minor enhancements made for clarity and conciseness.

Record

• I agree to follow this project's Code of Conduct
• I want to work on implementing this feature

[HarK-github]

@smokeyScraper would like to try this out. Can u assign this to me so that I can start working on it?

[smokeyScraper]

yep @HarK-github, you can give it a try.

[PrithvijitBose]

Hey @smokeyScraper 👋

I'd love to work on this issue. I’ve already contributed the thinking_node to improve query understanding, and I’d like to now focus on refining the classification logic for smarter agent activation.

🔧 My plan:

• Rework the classification_prompt.txt to include stricter conditions — such as only triggering on direct bot mentions or clearly scoped technical questions.
• Explicitly instruct the model to ignore greetings, general chatter, or unrelated discussions.
• Test with real examples to reduce false positives and ensure clean triggering.
• As a bonus, I’ll also review the other prompt files (e.g., agent_prompt, summarization_prompt) for clarity and alignment with the current flow.

Let me know if this direction works for you — and please assign me if it’s good to proceed! 🚀

[smokeyScraper]

Hey @PrithvijitBose, I'm preferring to assign one issue at a time. Please clarify which one you are willing to work on.

PS: @HarK-github are you working on this issue? I'm not very explicitly assigning issues prior a discussion. Please let me know.

[PrithvijitBose]

Hey @smokeyScraper if that's the case

Let me finish the FAQ handler one for now

I will take this one when i am finished with that
You can assign me and Rudra for that that issue

If it's ok

[DhruvK278]

Heyy @smokeyScraper I would like to work on this issue, also thinking to improve the logging as it is taking time to check agent status.

[aditi25srivastava]

@smokeyScraper Heyy!!!!! I would love to work on this issue. Could you please assign it to me ?