emqx-web-hook is blocking message being received by subscribers

[daadu]

Setup

version: 4.1.0
environment: k8s (installed with Helm)
mqtt: v3.1

Problem

The web-hook call made by emqx seems like a blocking call for subscribers from receiving "that" message. Until the web hook is processed the messages are not received by subscriber. This causes an issue particularly when the webserver receiving a webhook is also (temporarily) subscribing to a particular topic (in my case to perform request-response pattern messages when a certain message is received via webhook).

Possible Solution

1. trigger webhook after messages are routed to all subscribers.
2. fire-and-forget (async) webhook should be trigger (I posted regarding this in Confusion regarding emqx-rule-engine #196)

Remarks

If 4.2.0 is not happening soon then solution suggested in point 1 should be quickly implemented as a work around. Also will this issue also be raised when using emx-rule-engine instead of emqx-web-hook?

[hmonadjem]

Any updates on this?

We are running 4.2.6 and having still this problem

[KerryJava]

@hmonadjem can you use message queue to process webhook message , so that it won't block the whole process ?

[daadu]

@hmonadjem can you use message queue to process webhook message , so that it won't block the whole process ?

Already tried it, used "celery" for it, it causes hick-ups when doing request-response kind of a thing on the HTTP server.

[HJianBo]

Hi, guys, This has really been a problem with the web-hook plugin.

We will put this on the agenda in anticipation of a good solution. Thank you very much for your suggestions and concerns!

[HJianBo]

Have you guys tested the approximate throughput and TPS that can be supported by the current version?

[hmonadjem]

Thank you! The Throughput was never a problem for us (we have about 20 msg/s) but it is a problem when the target http server is booting up or shutting down or is unreachable. That basically blocks every message for 5s (timeout) and after that if the response is 500 the mqtt message is never delivered. Which is very bad for us.

[adrian-parisi]

Is there any progress on this? I'm running on the same issue.

[onyet]

I hope this gets resolved, because it's really bad, where message_publish and message_delivered don't work as they should.

[QBH-insist]

Hello, Is there any progress on this?