Azure Service Bus emulator incompatibility

[eduardobr]

I have tried to connect to Azure Service Bus emulator from Service Bus Explorer, using a connection string like:
Endpoint=sb://127.0.0.1;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true
Regular access from host machine worked, but Service Bus Explorer would fail to connect and present the artifacts from the emulator instance.
Running SB Explorer from source code, I could see that the error occurs when calling NamespaceManager.GetQueuesAsync (among other admin commands)
I believe the emulator doesn't have support for these commands, and it's not clear if it's to be supported in the near future.

An idea to make it work could be to check for the existing flag UseDevelopmentEmulator=true in the connection string (we might also have a checkbox for this in UI), and if true, do not make API calls like GetQueuesAsync, but instead allow us to specify the config.json path for the emulator and populate the tree of artifacts from there - see https://learn.microsoft.com/en-us/azure/service-bus-messaging/test-locally-with-service-bus-emulator?tabs=docker-linux-container for details.

In theory this should be enough to make it work, but all administrative commands should be disabled from UI when connected to an emulator instance.

[ErikMogensen]

Good idea. The problem is as always that there has to be someone to implement it.

Since the emulator only supports AMQP, you have to add ;TransportType=Amqp to the connection string.

[SeanFeldman]

The emulator is explicitly designed and provided for unit testing scenarios. Which, exercises code to work with messages and doesn't provide administrative functions that are leveraged by this tool as means of exploring namespace entities and messages. Why to force a square peg into a round hole? What would be the purpose of using SBE with a unit testing emulator?

[supereddie]

Personally I mainly use SBE to 'browse' queues / subscriptions to look at messages. For example if my application is not receiving a message, I use SBE to check if the message is actually in the queue / subscription.

[HashemSrojy]

Personally I mainly use SBE to 'browse' queues / subscriptions to look at messages. For example if my application is not receiving a message, I use SBE to check if the message is actually in the queue / subscription.

what do mean by SBE ?

[SeanFeldman]

Exactly. SBE (Service Bus Explorer) is not a unit testing tool. It's a management tool intended for browsing entities and messages.
The emulator is solely for testing.

This emulator is intended to facilitate local development experience for Service Bus, allowing developers to develop and test their code against Azure Service Bus, in isolation away from cloud interference.

If there are volunteers to implement integration with the ASB emulator, that's a different story.

[ErikMogensen]

What would be the purpose of using SBE with a unit testing emulator?

I see two reasons for that. The first one is when developing and troubleshooting unit tests, to make you see what's actually happening. The other reason is to add more unit tests to SBE since a lot of more scenarios are now possible to test. Preferably the build pipeline should install and configure the emulator then.

But it all goes back to someone implementing this.

[SeanFeldman]

The other reason is to add more unit tests to SBE since a lot of more scenarios are now possible to test.

This one made me smile. Quantity, not quality. Testing is a PITA with SBE and has nothing to do with the emulator but because the SBE code base is not testable at its current state.

Anyhow, just because something can be done, doesn't mean it should be done. The emulator has just gone out. Give it some time to settle, gain most of the features, and then re-evaluate what that connectivity would look like. Unless you've got time on your hands you're willing to donate. Then it's a different topic.

[eduardobr]

What would be the purpose of using SBE with a unit testing emulator?

1. Based on the official documentation I don't see unit testing as the only author purpose for the emulator.
   https://learn.microsoft.com/en-us/azure/service-bus-messaging/overview-emulator
   

2. On a personal level, I would be highly leveraging from both a simple, optimized inner development loop and cost efficiency.
   Today, in order to achieve the bare minimum of creating a topic in service bus, you need a Service Bus namespace on the Standard tier (even Basic tier doesn't support topics). Yes, it's a shameful move from Microsoft to make developers go through this for 15 years before releasing an emulator for local development.

3. Now about SBE, it would be the tool to complement this local development experience.

I am thankful that this community project exists and for all your efforts.
Should be needless to say, but this is not a request neither a complaint. It's an idea to start a discussion.
I see myself really using this, and would totally make the contribution when/if I had the spare time 🙂

[SeanFeldman]

What the MSFT marketing team is putting out there is one thing. What the reality looks like is a different thing. You can beat me up with the quote from their documentation on how it's for development and testing, but this emulator is not ready for real development. No support for management operations - use an external static config JSON file. There is no support for SQL filters, and it can't even be worked around. There is no support for entity statistics - we need to augment that to be able to see something meaningful.

So yeah, I stand behind my words. At this time, this is a testing emulator different from the emulator everyone expected.

Now, regarding the topic of SBE, should SBE support a half-baked emulator? My answer is 'not yet'. But I don't get to decide because I'm not the project's owner, and it's ultimately up to the individuals to determine what they want to do with their time.

Should be needless to say, but this is not a request neither a complaint. It's an idea to start a discussion.

And that's what we do. But a discussion should have some objectives. I don't see a clear one on this thread. It's mostly "Hey, we could really use this functionality," ignoring the maturity state of the emulator. The SBE codebase has its skeletons in the closet. We shouldn't be adding piling up more.

I see myself really using this, and would totally make the contribution when/if I had the spare time 🙂

I greatly appreciate every single contribution, including ideas. But ideas need to be based on reality. The emulator is a bleeding-edge ATM. Let it mature, stabilize, and gain features that would not require unnecessary throwaway investments into SBE. Maybe it will be almost effortless to get them to work together.

I'll also mention this: If people care to use the emulator as a proper development emulator, they should raise and upvote issues for Microsoft to get the emulator on par with the real service. Such as SQL fileters, Admin functionality, etc. Work smart, not hard.

[supereddie]

What the MSFT marketing team is putting out there is one thing. What the reality looks like is a different thing. You can beat me up with the quote from their documentation on how it's for development and testing, but this emulator is not ready for real development. No support for management operations - use an external static config JSON file. There is no support for SQL filters, and it can't even be worked around. There is no support for entity statistics - we need to augment that to be able to see something meaningful.

So yeah, I stand behind my words. At this time, this is a testing emulator different from the emulator everyone expected.

These issues are not something that actually bothers me for local development. Entity Statistics are not relevant while developing, and you shouldn't be using SQL Filters if you have higher message volumes since they are slow. No management operations are annoying but I can understand since the emulator doesn't persist its state between restarts. And because in my company we manage the Service Bus infrastructure entirely via Terraform there is no need to alter things on-the-fly, so none of our applications actually have manage permissions anyway.

I'll also mention this: If people care to use the emulator as a proper development emulator, they should raise and upvote issues for Microsoft to get the emulator on par with the real service. Such as SQL fileters, Admin functionality, etc. Work smart, not hard.

The current emulator is 'good enough' for developing applications that only send and receive messages - which I believe are most applications And having a way to send / browse / explore the messages in the queues and subscriptions in a visual way is very handy during development.

[SeanFeldman]

you shouldn't be using SQL Filters if you have higher message volumes since they are slow

If you could cover every scenario with correlation filters only, I'd agree. But that's not the case. Point in case, all NServiceBus users require SQL filters. And on the topic of slow/fast, at what point SQL filter is slower than correlation filter? Careful, because I could say it's 'good enough' for most applications.

What I see off about this issue is the focus on the wrong thing. Rather than giving the emulator a little time to implement what it will need to implement anyway, people are rushing to do it themselves. And what it will directly result in is additional unnecessary complexity and maintenance burden. Which is already a significant pain point of this project. Not to forget the use of the old ASB client, single platform, AAD/EntraID support, manual verifications, etc.

The focus should be on the emulator fixing the lack of management APIs. If that's done, every tool, including SBE, will work with minimal changes required. Plain and simple. Until then, if you feel that you want to invest your time into augmenting emulator 's missing functionality that will be coming done or later, you or anyone else is welcome to do so.

The good news is that this approach is not unique to this SBE issue. The Programmers’ Credo is far more common than it should be.

[searus]

You could just use RabbitMQ locally in Docker. It has it's own management UI in the image rabbitmq:3-management. Then you just need to abstract the functionality you need and use RabbitMQ locally and service bus in Azure. Best of both worlds. Works for our team and doesn't require much additional code (depending on what you're doing) - we just use it for pub/sub

[madsvcarlsen]

You could just use RabbitMQ locally in Docker. It has it's own management UI in the image rabbitmq:3-management. Then you just need to abstract the functionality you need and use RabbitMQ locally and service bus in Azure. Best of both worlds. Works for our team and doesn't require much additional code (depending on what you're doing) - we just use it for pub/sub

That is a limited use-case, since rabbitmq does not offer the full scope for many service bus applications. E.g. Azure Function Triggers.

[umair-me]

A new version has been released few weeks ago which now has new features:

This release introduces new features, bug fixes for Service Bus emulator. The details are as follows:

SQL Actions and Filters are now compatible with Service bus emulator.
Provides Health check API in Service Bus emulator. It can be accessed at http://localhost:5300/health
Provides user configurable health check interval for SQL.
Added support for updating LockDuration to a minimum of 5 seconds.
Other bug fixes.

https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-emulator-whats-new#latest-version-112