I recently needed to solve a problem for a client that sounded simple: whenever an order is created, prepared, or completed; send a notification about the order.
A simple problem with hidden complexity.
Of course, things are never as simple as they seem. In this case one of the challenges we faced was that the data for the order needed to be retrieved from multiple events and services.
Before we could send the order notification, we would need to listen for and correlate order creation and product updates messages. On top of this we needed the solution to scale to handle thousands of concurrent orders quickly, while being cost-effective. Lastly, it needed to be reliable so notifications would never be missed.
Figure 1: Gathering domain events and send out a notification
While the specific scenario might look different, we find that this pattern of gathering data from across numerous events and service is common for our clients. Though simple solutions are often adequate at first, high throughput and out-of-order messages can quickly bring a lot of complexity to these scenarios.
Fortunately on this project, we found that by using Azure Functions with Durable Entities we could easily build a scalable and reliable solution without a lot of additional overhead.
First Step: Azure Functions
All the data we needed to be able to send order notifications was available to us via Topics on the Azure Service Bus (ASB) and through internal services.
Since ASB events would be triggering the notification system, the obvious solution was to look at using Azure Functions with ASB triggers. Azure Functions on a consumption plan use a scale controller to monitor the number of messages and scale out, or in, appropriately. This approach would allow us to meet the scalability and cost requirements for us, but still left us with questions on how to address the compilation of event data and a need to ensure the reliable sending of messages.
Introducing Durable Entities
Over the last few months our team had been experimenting with Azure’s Durable Functions extension. One of the concepts recently introduced for this extension was Durable Entities as a way to explicitly manage state within a Durable Function instead of implicitly managing state through the function’s control flow. This approach had promise for our solution as it would allow us to compile the data from multiple events into a single entity that could be referenced easily from Azure Functions.
Durable Entities build on the state management of Durable Functions by providing users with an entity object that represents the function’s state. Each entity has a unique key and its own state which cannot be modified outside of the entity itself. The only way to modify an entity is to send it a message, and the messages are managed so that they will never be processed concurrently on the same entity. As a result, this model provides a great way to manage concurrent or event driven scenarios without adding complex locking mechanisms.
Additionally, since Durable Entities build on Durable Functions, they inherit the reliability of the Event Sourcing pattern Durable Functions are built upon. When entity events occur, they are appended to the history of the orchestration instance stored in an Azure Storage table and the work to process the event is queued. If any event processors fail to complete, they will be picked up when the function is restarted, and the state of the entity built up back from previously completed events.
Our Solution
Using Durable Entities turned out to be a great solution to our problem. Our implementation followed these basic steps:
- Create a Durable Entity function to store event data
- Create Azure Service Bus message listeners to update the Durable Entity
- Load up the entity state and send a notification with the retrieved data
- Creating a Durable Entity Function
Durable Entities functions are easiest to use with an entity class that defines the state and any actions that may act on it. To represent an order, we created a class like the following that stores the order data.
The special Run function is responsible for accepting signals from other Azure Functions and passing them along to the appropriate method on the entity class. The string name of the method is used as the signal name, but you can also use interfaces for your entity classes that enables you to ensure the name is correct.
Note that the Run function must be static and cannot reference dependencies you have injected via the constructor. In practice this is not typically an issue, since the only thing it needs to do is pass along the context to the correct method on the entity.
2. Azure Service Bus Message Listeners
Once we had the durable entity class and function prepared, we were ready to send it some signals! Remember that the only way to interact with a Durable Entity is via messages, so we didn’t need to instantiate this class ourselves. Instead we needed to send it a message informing it to update its state.
In our solution, we want to update this entity when we receive messages about the Order from the Azure Service Bus. We spun up several Functions with Service Bus triggers to do this.
We used the ServiceBusTrigger attribute in conjunction with the DurableClient attribute to easily update our Durable Entity from inside the function.
Now, whenever we received an update about our Order, we were able to simply send a message to our Durable Entity and tell it to update its state.
- Send Notifications
When we received an event that requires us to send a notification,
we simply loaded up the entity state and sent out the notification.
To call this we added a “Completed” event to our Azure Service Bus function, but the same could be done for any event that triggers a notification.
Problem solved!
Durable Entities for the Win!
Durable Entities provided a quick way to maintain state across events without the added complexity of manually managing storage.
In addition, they provided us with the confidence we needed in the scalability and reliability of our solution. Best yet, they enabled us to deliver this solution completely serverless, with all the cost benefits of the Azure Function consumption plan.
When dealing with event driven architectures, even simple problems can pose difficult challenges when needing to scale quickly and reliably. Often solving these challenges can require building a fair amount of infrastructure code to ensure messages are stored correctly and outgoing messages are sent reliably. By using Azure Functions with Durable Entities, much of this can be simplified, making event driven architectures that much more enjoyable to work with.
If Durable Entities sound interesting, you should check out this post by the original creator of Durable Functions, Chris Gillum.
Submit a Comment