One of the solutions to guarantee data consistency across the microservices are distributed transactions. However many modern database technologies such as NoSQL databases and message brokers don’t support distributed transactions. Another downfall of the distributed transactions is the fact that they use synchronous communication. Therefore mechanisms that base on a principle of loose coupling and asynchronous communication should be used.
One of those is the Saga pattern. It works on a principle of applying the sequence of local transactions, where each one updates data within a single service based on ACID principle. A coordinator triggers the first step of the saga, meaning the first local transaction executes. After one local transaction is completed, it triggers the execution of the next local transaction. The benefit of asynchronous messaging is that it ensures that all the steps of saga are executed, even if one of the participants is temporarily unavailable.
A in ACID stands for the atomicy and this means that all the steps of a transaction are executed or the transaction is rolled back in case of failures. To ensure the atomicity of the transactions in Saga, we need methods to rollback the changes made by local transactions. Those methods are called compensating transactions. It is important for the compensating transactions to be idempotent and retryable. Idempotency means that each operation can be executed multiple times and the result will always remain the same. And the mechanism of automatically retrying the transactions in case of failures ensures that there is no need for any manual interventions.
The diagram above shows the overview of the Saga flow. As you can see, each transaction is triggered once the previous is finished. And in case of failures, the appropriate compensating transaction is executed. The compensating transaction has to call other compensating transactions in reverse order to reset changes to the initial state.
We need a mechanism that will take care of the coordination of the Saga flow. A Saga coordinator is responsible for this case. It contains logic that selects the first participants of the Saga and invokes the execution of its local transaction. Then it has to invoke the next participants until all the steps are completed. If any of the steps fails, a coordinator has to execute the compensating transactions in reverse order.
There are two ways of the implementation for the Saga coordinator logic:
- choreography – coordination logic and sequencing is distributed among the participants, meaning the participant invokes the next step after its local transaction is completed. This king of communication is primarily based on the events
- orchestration – coordination logic is centralized in a dedicated component, which sends commands to other participants telling them which operation to execute
Choreography
In this way of implementation there is no central component to tell Saga participants what to do. Therefore the participants must take care that the next steps in the Saga flow are executed. Usually the participants are subscribed to each other’s events and the published events are an indicator which tells other participants what to do. Each participant executes its local transaction and publishes an event. This event triggers the next participant, which repeats the same thing. The flow is considered completed when all the participants have successfully finished their local transactions. In case of failures, participants have to publish the appropriate events that trigger compensation transactions to revert their local changes.
We can demonstrate the choreography based saga flow on an e-commerce example. When a user creates an order, an event order_created is sent to the message broker. From an order channel, a payment service consumes this event and processes the payment, and at the same time, an inventory service reserves quantity for ordered items. When a payment is processed, a new event payment_processed is published to the payment channel of the message broker. Inventory service can update quantity based on this event, and the order service can update the status. When the inventory is updated, an inventory_updated event is published and the order service can finish the order when consuming this event. The diagram below demonstrates the choreography based saga flow.
This approach is simple but there are some considerations, we have to be aware. It leads to a circular dependency and a possibility of tight coupling, because the participants have to subscribe to all the events. Therefore it’s suitable for the simple Sagas.
Orchestration
In the case of the orchestration, a special component for coordination is used. This component is called an orchestrator and is responsible for managing the whole Saga flow. The communication is based on the asynchronous command-reply pattern. An orchestrator sends the command to the participant and awaits for the response. Based on the response, it invokes the next steps of the Saga, or a compensation transaction in case of failure. Each command triggers a local transaction on the participant’s database. When sending commands, transactional messaging should be used, to ensure reliable communication between the orchestrator and the participants.
If we compare the orchestration based diagram with the choreography based, we can see some differences. There is one additional component, the orchestrator. And the whole flow goes through this component. In this example, the event based communication is used, and the orchestrator is a standalone component. That is not necessarily the case, it can be implemented in one of the services as well. We will keep the flow simple and we will skip the command to reserve quantity as it is in the choreography based flow.
When an order is created, a request is routed to the orchestrator, which creates a command message create_order. Order service consumes this message and creates a new order in its database, and replies to the order channel in the message broker with the event order_created. This event is consumed by the orchestrator, which sends the next command, process_payment. The payment service processes the payment and replies to the same channel, from which the orchestrator consumes the event payment_processed. At this point an additional command reserve quantity could be sent as in the choreography based flow, but for simplicity, we skipped this step. In the next step, the orchestrator sends a command to the inventory channel, and the inventory service updates the quantity. When the orchestrator receives the reply, the last command will be sent, that is the command to order channel to finish the order.
Orchestration based sagas bring some benefits such as simpler dependencies which prevents a circular dependency and loose coupling where each participant is independent of others. On the other hand, we have to be careful not to implement too much business logic within the orchestrator. This leads to the design where a smart orchestrator calls dumb services and it should be avoided. The orchestrator should be responsible only for sequencing and should not contain any business logic. Orchestration is the preferred way of implementing sagas.
References:
- Richardson, Chris. Microservices Patterns. Manning Publications, 2019.
- Musib, Somnath. “Saga Pattern in Microservices” Baeldung, 2023, https://www.baeldung.com/cs/saga-pattern-microservices
- Nanayakkara, Crishantha. “Microservices Patterns: The Saga Pattern” Cloud Native Daily, 2023, https://medium.com/cloud-native-daily/microservices-patterns-part-04-saga-pattern-a7f85d8d4aa3