Web Analytics Made
Easy - StatCounter

Menu

About us Contact us

Why would I consider SAGA on my next microservices project

Tech 2019 / 09 / 30

Why would I consider SAGA on my next microservices project



I used to think that I can design any microservices application using either Database Per Service or Shared Database pattern.
I was wrong, of course.
When I came to know about SAGA, I realized what problems I could get into and when my application could fail with other patterns.

In this post I will try to answer 2 simple questions about SAGA.

What is SAGA?

A microservices architectural pattern, which provides a solution to implement business transactions which need to span multiple microservices.

Why would I use SAGA?

Well, before answering that question I want to describe some problem scenarios which will explain why SAGA is even needed.

・・・

When existing solution is not enough

Assume we want to build an e-commerce application. High-level microservices architecture of the system might be like this-

E-commerce architecture

 

Let’s check how we can efficiently implement transactions in our microservices system.
We will check if we can use Database per Service or Shared database pattern, and what are the limitations.
Then will explore SAGA pattern, and how it can overcome those limitations.

Database per Service

Database per Service is a widely used and straight forward pattern to design microservices applications.
In this approach, each service has its own database. That means other microservices within the system can not access them directly. Data transaction between services can only occur via some well defined APIs.

Database per service pattern

 

This pattern introduces a limitation to the system- How to implement a business transaction that span multiple microservices?
Usually microservices need data from other services to implement logic. This can create very complex and hard to maintain interactions.

For example, let’s assume in our e-commerce system orders have a stock limit.
It will be difficult to make sure that an order will not exceed the stock limit. As Order and Stock are in different databases, the application cannot simply use a local ACID transaction.

So it becomes difficult to maintain data consistency of the entire system as a whole with Database per service pattern.

Shared database

Shared database pattern involves using a single monolithic database rather than using multiple service specific database.
This pattern tries to solve the same challenge by implementing a comparatively lenient way which is- using single database and providing all services access to it.

As developers can work in existing ways and ACID transactions can be used to enforce consistency, Shared database is considered safer for developers.

Shared database pattern

 

You already have noticed the problem of Shared database pattern right?

What is the use of microservices if we are forced to implement a single DB? We can not leverage most of the benefits of microservices architecture.
Database or table structure change has effects on multiple services. So developers in different teams need to coordinate to change schema.

If even we have a strong and organized team, and can handle these problems with some extra work, the real problem occurs when multiple services are trying to access the same database resource at the same time. Run-time error occurs!

・・・

What is Saga Pattern

We can solve our above mentioned transaction design problems by implementing SAGA.

SAGA is a sequence of local transactions, where the first transaction is initiated by an external source. And each local transaction-

updates data within that particular service.

publishes a message or event.

the event triggers next local transaction.

High level SAGA design would be something like this-

We can implement SAGA in a couple of ways. Two most popular of them are-

・Events/Choreography — each local transaction publishes domain events that trigger local transactions in other services.

・Command/Orchestration — an orchestrator (object) tells the participants what local transactions to execute.

Events/Choreography

In this approach, a distributed transaction is created.
The first service executes the first transaction and publishes an event. Other services listen to this event, execute transactions and publish new events.

The transaction ends when the last local transaction is executed by the last service, and no new event is created or event is not listened by any services.

Let’s see the event flow in our e-commerce system-

SAGA implementation with Events/Choreography

 

  1. Order Service-
    – 
    creates a new order.
    – sets the state as pending.
    – 
    publishes an event called ORDER_CREATED_EVENT.
  2. Payment Service-
    – 
    Listens to ORDER_CREATED_EVENT.
     Charges the client.
    – Publishes an event called BILLED_ORDER_EVENT.
  3. Stock Service-
    – Listens to BILLED_ORDER_EVENT.
     Updates the stock
    – Prepares the product purchased in the order.
    – Publish event ORDER_PREPARED_EVENT.
  4. Delivery Service-
    – 
    Listens to ORDER_PREPARED_EVENT.
     Picks up and delivers the product.
    – Publishes event ORDER_DELIVERED_EVENT.
  5. Finally, Order Service-
    – 
    Listens to ORDER_DELIVERED_EVENT
     Sets the state of the order as finished.

In this case, if the state of the order needs to be tracked, Order Service could listen to all events and update its state.

Command/Orchestration

In this approach, a new service is created.
This service has the sole responsibility of telling each SAGA participant what to do and when to do it.

The SAGA orchestrator communicates with each service in a command style, and tells them which operation should be performed.

Let’s see the event flow in our e-commerce system-

SAGA implementation with Command/Orchestration

 

  1. Order Service-
    – 
    Saves a pending order.
    – Asks Order Saga Orchestrator (OSO) to start a create order transaction.
  2. OSO-
    – 
    Sends an Execute Payment command to Payment Service.
    – Payment Service
     replies with a Payment Executed message.
  3. OSO-
    – 
    Sends a Prepare Order command to Stock Service.
    – Stock Service replies with an Order Prepared message.
  4. OSO-
    – Sends a Deliver Order command to Delivery Service.
    – Delivery Service replies with an Order Delivered message.

In this case, Order Saga Orchestrator knows which flow is needed to execute a Create order transaction.
If anything fails, OSO will initiate the rollback by sending commands to the participants to undo the previous operation.

・・・

Rollbacks in SAGA

Rollback is needed if a transaction fails because of violating a business rule.
SAGA generally handles rollback by executing a series of transactions, which at the end revert the changes that were made by the previous local transactions.

Rollback in SAGA’s Events/Choreography design

Rollback in a distributed transaction architecture is not very easy. It requires to implement another transaction to compensate the effect of earlier transactions.

Assume that Stock Service has failed while executing a transaction. Let’s see what the rollback would look like:

Rollback in SAGA’s Events/Choreography

 

  1. Stock Service-
    – 
    Produces PRODUCT_OUT_OF_STOCK_EVENT.
  2. Both Order Service and Payment Service-
    – Listen to PRODUCT_OUT_OF_STOCK_EVENT.
  3. Payment Service-
    – 
    Refund the client.
  4. Order Service-
    – 
    Set the order state as failed.

In SAGA, it is important to assign each transaction a unique ID. So that whenever an event is created, all listeners can identify which transaction it refers to.

Rollback in SAGA’s Command/Orchestration design

Rollback in Orchestration design is lot more simpler as there will always be an orchestrator to coordinate the transactions and events.

The process will be like this-

Rollback in SAGA’s Command/Orchestration

 

  1. Stock Service-
    Replies to OSO with an Out-Of-Stock message.
  2. OSO-
    – 
    Identifies that the transaction has failed and starts the rollback.
    – Only one transaction (Execute payment) was executed successfully before the failure. So OSO sends Refund Client command to Payment Service and set the order state as failed.

SAGA can be designed to make all participants reply to a fixed address. But best implementation would be to send reply address within the message. This way we can enable participants to reply to multiple orchestrators.

Conclusion

Most important reason of using SAGA pattern is that it maintains data consistency between multiple services without coupling them tightly. This is an extremely significant feature for a microservices architecture.

But data consistency is not the only thing we have to consider while designing a system. One of the major disadvantage of SAGA is implemtation complexity from a programmer’s perspective.
SAGA requires designing compensating transactions, which may make the implementation complicated.

However, SAGA is well suited to solve some specific challenges. So we should explore it when the need arises.

Check out other articles from our engineering team:
https://medium.com/monstar-lab-bangladesh-engineering

You have ideas, We have solutions.

CONTACT US!