Event-driven Architecture (EDA) in digital transformation of enterprises
The term "business event" is no longer unfamiliar in the economy in general and in IT in particular. Examples such as a customer registering for a credit card online, a confirmed order on an e-commerce website, a customer check-in time, or a lost luggage scenario are some of the important business events depending on each industry.
Has the enterprise identified and mastered its business events, and built a suitable software architecture? What has changed in recent years is how we build this architecture and the technologies behind it, not only in on-premises and server operations, but also in Event-Driven Architecture (EDA), which can now be understood as Event-Driven SOA or SOA 2.0.
1.Looking back at the history of software architecture and the direction of enterprises
First, let's take a look back at the evolution of architecture models and technologies in the software industry. Back in 1996, Gartner described a service-oriented architecture to build entire enterprise applications created from interfaces, contracts, and calls. By 2000, the concept of REST was introduced by Roy Fielding, laying the groundwork for REST-based APIs, and a few years later, Gartner began discussing cloud computing. In 2009, Gartner identified Web-oriented architecture created by SOA + WWW + REST as one of the most strategic technology trends.
In 2011, a variant of SOA was used to describe a new architectural style: Microservices. In the following years, as more customers shifted to the cloud, new cloud-oriented architectural models were adopted. With technologies based on Containers, Microservices, and high-performing event brokers, event-driven architecture began to be applied and discussed, no longer running solely on-premises but on the cloud with cloud-native technologies.
2. The evolving demands of the digital enterprise
The rapid development is not only in software architecture but also in the demands of businesses. Looking back at what has happened in industries that only applied on-premises solutions, it is easy to see that predicting the expected load of the business was feasible: in finance, stock feed data remained constant, and in manufacturing, sensor data was also a fixed stream. When the load demand increased, businesses typically just added more computational power or memory to scale vertically with more hardware requiring maintenance.
Most applications are built under the monolithic architecture, following the three-tier architecture model. Its drawbacks include difficulties in maintenance, becoming cumbersome and heavy when adding new features. In the long run, deploying new applications on such complex application systems will prolong startup times, hinder service delivery, and diminish service quality commitments. Additionally, modern applications today use different languages such as Java, Scala, Groovy, etc., which face challenges in adapting to software frameworks and platforms due to the limitations of these monoliths. When changes in load occur, the monolithic architecture requires investment in server hardware, storage, and personnel involved in maintenance. However, the monolithic architecture still optimizes small applications, with little impact on business factors and fewer users. One of the advantages of the monolithic architecture at this time is its speed, as it is less affected by network latency.
But modern-era business models like Netflix cannot apply the monolithic architecture. When transitioning to the cloud, businesses gain access to a larger customer base, which also means that the load is no longer easily predictable. Furthermore, customers now expect services to be available 24x7x365. This is most evident in the Retail and E-commerce industries. An online retail store needs to operate at night, typically with fewer resources. However, for a global company, the concept of nighttime no longer exists; the business needs to operate 24 hours a day. This leads to the need to optimize resources and scale horizontally depending on the load to meet customer requirements and enhance customer experience.
In short, cloud technology has completely changed the game, and a suitable software architecture is crucial.
3.Has the enterprise mastered its "Business Moments"?
Gartner refers to "Business Moments" as a focal point of business operations. These "business moments" can be exemplified by system failures within a few minutes, or a loyal customer present in your store or on your e-commerce website. Functionally, business events address three main business functions: operational visibility, track and trace, and business rules.
In banking, business events address challenges such as increasing response rates for marketing offers, building cross-selling rules between the bank and card products based on real-time transaction data, or technology responsiveness to business challenges. In aviation, business events are situations such as requests for comprehensive identification of issues regarding delayed flights, weather impacts, operational conflicts like lost luggage, and real-time data management requirements. In logistics, businesses need solutions to enhance supply chain and coordination between delivery and recipient personnel in processes running on distributed systems, to create value and increase competitiveness.
These moments are fleeting opportunities that need to be seized – the faster the response to the "event," the greater the value obtained. Essentially, events are asynchronous in nature. The processing of these asynchronous events can be built based on the concept of message brokers – a concept that is not new in Service-Oriented Architecture (SOA).
In essence, event-driven architecture (EDA) emphasizes the production, detection, consumption, and reaction to events, enabling rapid responses and real-time decision-making in business operations.
The first step in building an event-driven architecture is breaking down monoliths into smaller, highly specialized services to achieve the necessary flexibility and scalability:
Highly scalable: Microservices can scale horizontally, with certain services identified for development and deployment across multiple servers and infrastructure.
Resilient: Developed services are independent and do not affect other services. If one service fails, the entire system is not affected.
Easy to deploy: Microservice-based apps are easily deployable.
Faster time-to-market: Accelerated time to market due to shorter development cycles, supporting agile deployment.
Other essential components for building an event-driven architecture include:
- Service Discovery
- Observability: Ability to observe metrics, monitoring, distributed logging, and distributed tracing.
- Security and Access Control
- Resiliency for inter-service communications: Circuit-breaking, retries & timeouts, fault injection, fault handling, load balancing & failover.
However, microservices alone are not enough to build an event-driven architecture if the application still relies on the request-reply method to exchange messages. In this case, an intermediary is needed to help services send messages to a queue, and another service listens and receives messages. Apache Kafka is chosen to fulfill this task.
By identifying the two key technology factors as Microservices and Apache Kafka, the cloud-native event-driven architecture is described as above. Microservices not only perform the task of publishing events to the broker but also integrate with databases, third-party software service providers, on-premises systems, or deploy other scenarios. This software architecture combines the benefits of Microservices and event brokers when applied together with CQRS, independently scalable and responsive to the creative and innovative needs of future technology.
4. Use Cases & Success Stories
The manufacturing industry needs to cover various areas such as managing production plants, procurement departments needing to interact with suppliers, logistics departments for transporting goods, etc. During the production process, a large amount of data - events are generated. For essential events, managers need to take immediate action to avoid quality degradation or waste. To be able to exchange data with all relevant parties, businesses may need to integrate on-premises systems with software providers as a service (SaaS providers). With EDA, multiple microservices can be used for one sensor to increase data processing productivity and avoid data loss. With EDA, manufacturing businesses can successfully connect on-premises, public cloud, and private cloud services, while reducing marketing time to integrate new services, such as in cases of acquisitions or replacing existing systems.
The retail banking sector used to be one of the last industries to transition to online operations, but today the concept of traditional banking belongs to the past. Digital banking services help customers interact with banks in many different ways. Each individual activity can be modeled as an event, and EDA is the perfect solution to handle these activities as well as to protect against fraud. EDA also enables reducing time to market for launching products to execute marketing campaigns and to test fraud scenarios.
The online gambling and casino industry is an area with an explosion of data that needs to be processed in milliseconds due to last-minute bets. This means that services need to have the ability to scale instantly and often span across multiple systems to update odds. The ability to adopt new emerging technologies and fast time-to-market is the key to making a difference in highly competitive fields like gaming.
Retail is certainly an industry that has witnessed the importance of online customers. Most retailers strive to meet the demand for home delivery, but very few retailers actually do it well. EDA can support scaling and increase agility in updating new services. Additionally, EDA helps deploy real-time marketing and increase customer satisfaction with "at the right time, in the right place" offers. EDA is a supportive architecture for all sales plans and integration requirements.
Conclusion:
In a period of significant economic volatility, businesses in any industry are facing fierce competition, constantly changing customer needs and expectations, along with increasingly stringent legal frameworks. To solve the problem of prioritizing creative innovation in technology to meet business directions, the question is whether businesses, with their current resources, are ready for those changes?
ENAO is a priority partner of TIBCO in Vietnam, with many years of experience in software development and providing outsourcing services, consulting, and deploying solutions using advanced technical solutions, helping businesses solve complex issues that arise in the digital transformation journey.