AWS ‐ Integration Services ‐ EventBridge | Simple Queue Service (SQS) | Message Queue (MQ) | Simple Notification Service (SNS) | AWS Step Function | Simple Workflow (SWF) Service
-
This architecture comprises multiple decoupled micro services
-
Events refer to a change in state or update
-
Events trigger communication between services
-
Events can carry the state or act as messengers as notifications
-
Event Producers
-
Event Routers
-
Event Consumers
Event producer apps and consumer apps are decoupled, this enables the apps to independently deployed, update, scale.
- Independent scale and fail - no single point of failure
- As it is support decoupled architecture, apps/services can be built in varieties of tech stack in contrast with monolith arch
- Understand the resiliency, scaling aspects to accommodate the demand
- Objective is to build resiliency into workloads
- How do we Build resiliency in our applications
- Evaluate scaling resources available to accommodate customer demand
- Goal is to build resiliency in the application itself to handle the workloads, failures
The system should be designed and built to identify the failures and self heal
-
Managing Failures
- Transaction Failure - happens when a component is not available due to heavy load
- Partial Failure - happens when a batch processing being performed (increases load on resources, hence resource unavailable to serve)
- Intermittent Failure - happens when network failure
-
How do we deal with failures?
- Retry until succeed
- Improves reliability
- Reduces operational cost
-
Backoff: A better approach by enabling clients wait before retries
- As retrying is not always effective, when a system is already failing due to overload, then the retries will increase the system load
- Exponential backoff is most common approach to make clients wait for larger intervals exponentially before the retry
- Maximum delays and no.of retries are normally configured in this approach
-
Jitter - when Backoff is not enough
- In backoff approach, when multiple retries from all backoff systems simultaneously, there is a chance that requests will burst into the system, this will overburden the system, Jitter system modifies the timings of retries by adding some randomness (delaying requests) to the timing of backoff
- This can be used to reduce the impact of huge burst into the system
-
Dealing with Failed Messages (message queue) - below options provide ways to handle messages cannot be processed successfully by a system
- Failures Destinations - is an alternate destination for messages that could not be processed, separate event destinations can be configured for failure messages
- Dead-letter queues (DLQ) - It is a specific failure destination to store the messages that cannot be processed by a system due to an error or lack of resources, these messages can be inspected and either retried or discarded.
- AWS Step Functions - Orchestrating long-running transactions - Where series of transaction steps can be configured with error handling and retry capabilities
- Distributed Transactions - AWS step functions help focusing on business logic instead of orchestration logic
Essential for component to component communication in a serverless event-driven application
- Event indicates that something in the environment is changed, most of services in aws (cloudformation, ec2, etc) generates events
- It is a pipeline for receiving events
- Events that are coming to event bridge, are linked to a single event bus only, where the events are evaluated based on rules defined with the event bus, which means the rules are applied to events on the event bus it associated with.
- Uses Resource-based policy to manage permissions for event bus
- Maximum of 300 rules can be configured per event bus
- Custom event bus can also be created for custom applications, also applies with max 300 rules to be configured.
A Rule matches incoming events and sends events to one or more targets to process
- Multiple targets run in parallel
- Example target: A lambda function, SNS Topic, SQS Queue etc
- Indicates something is changed in the system (ex: ec2 scale out/in, clouformation etc)
- Scheduled Events
- Events are represented as Objects in JSON Format
- Target is a Resource or endpoint to route the events to
- Permission is required to send the data to the target resource and access the target resource
- Maximum 5 targets per rule can be created
- Delay with new targets, brief delay should be given propagate the changes to new targets
It is an event structure definition for the event
