In this tutorial we'll see how to configure circuit breaker using Resilience4j in your Spring Boot microservices.
What is circuit breaker pattern
In terms of electronics, circuit breaker is a device that interrupts the flow of current. In case of any fault, circuit is opened interrupting the flow of current and protecting the electrical system from any damage.
In terms of microservice if two services are communicating, for example ServiceA and ServiceB. If ServiceB is down because of any problem then request from ServiceA won't get a response and there is no way to inform other requests from ServiceA about the problem downstream. So repeated requests from ServiceA will all fail resulting in failure of ServiceA too.
It may also result in a cascading effect on other services in the system and affect the overall system availability.
That's where circuit breaker pattern can be used in microservices to open the circuit in case of any problem so that further requests are not sent downstream until the problem is solved.
How does circuit breaker pattern work for microservices
With the circuit breaker configuration, you can define a threshold value for the circuit to open. When the service trying to communicate to another service fails more times than the threshold value, circuit opens for a given time period. During this specific time period (which is also configurable) requests are not sent to the failing service.
Once the time duration ends circuit breaker transitions to half-open state during which limited number of calls are sent to the failing service to check whether it is working fine now or problem is still there. If requests are processed, services start working normally otherwise circuit again goes to open state for the given time period.
Resilience4J Library
Resilience4j is a lightweight fault tolerance library providing the following functionalities-
- Circuit Breaker- To prevent cascading failures and improve fault tolerance.
- Rate Limiter- To limit the number of requests
- Time Limiter- To limit the time service waits for an operation to complete.
- Bulkhead- To limit the number of concurrent executions.
- Retry- To manage retry for failed operations.
In Resilience4J, CircuitBreaker is implemented via a finite state machine with three normal states:
- CLOSED- This is the initial state when both services are running and interaction between services is smooth. When in closed state, Circuit breaker keeps track of the requests and count of failed requests.
- OPEN- If the count of failed requests exceeds the threshold value circuit breaker changes to open state. During open state communication between microservices is blocked and the circuit breaker throws a CallNotPermittedException for any request to the failing service.
- HALF_OPEN- After the configured timeout period, circuit breaker transitions to half_open state. In this state a limited number of requests to the failing service are allowed, if the requests are successful then the circuit breaker goes to closed state otherwise it goes back to open state blocking any requests for the specific time period.
- DISABLED
- FORCED_OPEN
Spring Boot Microservice - Resilience4J Circuit breaker example
In this microservice circuit breaker example we'll use the example shown in this post- Spring Boot Microservice - Service Registration and Discovery With Eureka as our base example and add the Resilience4J circuit breaker related configurations.
In the example we'll have two services CustomerService and AccountService to cater to customer and account related functionality respectively. When customer information is retrieved it should also get the accounts associated with the specific customer. For getting the associated accounts for a customer we'll have to make a call from CustomerService to AccountService.
Adding maven dependencies for Resilience4J
There are two starters for the Resilience4J implementations, one for reactive applications and one for non-reactive applications.
org.springframework.cloud:spring-cloud-starter-circuitbreaker-resilience4j - non-reactive applications
org.springframework.cloud:spring-cloud-starter-circuitbreaker-reactor-resilience4j - reactive applications
Here I have added the reactive one. Dependency for actuator is also added so that we can check the status of service later.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>
Resilience4J circuit breaker configuration
You can configure your CircuitBreaker instance in Spring Boot's application.yml config file. For our example this will be done in the application.yml of CustomerService.
server:
port: 8081
spring:
datasource:
url: jdbc:mysql://localhost:3306/customer
username: root
password: admin
application:
name: customer-service
jpa:
properties:
hibernate:
sqldialect: org.hibernate.dialect.MySQLDialect
showsql: true
eureka:
client:
serviceUrl:
defaultZone: http://localhost:8761/eureka
management:
endpoint:
health:
show-details: always
endpoints:
web:
exposure:
include: health
health:
circuitbreakers:
enabled: true
resilience4j:
circuitbreaker:
configs:
default:
register-health-indicator: true
failure-rate-threshold: 60
minimum-number-of-calls: 5
automatic-transition-from-open-to-half-open-enabled: true
wait-duration-in-open-state: 5s
permitted-number-of-calls-in-half-open-state: 3
sliding-window-size: 10
sliding-window-type: COUNT_BASED
instances:
customer-service:
base-config: default
As you can see changes in the existing application.yml are the configuration for the actuator and for the Resilience4J circuit breaker.
Spring Boot Actuator health information can be used to check the status of your running application. By default, the CircuitBreaker health indicators are disabled, but you can enable them via the configuration.
health:
circuitbreakers:
enabled: true
You can provide the default circuit breaker configuration and later associate it with service instances.
instances:
customer-service:
base-config: default
That way you can override default configuration if needed. For example
instances:
customer-service:
base-config: default
waitDurationInOpenState: 10s
Explanation for the configured properties
failureRateThreshold- Configures the failure rate threshold in percentage. When the failure rate is equal or greater than the threshold the CircuitBreaker transitions to open and starts short-circuiting calls. Default is 50.
slidingWindowSize- Configures the size of the sliding window to record the result of the calls. For example, if size is 10 it will keep the record of 10 latest calls.
slidingWindowType- Configures the type of the sliding window. It can be COUNT_BASED or TIME_BASED. If slidingWindowSize is 10 and the slidingWindowType is COUNT_BASED then the last 10 calls are recorded and aggregated. If the type is TIME_BASED the the calls of the last 10 seconds are recorded and aggregated.
minimumNumberOfCalls- Configures the minimum number of calls which are required before the CircuitBreaker can calculate
the error rate or slow call rate. For example, if minimumNumberOfCalls is 5, then at least 5 calls must be recorded, before
the failure rate can be calculated.
If 3 out of those 5 calls fail then the failureRateThreshold of 60% is hit and the circuit is opened.
automaticTransitionFromOpenToHalfOpenEnabled- If set to true it means that the CircuitBreaker will automatically transition from open to half-open state and no call is needed to trigger the transition.
waitDurationInOpenState- The time that the CircuitBreaker should wait before transitioning from open to half-open.
permittedNumberOfCallsInHalfOpenState- Configures the number of permitted calls when the CircuitBreaker is half open.
Adding @CircuitBreaker annotation
We need to add @CircuitBreaker annotation with the service name and the fallback method name.
If a fallback method is configured, every exception is forwarded to a fallback method executor. Fallback method should be placed in the same class where you have the real method and must have the same method signature as the real method with just one extra target exception parameter. Here is the changed CustomerServiceImpl class.
import io.github.resilience4j.circuitbreaker.annotation.CircuitBreaker;
@Service
public class CustomerServiceImpl implements CustomerService{
private final CustomerRepository customerRepository;
private final RestTemplate restTemplate;
CustomerServiceImpl(CustomerRepository customerRepository, RestTemplate restTemplate){
this.customerRepository = customerRepository;
this.restTemplate = restTemplate;
}
@Override
public Customer saveCustomer(Customer customer) {
return customerRepository.save(customer);
}
@Override
@CircuitBreaker(name="customer-service", fallbackMethod = "getDefaultCustomer")
public CustomerDto getCustomerById(Long id) {
Customer customer = customerRepository.findById(id)
.orElseThrow(()-> new CustomerServiceException("No customer found for the given id - " + id));
ResponseEntity<List<AccountDto>> response = restTemplate.exchange("http://ACCOUNT-SERVICE/account/customer/"+id, HttpMethod.GET, null,
new ParameterizedTypeReference<List<AccountDto>>(){});
List<AccountDto> accounts = response.getBody();
CustomerDto customerDto = CustomerDto.builder()
.id(customer.getId())
.name(customer.getName())
.age(customer.getAge())
.city(customer.getCity())
.accounts(accounts)
.build();
return customerDto;
}
// Fallback method
public CustomerDto getDefaultCustomer(Long id, Exception e){
//log the error properly
System.out.println("Error... " + e.getMessage());
Customer customer = customerRepository.findById(id)
.orElseThrow(()-> new CustomerServiceException("No customer found for the given id - " + id));
CustomerDto customerDto = CustomerDto.builder()
.id(customer.getId())
.name(customer.getName())
.age(customer.getAge())
.city(customer.getCity())
.accounts(new ArrayList<AccountDto>())
.build();
return customerDto;
}
}
Note that in the fallback method accounts property in CustomerDTO is an empty list. As this fallback method is called when AccountService is failing so account information is kept as an empty list.
Test the circuit breaker
Start the eureka-discovery-service, customer-service and account-service. If I try to access URL http://localhost:8081/customer/1 I should get the customer information along with the associated accounts.
You can also check the health metrics by accessing http://localhost:8081/actuator/health
status "UP"
components
circuitBreakers
status "UP"
details
customer-service
status "UP"
details
failureRate "-1.0%"
failureRateThreshold "60.0%"
slowCallRate "-1.0%"
slowCallRateThreshold "100.0%"
bufferedCalls 1
slowCalls 0
slowFailedCalls 0
failedCalls 0
notPermittedCalls 0
state "CLOSED"
As you can see this is the scenario where both services are running, interaction between them is happening and the circuit has the state "CLOSED". If you click the URL http://localhost:8081/customer/1 again you can see that the bufferedCalls changes to 2. Which means circuit breaker is keeping track of the records.
Now let's shut down the account-service so it is not available any more and then notice the health of the system. If you access the URL http://localhost:8081/customer/1 now the result you get is as given below. Only customer details are there for accounts an empty list is returned. This is through the fallback method.
id 1 name "Ram" age 25 city "New Delhi" accounts []
In the logs of customer-service you should get this line-
Error... I/O error on GET request for "http://ACCOUNT-SERVICE/account/customer/1": Connection refused: connect
If you go to the URL http://localhost:8081/actuator/health, you will notice that circuit is still closed as the threshold value is not met yet. Click the URL http://localhost:8081/customer/1 two more times so that it fails 3 times out of total 5 calls. That makes the failed call percentage as 60 which is equal to the threshold value. This opens the circuit.
At this time if you access the URL http://localhost:8081/actuator/health, you should get a result similar to as given below.
status "UP"
components
circuitBreakers
status "UNKNOWN"
details
customer-service
status "CIRCUIT_OPEN"
details
failureRate "60.0%"
failureRateThreshold "60.0%"
slowCallRate "0.0%"
slowCallRateThreshold "100.0%"
bufferedCalls 5
slowCalls 0
slowFailedCalls 0
failedCalls 3
notPermittedCalls 0
state "OPEN"
If you go to the logs of customer-service there also you should see the message.
Error... CircuitBreaker 'customer-service' is OPEN and does not permit further calls
If you wait for 5 seconds, circuit breaker should automatically transition to HALF_OPEN state as per the given configuration.
status "UP"
components
circuitBreakers
status "UNKNOWN"
details
customer-service
status "CIRCUIT_HALF_OPEN"
details
failureRate "-1.0%"
failureRateThreshold "60.0%"
slowCallRate "-1.0%"
slowCallRateThreshold "100.0%"
bufferedCalls 0
slowCalls 0
slowFailedCalls 0
failedCalls 0
notPermittedCalls 0
state "HALF_OPEN"
If you start the account-service again and click the URL http://localhost:8081/customer/1 three times which is equal to the value given for the property "permitted-number-of-calls-in-half-open-state" then circuit breaker should change to "CLOSED" state since all the calls will work.
If you don't start the account-service and click the URL http://localhost:8081/customer/1 three times when circuit breaker is in "HALF_OPEN" state, it will again change to "OPEN" state as all the calls will fail.
That's all for this topic Spring Boot Microservice Circuit Breaker Using Resilience4j. If you have any doubt or any suggestions to make please drop a comment. Thanks!
>>>Return to Spring Tutorial Page
Related Topics
You may also like-
No comments:
Post a Comment